EAGER: Exploring Multimedia Information Networks

EAGER：探索多媒体信息网络

• 批准号: 1144111
• 负责人: Thomas Huang
• 金额: $ 19.94万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2013-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1144111&HistoricalAwards=false
• 关键词: EAGER; Exploring; Multimedia; Information; Networks

This project is built on the interdisciplinary collaboration between multimedia and natural language processing (NLP) researchers. The goal of this exploratory research is to provide effective methods for organizing, searching, mining and reasoning with web-scale multimedia. The approach is based on formulation of a structured multimedia database, called Multimedia Information Networks (MINets) which enables new searching and mining paradigms such as keyword query of unlabeled image/video databases, query expansion in content-based image retrieval (CBIR), and measuring similarity between different modalities such as an image and a piece of text). In addition, MINets framework is expected to provide the ability to perform robust inference (e.g., recognizing objects and activities in images or videos) in the presence of noise and uncertainty. In its simplest form, a MINet is a graph where nodes are either concepts (text) or data (such as images), and links are ontological/semantic relationships between concepts, attachment of images to concepts, and visual similarity measures between images. Construction of an experimental MINets framework involves crawling the Web for a particular domain, gathering images with associated text and exploiting natural language processing, computer vision and mining techniques in establishing the concepts, associated images, interconnecting links and an ontology that supports inference. Validation against current web search engines and CBIR techniques is expected to provide a proof-of-concept for the novel MINets framework.This interdisciplinary exploratory project is expected to yield general theoretical and algorithmic MINets framework that will provide new searching, mining and reasoning capabilities for multimedia data. It will help to define new research areas in effective utilization of multimedia information sources for cross-media and cross-conceptual knowledge discovery and analysis, large-scale annotation, information fusion and inference. Project results, including open source software, annotated corpora, scoring metrics will be disseminated via project Website (https://netfiles.uiuc.edu/qi4/www/MINets.htm). This project will provide research opportunities for graduate and undergraduate students.

这个项目是建立在多媒体和自然语言处理(NLP)研究人员之间跨学科合作的基础上的。本探索性研究的目的是为网络多媒体的组织、搜索、挖掘和推理提供有效的方法。该方法基于结构化多媒体数据库的形成，称为多媒体信息网络(MINETS)，它支持新的搜索和挖掘范例，例如对未标记的图像/视频数据库进行关键字查询，在基于内容的图像检索(CBIR)中进行查询扩展，以及测量不同形式(如图像和文本片段)之间的相似性。此外，预期MINETS框架能够在存在噪声和不确定性的情况下进行稳健的推理(例如，识别图像或视频中的目标和活动)。在最简单的形式中，Minet是一个图，其中节点是概念(文本)或数据(如图像)，链接是概念之间的本体/语义关系、图像到概念的附件以及图像之间的视觉相似性度量。构建实验性MINETS框架涉及为特定领域爬行Web，收集具有关联文本的图像，并利用自然语言处理、计算机视觉和挖掘技术来建立概念、关联图像、互连链接和支持推理的本体。针对当前网络搜索引擎和CBIR技术的验证有望为新的MINETS框架提供概念验证。这一跨学科的探索性项目有望产生通用的理论和算法MINETS框架，将为多媒体数据提供新的搜索、挖掘和推理能力。这将有助于在有效利用多媒体信息源进行跨媒体和跨概念的知识发现和分析、大规模标注、信息融合和推理方面开辟新的研究领域。项目结果，包括开放源码软件、带注释的语料库、评分指标将通过项目网站(https://netfiles.uiuc.edu/qi4/www/MINets.htm).发布该项目将为研究生和本科生提供研究机会。