融合多模态文本关联分析与挖掘的跨媒体社会图像检索方法研究

本课题在传统自然语言处理和信息检索基本理论的基础上，针对互联网环境下大规模社会图像检索的实际需要，侧重于研究跨媒体社会图像检索中多模态文本关联分析与挖掘关键技术，构建融合多模态文本关联分析与挖掘的跨媒体社会图像检索新颖实现框架，并由此引发针对有别于传统文本形式的多模态文本分析与处理、乃至跨媒体社会图像检索策略的深入研究，为社会图像检索用户提供一种高效、精确、易用的智能助手。其中，采取基于自动生成机制构建蕴涵丰富语义和视觉关联信息的多模态关联网络作为重要知识源，有效利用基于多重属性的概念文本项自动抽取模式、跨模态语义评估与消歧模式、跨模态语义扩展与优化模式、跨模态语义排序与过滤模式，特别是跨模态实体-对象语义关联模式、跨模态语义聚合关联模式、面向多模态文本关联分析与挖掘的联想机制等多种新型多模态文本深层次分析、处理、挖掘与联想关键技术，构成功能更加强壮且性能更加完善的跨媒体社会图像检索过程。

To satisfy the practical needs of large-scale social image retrieval under the environment of Internet, based on the basic theory of the traditional Natural Language Processing and Information Retrieval, this project focuses on researching the key techniques of multimodal text association analysis and mining in cross-media social image retrieval and constructing a novel implementation framework fusing with such multimodal text association analysis and mining techniques. Thus the in-depth exploration is triggered for the analysis and processing mechanisms of the multimodal text that is different from the traditional text, and even the cross-media social image retrieval strategies, which aims at providing an efficient, precise and easy-to-use intelligent assistant for social image retrieval users. In this project, multiple new key techniques for multimodal text in-depth analysis, processing, mining and imagination are deeply studied and developed to facilitate more effective cross-media social image retrieval with the stronger function and more perfect performance, which include automatic multimodal association network generation, automatic extraction of concept text term with multiple attributes, cross-modal semantic evaluation and disambiguation, cross-modal semantic expansion and optimization, cross-modal semantic ranking and filtering, especially cross-modal entity-object semantic association, cross-modal semantic aggregation association, and imagination oriented to multimodal text association analysis and mining, etc.