Mining Semantic Correlation of Heterogeneous Multimedia Data for Cross-Media Retrieval
Mining Semantic Correlation of Heterogeneous Multimedia Data for Cross-Media Retrieval
复制标题
DOI：
10.1109/tmm.2007.911822
复制
发表时间：
2008-02
影响因子：
7.3
通讯作者：
Yueting Zhuang;Yi Yang;Fei Wu
中科院分区：
文献类型：
--
作者：
Yueting Zhuang;Yi Yang;Fei Wu
虽然图像、音频和文本等多媒体对象具有不同的模态，但它们之间存在着大量的语义关联。在本文中，我们提出了一种换向学习的方法来挖掘不同模态媒体对象之间的语义相关性，从而实现跨媒体检索。跨媒体检索是一种新的检索技术，通过该技术，查询示例和返回结果可以是不同的形式，例如通过音频示例查询图像。首先，根据媒体对象的特征及其共存信息，构建统一的跨媒体关联图，统一表示不同形态的媒体对象；为了执行跨媒体检索，将给查询示例分配一个正分数；分数沿着图形扩散，并返回得分最高的目标模态或mmd的媒体对象。为了提高检索性能，我们还提出了长期和短期相关反馈的不同方法来挖掘正例和反例中包含的信息。
Although multimedia objects such as images, audios and texts are of different modalities, there are a great amount of semantic correlations among them. In this paper, we propose a method of transductive learning to mine the semantic correlations among media objects of different modalities so that to achieve the cross-media retrieval. Cross-media retrieval is a new kind of searching technology by which the query examples and the returned results can be of different modalities, e.g., to query images by an example of audio. First, according to the media objects features and their co-existence information, we construct a uniform cross-media correlation graph, in which media objects of different modalities are represented uniformly. To perform the cross-media retrieval, a positive score is assigned to the query example; the score spreads along the graph and media objects of target modality or MMDs with the highest scores are returned. To boost the retrieval performance, we also propose different approaches of long-term and short-term relevance feedback to mine the information contained in the positive and negative examples.