Cross-media retrieval by intra-media and inter-media correlation mining
Cross-media retrieval by intra-media and inter-media correlation mining
复制标题
通过媒体内和媒体间相关性挖掘进行跨媒体检索
DOI：
10.1007/s00530-012-0297-6
复制
发表时间：
2013-10
期刊：
影响因子：
--
通讯作者：
Xiao, Jianguo
中科院分区：
文献类型：
--
作者：
Zhai, Xiaohua;Peng, Yuxin;Xiao, Jianguo
随着互联网上多媒体内容的迅速发展，跨媒体检索已成为研究和应用的关键问题。跨媒体检索能够检索与查询语义相同但具有不同媒体类型的结果。例如，给定Moraine Lake的查询图像，跨媒体检索系统除了检索与Moraine Lake相关的图像外，还可以检索不同媒体类型的相关媒体内容，如文本描述。因此，测量不同媒体之间的内容相似性是一个具有挑战性的问题。在本文中，我们提出了一种新的跨媒体相似性度量。它考虑了媒体内和媒体间的相关性，这是被现有的作品忽略。媒体内相关性主要关注每种媒体内部的语义类别信息，而媒体间相关性主要关注不同媒体类型之间的正相关和负相关。两者都是非常重要的，它们的自适应融合可以相辅相成。为了挖掘媒体内的相关性，我们提出了一个异构的相似性度量与最近的邻居（HSNN）。通过计算两个媒体对象属于同一语义类别的概率来获得异质相似性。为了挖掘媒体间的相关性，我们提出了一个跨媒体相关传播（CMCP）的方法，同时处理不同媒体类型的媒体对象之间的正相关性和负相关性，而现有的作品只专注于正相关性。负相关非常重要，因为它提供了有效的排他性信息。相关性被建模为必须链接的约束和不能链接的约束，分别。此外，我们的方法是能够传播异质模态之间的相关性。最后，HSNN和CMCP都是灵活的，因此可以合并任何传统的相似性度量。通过AdaRank进一步融合多种相似性度量，学习出一种有效的跨媒体检索排序模型。两个数据集上的实验结果表明，我们提出的方法的有效性，与国家的最先进的方法相比。
With the rapid development of multimedia content on the Internet, cross-media retrieval has become a key problem in both research and application. Cross-media retrieval is able to retrieve the results of the same semantics with the query, but with different media types. For instance, given a query image of Moraine Lake, besides retrieving the images about Moraine Lake, cross-media retrieval system can also retrieve the related media contents of different media types such as text description. As a result, measuring content similarity between different media is a challenging problem. In this paper, we propose a novel cross-media similarity measure. It considers both intra-media and inter-media correlation, which are ignored by existing works. Intra-media correlation focuses on semantic category information within each media, while inter-media correlation focuses on positive and negative correlations between different media types. Both of them are very important and their adaptive fusion can complement each other. To mine the intra-media correlation, we propose a heterogeneous similarity measure with nearest neighbors (HSNN). The heterogeneous similarity is obtained by computing the probability for two media objects belonging to the same semantic category. To mine the inter-media correlation, we propose a cross-media correlation propagation (CMCP) approach to simultaneously deal with positive and negative correlation between media objects of different media types, while existing works focus solely on the positive correlation. Negative correlation is very important because it provides effective exclusive information. The correlations are modeled as must-link constraints and cannot-link constraints, respectively. Furthermore, our approach is able to propagate the correlation between heterogeneous modalities. Finally, both HSNN and CMCP are flexible, so that any traditional similarity measure could be incorporated. An effective ranking model is learned by further fusion of multiple similarity measures through AdaRank for cross-media retrieval. The experimental results on two datasets show the effectiveness of our proposed approach, compared with state-of-the-art methods.
登录
查看更多内容
跨媒体检索：最先进的和开放的问题
DOI：
10.1504/ijmis.2010.035970
发表时间：
2010-10
期刊：
International Journal of Multimedia Intelligence and Security
影响因子：
--
作者：
Jing Liu;Changsheng Xu;Hanqing Lu
通讯作者：
Hanqing Lu
DOI：
10.1145/1390156.1390229
发表时间：
2008-07
影响因子：
1
作者：
Zhenguo Li;Jianzhuang Liu;Xiaoou Tang
通讯作者：
Zhenguo Li;Jianzhuang Liu;Xiaoou Tang
影响因子：
2.7
作者：
H. Hotelling
通讯作者：
H. Hotelling
DOI：
10.1145/1460096.1460125
发表时间：
2008-10
期刊：
--
影响因子：
--
作者：
H. Escalante;Carlos A. Hernández;L. Sucar;M. Montes-y-Gómez
通讯作者：
H. Escalante;Carlos A. Hernández;L. Sucar;M. Montes-y-Gómez
DOI：
10.1007/978-3-642-27355-1_30
发表时间：
2012-01
期刊：
--
影响因子：
--
作者：
Xiaohua Zhai;Yuxin Peng;Jianguo Xiao
通讯作者：
Xiaohua Zhai;Yuxin Peng;Jianguo Xiao