Click Prediction for Web Image Reranking Using Multimodal Sparse Coding
Click Prediction for Web Image Reranking Using Multimodal Sparse Coding
复制标题
DOI：
10.1109/tip.2014.2311377
复制
发表时间：
2014-03
影响因子：
10.6
通讯作者：
Jun Yu;Y. Rui;D. Tao
中科院分区：
文献类型：
--
作者：
Jun Yu;Y. Rui;D. Tao
图像重排序是提高基于文本的图像搜索性能的有效方法。然而，现有的重排序算法受到两个主要原因的限制：1）与图像相关联的文本元数据通常与其实际视觉内容不匹配; 2）提取的视觉特征不能准确地描述图像之间的语义相似性。最近，用户点击信息已被用于图像重新排名，因为点击已被证明可以更准确地描述检索到的图像与搜索查询的相关性。然而，基于点击的方法的一个关键问题是缺乏点击数据，因为只有少量的Web图像实际上被用户点击。因此，我们的目标是通过预测图像点击来解决这个问题。我们提出了一种基于多模态超图学习的图像点击预测稀疏编码方法，并将获得的点击数据应用于图像的重新排序。我们采用超图来构造一组流形，通过一组权值来探索不同特征之间的互补性。与两个顶点之间有一条边的图不同，超图中的超边连接一组顶点，并有助于保持所构造的稀疏码的局部光滑性。然后进行交替优化过程，并且同时获得不同模态和稀疏码的权重。最后，投票策略被用来描述预测点击作为一个二进制事件（点击或不点击），从图像的相应的稀疏代码。在一个包含近330 K图像的大规模数据库上进行的深入的实证研究表明，与其他几种方法相比，我们的方法对点击预测是有效的。在真实世界数据上的额外图像重排序实验表明，使用点击预测有利于提高基于图的图像重排序算法的性能。
Image reranking is effective for improving the performance of a text-based image search. However, existing reranking algorithms are limited for two main reasons: 1) the textual meta-data associated with images is often mismatched with their actual visual content and 2) the extracted visual features do not accurately describe the semantic similarities between images. Recently, user click information has been used in image reranking, because clicks have been shown to more accurately describe the relevance of retrieved images to search queries. However, a critical problem for click-based methods is the lack of click data, since only a small number of web images have actually been clicked on by users. Therefore, we aim to solve this problem by predicting image clicks. We propose a multimodal hypergraph learning-based sparse coding method for image click prediction, and apply the obtained click data to the reranking of images. We adopt a hypergraph to build a group of manifolds, which explore the complementarity of different features through a group of weights. Unlike a graph that has an edge between two vertices, a hyperedge in a hypergraph connects a set of vertices, and helps preserve the local smoothness of the constructed sparse codes. An alternating optimization procedure is then performed, and the weights of different modalities and the sparse codes are simultaneously obtained. Finally, a voting strategy is used to describe the predicted click as a binary event (click or no click), from the images' corresponding sparse codes. Thorough empirical studies on a large-scale database including nearly 330 K images demonstrate the effectiveness of our approach for click prediction when compared with several other methods. Additional image reranking experiments on real-world data show the use of click prediction is beneficial to improving the performance of prominent graph-based image reranking algorithms.