Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search
Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search
复制标题
DOI：
10.1109/tip.2012.2202676
复制
发表时间：
2013-01-01
影响因子：
10.6
通讯作者：
Wu, Xindong
中科院分区：
文献类型：
--
作者：
Gao, Yue;Wang, Meng;Wu, Xindong
由于社交媒体网站的流行，广泛的研究工作已经致力于基于标签的社交图像搜索。视觉信息和标签在研究领域都有研究。然而，大多数现有的方法使用标签和视觉特征单独或顺序，以估计图像的相关性。在本文中，我们提出了一种方法，同时利用视觉和文本信息来估计用户标记的图像的相关性。利用超图学习方法确定相关性估计。在该方法中，构建了一个社会图像超图，其中顶点表示图像，超边表示视觉或文本术语。学习是通过使用一组伪正图像来实现的，其中超边的权重在整个学习过程中进行更新。通过这种方式，可以自动调制不同标签和视觉词的影响。在包含370多幅图像的数据集上进行的实验结果表明了该方法的有效性。
Due to the popularity of social media websites, extensive research efforts have been dedicated to tag-based social image search. Both visual information and tags have been investigated in the research field. However, most existing methods use tags and visual characteristics either separately or sequentially in order to estimate the relevance of images. In this paper, we propose an approach that simultaneously utilizes both visual and textual information to estimate the relevance of user tagged images. The relevance estimation is determined with a hypergraph learning approach. In this method, a social image hypergraph is constructed, where vertices represent images and hyperedges represent visual or textual terms. Learning is achieved with use of a set of pseudo-positive images, where the weights of hyperedges are updated throughout the learning process. In this way, the impact of different tags and visual words can be automatically modulated. Comparative results of the experiments conducted on a dataset including 370+ images are presented, which demonstrate the effectiveness of the proposed approach.