Cross-media retrieval with collective deep semantic learning
Cross-media retrieval with collective deep semantic learning
复制标题
集体深度语义学习的跨媒体检索
DOI：
10.1007/s11042-018-5896-6
复制
发表时间：
2018-09-01
影响因子：
3.6
通讯作者：
Zhang, Huaxiang
中科院分区：
文献类型：
--
作者：
Zhang, Bin;Zhu, Lei;Zhang, Huaxiang
跨媒体检索是信息检索技术发展的新趋势。它受到了学术界和工业界的极大关注。在本文中，我们提出了一种有效的检索方法，称为跨媒体检索与集体深度语义学习（CR-CDSL），来解决这个问题。首先学习两个互补的深度神经网络，将图像和文本样本共同投影到联合语义表示中。在此基础上，为未标注的图像和文本生成相应的弱语义标签。利用预标注的训练样本对检索模型进行再训练，发现具有区分性的共享语义空间，实现跨媒体检索。具体而言，采用深度限制玻尔兹曼机（DRBM）来初始化两个深度神经网络的权重。通过集体深度语义学习产生的弱标签，可以增强检索模型的区分能力。从而提高了模型的检索性能。在几个公开的跨媒体数据集上对实验进行了评估。实验结果表明，该方法与现有的几种技术相比具有上级性能。
Cross-media retrieval is becoming a new trend of information retrieval technique. It has been received great attentions from both academia and industry. In this paper, we propose an effective retrieval method, dubbed as Cross-media Retrieval with Collective Deep Semantic Learning (CR-CDSL), to solve the problem. Two complementary deep neural networks are first learned to collectively project image and text samples into a joint semantic representation. Based on it, weak semantic labels are then generated accordingly for unlabeled images and texts. They are exploited further with the pre-labeled training samples to retrain the retrieval model, which can discover a discriminative shared semantic space for achieving cross-media retrieval. Specifically, Deep Restricted Boltzmann Machines (DRBM) is employed to initialize the weights of two deep neural networks. With the weak labels generated from collective deep semantic learning, the discriminative capability of retrieval model can be enhanced. Thus, the retrieval performance of the model could be improved. Experiments are evaluated on several publicly available cross-media datasets. The obtained experimental results demonstrate the superior performance of the proposed approach compared with several state-of-the-art techniques.