Multi-Pathway Generative Adversarial Hashing for Unsupervised Cross-Modal Retrieval
Multi-Pathway Generative Adversarial Hashing for Unsupervised Cross-Modal Retrieval
复制标题
DOI：
10.1109/tmm.2019.2922128
复制
发表时间：
2020-01
影响因子：
7.3
通讯作者：
Jian Zhang-;Yuxin Peng
中科院分区：
文献类型：
--
作者：
Jian Zhang-;Yuxin Peng
跨模态哈希算法将异构的跨模态数据映射到一个共同的汉明空间中，实现跨模态的快速、灵活检索。无监督跨模态哈希比监督方法更灵活和适用，因为不涉及密集的标记工作。然而，现有的无监督方法通过保留内部和内部相关性来学习散列函数，同时忽略不同模态之间的底层流形结构，这对于捕获不同模态的有意义的最近邻居以进行跨模态检索非常有帮助。此外，现有的工作主要集中在成对关系建模，而忽略了多个模态之间的相关性。为了解决上述问题，在本文中，我们提出了一种用于无监督跨模态检索的多路径生成对抗哈希方法，该方法充分利用了生成对抗网络的无监督表示学习能力，以利用跨模态数据的底层流形结构。主要贡献可以总结如下：首先，我们提出了一个多路径生成对抗网络，以无监督的方式对跨模式哈希进行建模。在所提出的网络中，给定一种模态的数据，生成模型试图拟合流形结构上的分布，并选择其他模态的信息数据来挑战判别模型。判别模型学习区分生成的数据和从相关图中采样的真阳性数据，以实现更好的检索精度。这两个模型以对抗的方式进行训练，以相互改进并促进哈希函数学习。其次，我们提出了一种基于相关图的方法来捕获跨不同模态的底层流形结构，以便不同模态但在同一流形内的数据可以具有较小的汉明距离，以提高检索精度。在三个广泛使用的数据集上与最先进的方法进行了广泛的实验，验证了我们所提出的方法的有效性。
Cross-modal hashing aims to map heterogeneous cross-modal data into a common Hamming space, which can realize fast and flexible retrieval across different modalities. Unsupervised cross-modal hashing is more flexible and applicable than supervised methods, since no intensive labeling work is involved. However, existing unsupervised methods learn the hashing functions by preserving inter- and intra-correlations while ignoring the underlying manifold structure across different modalities, which is extremely helpful in capturing the meaningful nearest neighbors of different modalities for cross-modal retrieval. Furthermore, existing works mainly focus on pairwise relation modeling while ignoring the correlations within multiple modalities. To address the above-mentioned problems, in this paper, we propose a multi-pathway generative adversarial hashing approach for unsupervised cross-modal retrieval, which makes full use of a generative adversarial network's ability for unsupervised representation learning to exploit the underlying manifold structure of cross-modal data. The main contributions can be summarized as follows: First, we propose a multi-pathway generative adversarial network to model cross-modal hashing in an unsupervised fashion. In the proposed network, given the data of one modality, the generative model tries to fit the distribution over the manifold structure and selects informative data of other modalities to challenge the discriminative model. The discriminative model learns to distinguish the generated data and the true positive data sampled from the correlation graph to achieve better retrieval accuracy. These two models are trained in an adversarial way to improve each other and promote hashing function learning. Second, we propose a correlation graph-based approach to capture the underlying manifold structure across different modalities so that data of different modalities but within the same manifold can have a smaller Hamming distance to promote retrieval accuracy. Extensive experiments compared with state-of-the-art methods on three widely used datasets verify the effectiveness of our proposed approach.