Semantic Topic Multimodal Hashing for Cross-Media Retrieval
Semantic Topic Multimodal Hashing for Cross-Media Retrieval
复制标题
DOI：
--
复制
发表时间：
2015-07
期刊：
影响因子：
--
通讯作者：
Di Wang;Xinbo Gao;Xiumei Wang;Lihuo He
中科院分区：
文献类型：
--
作者：
Di Wang;Xinbo Gao;Xiumei Wang;Lihuo He
多模态散列算法以其存储开销小、查询速度快等优点成为跨媒体相似性搜索的关键技术。现有的多模态散列方法大多将异构数据嵌入到一个公共的低维Hamming空间中，然后对连续嵌入进行舍入以获得二进制码。然而，他们通常忽略了散列固有的离散性放松离散约束，这将导致检索性能下降，特别是对于长代码。为此，在编码过程中考虑潜在语义信息，提出了一种新的语义主题多模式哈希算法。该算法首先发现文本的聚类模式，然后对图像矩阵进行鲁棒分解，得到文本的多个语义主题和图像的多个概念。然后，学习多模态语义特征转换到一个共同的子空间，通过它们的相关性。最后，通过判断文本或图像中是否包含主题或概念，可以直接生成统一哈希码的每一位。因此，STMH得到的模型更适合于哈希方案，因为它直接在编码过程中学习离散哈希码。实验结果表明，该方法优于几个国家的最先进的方法。
Multimodal hashing is essential to cross-media similarity search for its low storage cost and fast query speed. Most existing multimodal hashing methods embedded heterogeneous data into a common low-dimensional Hamming space, and then rounded the continuous embeddings to obtain the binary codes. Yet they usually neglect the inherent discrete nature of hashing for relaxing the discrete constraints, which will cause degraded retrieval performance especially for long codes. For this purpose, a novel Semantic Topic Multimodal Hashing (STMH) is developed by considering latent semantic information in coding procedure. It first discovers clustering patterns of texts and robust factorizes the matrix of images to obtain multiple semantic topics of texts and concepts of images. Then the learned multimodal semantic features are transformed into a common subspace by their correlations. Finally, each bit of unified hash code can be generated directly by figuring out whether a topic or concept is contained in a text or an image. Therefore, the obtained model by STMH is more suitable for hashing scheme as it directly learns discrete hash codes in the coding process. Experimental results demonstrate that the proposed method outperforms several state-of-the-art methods.