Sparse Relational Topical Coding on multi-modal data
Sparse Relational Topical Coding on multi-modal data
复制标题
DOI：
10.1016/j.patcog.2017.08.005
复制
发表时间：
2017-12
期刊：
影响因子：
--
通讯作者：
Lingyun Song;Jun Liu;Minnan Luo;B. Qian;Kuan Yang
中科院分区：
文献类型：
--
作者：
Lingyun Song;Jun Liu;Minnan Luo;B. Qian;Kuan Yang
多模态数据建模是近年来模式识别领域的一个研究热点。现有的研究主要集中在对多模态文档的内容建模，而文档之间的链接往往被忽略。然而，链接信息已经显示出在许多应用中的关键重要性，例如文档导航、分类和聚类。在本文中，我们提出了一个非概率公式的关系主题模型（RTM），即，稀疏关系多模态主题编码（SRMMTC），用于对多模态文档和相应的链接信息进行建模。SRMMTC具有以下三个吸引人的特性：i）它可以通过直接施加稀疏诱导正则化器来有效地产生稀疏潜在表示。ii）它通过分别为正链接和负链接引入正则化参数来处理多模态数据收集上的不平衡问题; iii）它可以通过有效的坐标下降算法来解决。我们还探讨了一个广义版本的SRMMTC，以找到主题之间的相互作用。我们的方法也能够执行链接预测的文件，以及预测的注释文字的陪同人员在文件中的图像。一组基准数据集的实证研究表明，我们提出的模型显着优于许多国家的最先进的方法。
Multi-modal data modeling lately has been an active research area in pattern recognition community. Existing studies mainly focus on modeling the content of multi-modal documents, whilst the links amongst documents are commonly ignored. However, link information has shown being of key importance in many applications, such as document navigation, classification, and clustering. In this paper, we present a non-probabilistic formulation of Relational Topic Model (RTM), i.e., Sparse Relational Multi-Modal Topical Coding (SRMMTC), to model both multi-modal documents and the corresponding link information. SRMMTC has the following three appealing properties: i) It can effectively produce sparse latent representations via directly imposing sparsity-inducing regularizers. ii) It handles the imbalance issues on multi-modal data collections by introducing regularization parameters for positive and negative links, respectively; iii) It can be solved by an efficient coordinate descent algorithm. We also explore a generalized version of SRMMTC to find pairwise interactions amongst topics. Our methods are also capable of performing link prediction for documents, as well as the prediction of annotation words for attendant images in documents. Empirical studies on a set of benchmark datasets show that our proposed models significantly outperform many state-of-the-art methods.