Weakly Supervised Deep Matrix Factorization for Social Image Understanding
Weakly Supervised Deep Matrix Factorization for Social Image Understanding
复制标题
DOI：
10.1109/tip.2016.2624140
复制
发表时间：
2017
影响因子：
10.6
通讯作者：
Zechao Li;Jinhui Tang
中科院分区：
文献类型：
--
作者：
Zechao Li;Jinhui Tang
近年来，与弱监督的用户提供的标签相关联的图像的数量急剧增加。用户提供的标签是不完整的，主观的和嘈杂的。在本文中，我们专注于社会形象理解的问题，即，标签细化、标签分配和图像检索。与以前的工作不同，我们提出了一种新的弱监督深度矩阵分解算法，通过协同探索弱监督标记信息，视觉结构和语义结构，揭示了隐藏在潜在子空间中的潜在图像表示和标签表示。由于众所周知的语义差距，隐藏的图像表示学习的层次模型，这是逐步从视觉特征空间转换。它可以使用学习的深度架构自然地将新图像嵌入子空间。语义和视觉结构被联合合并以学习语义子空间，而不会过度拟合嘈杂的、不完整的或主观的标签。此外，为了去除噪声或冗余的视觉特征，在深层架构中的第一层的变换矩阵上施加稀疏模型。最后，一个统一的优化问题与一个定义良好的目标函数，制定提出的问题，并解决了曲线搜索的梯度下降过程。在现实世界的社会图像数据库上进行了广泛的实验图像理解的任务：图像标签细化，分配和检索。通过与现有算法的比较，取得了令人鼓舞的结果，证明了该方法的有效性。
The number of images associated with weakly supervised user-provided tags has increased dramatically in recent years. User-provided tags are incomplete, subjective and noisy. In this paper, we focus on the problem of social image understanding, i.e., tag refinement, tag assignment, and image retrieval. Different from previous work, we propose a novel weakly supervised deep matrix factorization algorithm, which uncovers the latent image representations and tag representations embedded in the latent subspace by collaboratively exploring the weakly supervised tagging information, the visual structure, and the semantic structure. Due to the well-known semantic gap, the hidden representations of images are learned by a hierarchical model, which are progressively transformed from the visual feature space. It can naturally embed new images into the subspace using the learned deep architecture. The semantic and visual structures are jointly incorporated to learn a semantic subspace without overfitting the noisy, incomplete, or subjective tags. Besides, to remove the noisy or redundant visual features, a sparse model is imposed on the transformation matrix of the first layer in the deep architecture. Finally, a unified optimization problem with a well-defined objective function is developed to formulate the proposed problem and solved by a gradient descent procedure with curvilinear search. Extensive experiments on real-world social image databases are conducted on the tasks of image understanding: image tag refinement, assignment, and retrieval. Encouraging results are achieved with comparison with the state-of-the-art algorithms, which demonstrates the effectiveness of the proposed method.