Completion of Missing Labels for Multi-Label Annotation by a Unified Graph Laplacian Regularization
Completion of Missing Labels for Multi-Label Annotation by a Unified Graph Laplacian Regularization
复制标题
DOI：
10.1587/transinf.2019edp7318
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Jonathan Mojoo;Yu Zhao;M. Kavitha;J. Miyao;Takio Kurita
中科院分区：
文献类型：
--
作者：
Jonathan Mojoo;Yu Zhao;M. Kavitha;J. Miyao;Takio Kurita
图像标注任务对于从网络和其他大型数据库中高效检索图像变得非常重要。然而，大量的语义信息和图像上标签的复杂依赖关系使得这项任务具有挑战性。因此，确定图像上多个标签之间的语义相似性对于理解任何不完整的图像检索标签分配是有用的。本文提出了一种新的方法，通过统一深度卷积神经网络（CNN）中两种不同类型的拉普拉斯正则化项来解决多标签图像标注问题，以获得鲁棒标注性能。实现了统一的拉普拉斯正则化模型，通过标签的语义相似度产生标签内部和外部的上下文相似度，有效地解决了缺失标签问题，这是本研究的主要贡献。具体而言，我们内部使用Hayashi的量化方法type III生成标签之间的相似矩阵，外部使用word2vec方法生成标签之间的相似矩阵。然后将两种不同方法生成的相似矩阵组合为一个拉普拉斯正则化项，作为深度CNN的新目标函数。本研究中实现的正则化项能够解决多标签标注问题，使训练的神经网络更加有效。在公共基准数据集上的实验结果表明，所提出的深度CNN统一正则化模型在预测缺失标签方面的效果明显优于未正则化的基线CNN和其他最先进的方法。
SUMMARY The task of image annotation is becoming enormously important for e ffi cient image retrieval from the web and other large databases. However, huge semantic information and complex dependency of labels on an image make the task challenging. Hence determining the semantic similarity between multiple labels on an image is useful to understand any incomplete label assignment for image retrieval. This work proposes a novel method to solve the problem of multi-label image annotation by unifying two di ff erent types of Laplacian regularization terms in deep convolutional neural network (CNN) for robust annotation performance. The unified Laplacian regularization model is implemented to address the missing labels e ffi ciently by generating the contextual similarity between labels both internally and externally through their semantic similarities, which is the main contribution of this study. Specifically, we generate similarity matrices between labels internally by using Hayashi’s quantification method-type III and externally by using the word2vec method. The generated similarity matrices from the two di ff erent methods are then combined as a Laplacian regularization term, which is used as the new objective function of the deep CNN. The Regularization term implemented in this study is able to address the multi-label annotation problem, enabling a more e ff ectively trained neural network. Experimental results on public benchmark datasets reveal that the proposed unified regularization model with deep CNN produces significantly better results than the baseline CNN without regularization and other state-of-the-art methods for predicting missing labels.