Automatic Tagging by Leveraging Visual and Annotated Features in Social Media
Automatic Tagging by Leveraging Visual and Annotated Features in Social Media
复制标题
DOI：
10.1109/tmm.2021.3055037
复制
发表时间：
2021-01
影响因子：
7.3
通讯作者：
Jinpeng Chen;Pinguang Ying;Xiangling Fu;Xiaopeng Luo;Hao Guan;Kaimin Wei
中科院分区：
文献类型：
--
作者：
Jinpeng Chen;Pinguang Ying;Xiangling Fu;Xiaopeng Luo;Hao Guan;Kaimin Wei
自动图像标注是帮助提取图像含义的研究领域之一，其目的是为图像产生一组语义标注，以帮助更好地表达概念。在过去的几十年里，研究人员开发了许多自动图像注释的方法。然而，先前的研究尚未完全解释视觉特征和注释特征。因此，通过结合视觉和注释信息仍然可以实现更好的注释性能。在本研究中，我们的目标是将多个语义标签与给定图像相关联。特别是，我们检测如何利用视觉和注释信息来获取图像注释。为了利用视觉信息，我们首先设计了一种改进的神经网络方法来获取图像内容的特征。此外，为了获得带注释的特征，我们利用了一种聚合网络嵌入方法，该方法由注释嵌入、社交嵌入、个人资料嵌入和语义嵌入组成。最后，为了产生准确的图像标注，我们整合了上述两种方法，即将视觉信息和标注信息结合起来，构建了一个统一的协作训练框架。三个真实世界数据集的实验结果表明，我们提出的方法优于当前流行的图像注释方法。
Automatic image annotation is one of the research fields helping to extract the meaning of images, which aims at the production of a set of semantic annotations for an image to help better present the concept. Over the past few decades, researchers have developed many approaches for automatic image annotation. Nevertheless, previous studies have not fully accounted for visual features and annotated features. Therefore, it is still possible to achieve a better annotation performance by combining visual and annotated information. In this study, we aim to associate multiple semantic tags with a given image. In particular, we detect how to obtain the image annotation by utilizing visual and annotated information. To take advantage of visual information, we first designed a modified neural network method to acquire the features of the image content. In addition, to obtain the annotated features, we exploit an aggregated network embedding approach that consists of annotation embedding, social embedding, profile embedding, and semantic embedding. Finally, to produce an accurate image annotation, we integrate the two aforementioned methods, that is, combining the visual and annotated information, to build a unified cooperative training framework. The experimental results on three real-world datasets clarify that our presented method is superior to the currently popular image annotation approaches.