Semi-automatic dynamic auxiliary-tag-aided image annotation
Semi-automatic dynamic auxiliary-tag-aided image annotation
复制标题
DOI：
10.1016/j.patcog.2009.03.009
复制
发表时间：
2010-02
期刊：
影响因子：
--
通讯作者：
Shile Zhang;Bin Li;X. Xue
中科院分区：
文献类型：
--
作者：
Shile Zhang;Bin Li;X. Xue
图像注释是许多现实应用的基础。在Web 2.0时代，图像搜索和浏览在很大程度上是基于图像的标签。在本文中，我们制定了图像标注作为一个多标签学习问题，并开发了一个半自动的图像标注系统。所提出的系统从词汇表中选择合适的词作为给定图像的标签，并在用户反馈的帮助下改进标签。这种改进相当于一种新的多标签学习框架，称为半自动动态辅助标签辅助（SADATA），其中一个特定标签（目标标签）的分类结果可以通过其他标签（辅助标签）的子集的分类结果来提升。根据归一化互信息确定与目标标签具有强相关性的辅助标签。我们只选择那些相关性超过阈值的标签作为辅助标签，因此辅助标签集是稀疏的。辅助标签的贡献大小取决于图像，因此我们还建立了一个以辅助标签和输入图像为条件的概率模型，以动态调整辅助标签的权重。对于给定的图像，用户对标签的反馈校正了辅助分类器的输出，SADATA将在下一轮推荐更合适的标签。SADATA在大量Corel图像上进行评估。实验结果验证了该方法的有效性。此外，性能还受益于用户反馈，使得注释过程可以显著加快。
Image annotation is the foundation for many real-world applications. In the age of Web 2.0, image search and browsing are largely based on the tags of images. In this paper, we formulate image annotation as a multi-label learning problem, and develop a semi-automatic image annotation system. The presented system chooses proper words from a vocabulary as tags for a given image, and refines the tags with the help of the user's feedback. The refinement amounts to a novel multi-label learning framework, named Semi-Automatic Dynamic Auxiliary-Tag-Aided (SADATA), in which the classification result for one certain tag (target tag) can be boosted by the classification results of a subset of the other tags (auxiliary tags). The auxiliary tags, which have strong correlations with the target tag, are determined in terms of the normalized mutual information. We only select those tags whose correlations exceed a threshold as the auxiliary tags, so the auxiliary set is sparse. How much an auxiliary tag can contribute is dependent on the image, so we also build a probabilistic model conditioned on the auxiliary tag and the input image to adjust the weight of the auxiliary tag dynamically. For an given image, the user feedback on the tags corrects the outputs of the auxiliary classifiers and SADATA will recommend more proper tags next round. SADATA is evaluated on a large collection of Corel images. The experimental results validate the effectiveness of our dynamic auxiliary-tag-aided method. Furthermore, the performance also benefits from user feedbacks such that the annotation procedure can be significantly speeded up.