Clickstream Analysis for Crowd-Based Object Segmentation with Confidence
Clickstream Analysis for Crowd-Based Object Segmentation with Confidence
复制标题
自信地进行基于人群的对象分割的点击流分析
DOI：
10.1109/tpami.2017.2777967
复制
发表时间：
2016
影响因子：
23.6
通讯作者：
L. Maier
中科院分区：
文献类型：
--
作者：
Eric Heim;A. Seitel;Jonas Andrulis;Fabian Isensee;C. Stock;T. Ross;L. Maier
随着人们对基于机器学习的自动图像注释解决方案的兴趣迅速增加，用于算法训练的参考注释的可用性是该领域的主要瓶颈之一。众包已经发展成为低成本和大规模数据注释的一个有价值的选择；然而，质量控制仍然是一个需要解决的主要问题。据我们所知，我们是第一个分析标注过程以改进众包图像分割的人。我们的方法包括训练一个回归器来估计从注释者的点击流数据中分割的质量。质量估计可用于识别垃圾邮件，并在合并一张图像的多个分割时，根据其（估计的）质量对单个注释进行加权。通过对不同对象类的公开可用数据执行的总共29,000个人群注释，我们表明：(1)我们的方法在基于点击流数据估计分割质量方面非常准确，(2)在合并多个注释方面优于最先进的方法。由于回归量不需要对其应用的对象类进行训练，因此可以将其视为基于人群的图像注释环境中质量控制和置信度分析的低成本选择。
With the rapidly increasing interest in machine learning based solutions for automatic image annotation, the availability of reference annotations for algorithm training is one of the major bottlenecks in the field. Crowdsourcing has evolved as a valuable option for low-cost and large-scale data annotation; however, quality control remains a major issue which needs to be addressed. To our knowledge, we are the first to analyze the annotation process to improve crowd-sourced image segmentation. Our method involves training a regressor to estimate the quality of a segmentation from the annotator's clickstream data. The quality estimation can be used to identify spam and weight individual annotations by their (estimated) quality when merging multiple segmentations of one image. Using a total of 29,000 crowd annotations performed on publicly available data of different object classes, we show that (1) our method is highly accurate in estimating the segmentation quality based on clickstream data, (2) outperforms state-of-the-art methods for merging multiple annotations. As the regressor does not need to be trained on the object class that it is applied to it can be regarded as a low-cost option for quality control and confidence analysis in the context of crowd-based image annotation.
DOI：
10.1007/978-3-319-16814-2_39
发表时间：
2014-11
期刊：
--
影响因子：
--
作者：
D. Kondermann;Rahul Nair;S. Meister;W. Mischler;Burkhard Güssefeld;Katrin Honauer;Sabine Hofmann
通讯作者：
D. Kondermann;Rahul Nair;S. Meister;W. Mischler;Burkhard Güssefeld;Katrin Honauer;Sabine Hofmann