Improving Label Quality in Crowdsourcing Using Noise Correction
Improving Label Quality in Crowdsourcing Using Noise Correction
复制标题
DOI：
10.1145/2806416.2806627
复制
发表时间：
2015-10
期刊：
影响因子：
--
通讯作者：
J. Zhang;Victor S. Sheng;Jian Wu;Xiaoqin Fu;Xindong Wu
中科院分区：
文献类型：
--
作者：
J. Zhang;Victor S. Sheng;Jian Wu;Xiaoqin Fu;Xindong Wu
本文提出了一种新的框架，引入噪声校正技术，以进一步提高标签质量后，地面真相推断众包。在该框架中，提出了一种自适应投票噪声校正算法（AVNC），以识别和纠正最可能的噪声的帮助下，估计质量的标签提供的地面真值推理。在两个真实数据集上的实验结果表明：（1）该框架可以提高标签质量，无论推理算法如何，特别是在每个示例都有少量噪声标签的情况下;（2）由于AVNC算法同时考虑了潜在噪声的数量和概率，因此优于基线噪声校正算法。
This paper proposes a novel framework that introduces noise correction techniques to further improve label quality after ground truth inference in crowdsourcing. In the framework, an adaptive voting noise correction algorithm (AVNC) is proposed to identify and correct the most likely noises with the help of estimated qualities of labelers provided by the ground truth inference. The experimental results on two real-world datasets show that (1) the framework can improve label quality regardless of inference algorithms, especially under the circumstance that each example has a few noisy labels; and (2) since the algorithm AVNC considers both the number of and the probability of potential noises, it outperforms a baseline noise correction algorithm.