Improving Crowdsourced Label Quality Using Noise Correction
Improving Crowdsourced Label Quality Using Noise Correction
复制标题
使用噪声校正提高众包标签质量
DOI：
10.1109/tnnls.2017.2677468
复制
发表时间：
2018-05
影响因子：
10.4
通讯作者：
Xindong Wu
中科院分区：
文献类型：
--
作者：
Jing Zhang（张静）;Victor S. Sheng;Tao Li;Xindong Wu
众包系统提供了一种经济有效且方便的方式来收集标签，但它们往往无法保证标签的质量。本文提出了一种新的框架，该框架引入了噪声校正技术，以进一步提高从物体的多个噪声标签推断的综合标签的质量。在所提出的通用框架中，通过前端地面真理推理算法估计标记器的质量信息来指导后续的标签噪声过滤和校正。该框架使用一种名为自适应投票噪声校正(AVNC)的新算法来精确识别和校正潜在的噪声标签。在过滤掉含有噪声标签的实例后，将剩余的净化数据集用于创建多个弱分类器，并在此基础上引入一个强大的集成分类器来校正这些噪声。在8个具有不同特征的模拟数据集和2个不同领域的真实众包数据集上的实验结果一致表明：1)无论采用何种推理算法，该框架都能提高标签质量，特别是在每个实例具有较少重复标签的情况下；2)由于所提出的AVNC算法同时考虑了潜在标签噪声的数量和概率，因此其性能优于现有的噪声校正算法。
Crowdsourcing systems provide a cost effective and convenient way to collect labels, but they often fail to guarantee the quality of the labels. This paper proposes a novel framework that introduces noise correction techniques to further improve the quality of integrated labels that are inferred from the multiple noisy labels of objects. In the proposed general framework, information about the qualities of labelers estimated by a front-end ground truth inference algorithm is utilized to supervise subsequent label noise filtering and correction. The framework uses a novel algorithm termed adaptive voting noise correction (AVNC) to precisely identify and correct the potential noisy labels. After filtering out the instances with noisy labels, the remaining cleansed data set is used to create multiple weak classifiers, based on which a powerful ensemble classifier is induced to correct these noises. Experimental results on eight simulated data sets with different kinds of features and two real-world crowdsourcing data sets in different domains consistently show that: 1) the proposed framework can improve label quality regardless of inference algorithms, especially under the circumstance that each instance has a few repeated labels and 2) since the proposed AVNC algorithm considers both the number of and the probability of potential label noises, it outperforms the state-of-the-art noise correction algorithms.
登录
查看更多内容
DOI：
10.1007/11871637_49
发表时间：
2006-09
期刊：
--
影响因子：
--
作者：
E. Frank;R. Bouckaert
通讯作者：
E. Frank;R. Bouckaert
DOI：
10.1609/aaai.v26i1.8105
发表时间：
2012-07
期刊：
Proceedings of the AAAI Conference on Artificial Intelligence
影响因子：
--
作者：
Hiroshi Kajino;Yuta Tsuboi;H. Kashima
通讯作者：
Hiroshi Kajino;Yuta Tsuboi;H. Kashima
DOI：
10.1109/icde.2016.7498229
发表时间：
2016-05
期刊：
2016 IEEE 32nd International Conference on Data Engineering (ICDE)
影响因子：
--
作者：
Huiqi Hu;Yudian Zheng;Z. Bao;Guoliang Li;Jianhua Feng;Reynold Cheng
通讯作者：
Huiqi Hu;Yudian Zheng;Z. Bao;Guoliang Li;Jianhua Feng;Reynold Cheng
DOI：
10.1109/vlhcc.2011.6070377
发表时间：
2011-11
期刊：
2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)
影响因子：
--
作者：
Amber Shinsel;Todd Kulesza;Margaret M. Burnett;William Curran;Alex Groce;Simone Stumpf;Weng-Keen Wong
通讯作者：
Amber Shinsel;Todd Kulesza;Margaret M. Burnett;William Curran;Alex Groce;Simone Stumpf;Weng-Keen Wong
DOI：
--
发表时间：
1997-08
期刊：
--
影响因子：
--
作者：
Seishi Okamoto;N. Yugami
通讯作者：
Seishi Okamoto;N. Yugami