Multi-Label Truth Inference for Crowdsourcing Using Mixture Models
Multi-Label Truth Inference for Crowdsourcing Using Mixture Models
复制标题
使用混合模型进行众包的多标签真相推断
DOI：
10.1109/tkde.2019.2951668
复制
发表时间：
2019-11
影响因子：
8.9
通讯作者：
Wu Xindong
中科院分区：
文献类型：
--
作者：
Zhang Jing（张静）;Wu Xindong
When acquiring labels from crowdsourcing platforms, a task may be designed to include multiple labels and the values of each label may belong to a set of various distinct options, which is the so-called multi-class multi-label annotation. To improve the quality of labels, requesters usually let one task be independently completed by a group of heterogeneous crowdsourced workers. Then, the true values of the multiple labels of each task are inferred from these repeated noisy labels. In this paper, we propose two novel probabilistic models MCMLI and MCMLD to address the multi-class multi-label inference problem in crowdsourcing. MCMLI assumes that the labels of each task are mutually independent and MCMLD utilizes a mixture of multiple independently multinoulli distributions to capture the correlation among the labels. Both models can jointly infer multiple true labels of each instance as well as estimate the reliability of crowdsourced workers modeled by a set of confusion matrices with an expectation–maximization algorithm. Experiments with three typical crowdsourcing scenarios and a real-world dataset show that our proposed models significantly outperform existing competitive alternatives. When the labels are strongly correlated, MCMLD substantially outperforms MCMLI. Furthermore, our models can be easily simplified to the one-coin models, which show more advantageous when errors are uniformly distributed, or labels are sparse.
登录
查看更多内容
使用噪声校正提高众包标签质量
DOI：
10.1109/tnnls.2017.2677468
发表时间：
2018-05
影响因子：
10.4
作者：
Jing Zhang（张静）;Victor S. Sheng;Tao Li;Xindong Wu
通讯作者：
Xindong Wu
DOI：
--
发表时间：
2011-12
期刊：
--
影响因子：
--
作者：
David R Karger;Sewoong Oh;Devavrat Shah
通讯作者：
David R Karger;Sewoong Oh;Devavrat Shah
影响因子：
2.7
作者：
Eran Toch;B. Lerner;Eyal Ben Zion;I. Ben-Gal
通讯作者：
Eran Toch;B. Lerner;Eyal Ben Zion;I. Ben-Gal
DOI：
--
发表时间：
2012-03
期刊：
--
影响因子：
--
作者：
Hyun-Chul Kim;Zoubin Ghahramani
通讯作者：
Hyun-Chul Kim;Zoubin Ghahramani
DOI：
--
发表时间：
2015-08
期刊：
ArXiv
影响因子：
--
作者：
Shao-Yuan Li;Yuan Jiang;Zhi-Hua Zhou
通讯作者：
Shao-Yuan Li;Yuan Jiang;Zhi-Hua Zhou