数据标记获取的众包过程分析与设计

结题报告

项目介绍

AI项目解读

基本信息

批准号：
61673202
项目类别：
面上项目
资助金额：
61.0万
负责人：
王魏
依托单位：
南京大学
学科分类：
F0603.机器学习
结题年份：
2020
批准年份：
2016
项目状态：
已结题
起止时间：
2017-01-01 至2020-12-31

项目参与者：
张宇；徐淼；侯博建；王璐；郭翔宇；吴西竹；刘冲；宋歌；周宇航；
关键词：
PAC理论众包计算学习理论

项目摘要

In the era of big data, data can often be obtained abundantly and cheaply, but providing labels for these large-scale data has always been a challenge because labelling data is expensive and time-consuming. Crowdsourcing has been an effective and efficient paradigm for providing labels for large-scale data, in which users submit their “micro-tasks” in the internet that can be completed by voluntary workers in exchange for small monetary payments. Once the tasks are posted by the taskmaster, thousands of workers have internet access to them, and the users can collect labels for these tasks in a short period of time. In this project, we focus on the theoretical study on crowdsourcing and provide theoretical analyses which help to fill the gap between theory and algorithm on crowdsourcing. 1. We present theoretical analysis on inferring the label from the crowd and study how to provide the generalization error bound for the inferred labels without strong assumptions and much label cost. 2. We present theoretical analysis on worker selection and study how to eliminate the low-quality and dishonest workers from the crowd. 3. We present theoretical analysis on task assignment, and study the predictability of the difficulty of the tasks and how to assign the tasks with respect to the predicted difficulty. 4. We develop new crowdsourcing process to match the workers with the tasks and provide theoretical support. As an application of the new crowdsourcing process, we will build a prototype system. It is expected to publish 6-8 papers on important international journals and conferences and native top journals, apply 1-2 patents, and supervise 3-5 graduate students.

在大数据时代获取数据已非常容易，但为这些海量数据提供标记却仍然十分困难。众包是近年来兴起的一种高效地为海量数据提供标记的方法，本项目对众包过程进行理论研究，围绕其中的标记推导、雇员选择和任务分配三大步骤展开, 以填补理论和算法之间存在的间隙：第一，对标记推导进行理论分析，研究如何在不显著增加标记代价的前提下，基于较少的模型假设分析推导获取标记的泛化误差界；第二，对雇员选择进行理论分析，研究如何在不显著增加标记代价和不影响任务完成的前提下，找到除去低质量雇员和欺骗型雇员的条件；第三，对任务分配进行理论分析，研究任务困难程度的可预测性及根据任务的困难程度进行任务分配的准则；第四，依据分析得到的理论结果设计新的众包过程以完善雇员和任务之间的匹配，并提供理论支撑。本项目将为上述问题提供解决方案并研制原型系统，发表国际期刊/会议和国内一级学报论文6-8篇，申请专利1-2项，培养研究生3-5名。

结项摘要

在大数据时代获取数据已非常容易，但为这些海量数据提供标记却仍然十分困难。众包是近年来兴起的一种高效地为海量数据提供标记的方法，本项目针对众包过程中的雇员选择、任务分配和标记推导三大步骤展开研究，设计新的众包过程以完善雇员和任务之间的匹配和标记推导，提高标记精度，并研制原型系统。基于研究成果，发表论文12篇，其中国内期刊论文《Frontiers of Computer Science》和《National Science Review》各1篇、CCF-A类论文10篇，获国家发明专利授权1项。