权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

RI: Small: Actively Learning From The Crowd

RI：小型：积极向大众学习

基本信息

批准号：
1816986
负责人：
Reinhard Heckel
金额：
$ 47.43万
依托单位：
William Marsh Rice University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-09-01 至 2023-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816986&HistoricalAwards=false
关键词：
RI Small Actively Learning Crowd

项目摘要

How can a large number of questions be answered from few and only partially correct responses? This problem lies at the heart of labeling large collections of unlabeled data, which are common in machine learning and data science. It also lies at the heart of learning about preferences of people and quality of items by carrying out surveys. A popular approach to address this problem is to crowdsource the labeling or learning task by paying a large number of people small amounts of money to answer questions on the internet through a crowdsourcing platform. However, the quality of the workers responses varies significantly due to different abilities of the people and difficulties of the questions. To account for the uncertainty of the responses, each question is assigned to multiple people and their responses are aggregated. However, the assignment process is often agnostic to the peoples abilities and questions difficulties, since both are unknown a priori. This project will develop algorithms that adapt to the people and questions and thereby significantly reduces the number of responses required to enable machine learning algorithms to perform well and surveys to be informative. Besides the research objectives, the researchers will pursue educational objectives by integrating parts of this project into a graduate class, promoting undergraduate research, and fostering exchange across disciplines by running an interdisciplinary machine learning seminar. The core optimization problem in crowdsourcing is to achieve confidence in the final answers at minimal cost, by assigning only few tasks to the people (or workers). Intuitively, that can be accomplished by only posing a question to the workers best qualified to answer that question. This project will develop efficient and practical active schemes for crowdlabeling and crowdsourcing that adaptively choose which question to pose to which worker. For each algorithm, the project will prove corresponding rigorous computational and statistical problem-instance dependent performance guarantees, as opposed to worst-case performance guarantees. The theoretical results will be complemented with practical open-source implementations and experiments on real-world data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

如何能从很少的、只有部分正确的回答中回答大量的问题？这个问题是标记大量未标记数据的核心，这在机器学习和数据科学中很常见。它也是通过进行调查了解人们的偏好和物品质量的核心。解决这个问题的一个流行方法是众包标签或学习任务，通过支付大量的人少量的钱来回答互联网上的问题，通过众包平台。然而，由于人们的能力和问题的难度不同，工人的回答质量差异很大。考虑到回答的不确定性，每个问题被分配给多个人，他们的回答被汇总。然而，分配过程往往是不可知的人的能力和问题的困难，因为两者都是未知的先验。该项目将开发适应人员和问题的算法，从而大大减少使机器学习算法能够良好执行和调查所需的响应数量。除了研究目标外，研究人员还将通过将该项目的部分内容整合到研究生课程中，促进本科生研究，并通过举办跨学科机器学习研讨会来促进跨学科交流，从而实现教育目标。众包中的核心优化问题是通过只向人（或工人）分配很少的任务，以最小的成本获得对最终答案的信心。直觉上，这可以通过向最有资格回答这个问题的工人提出一个问题来实现。该项目将开发有效和实用的主动方案，用于众包和众包，自适应地选择向哪个工人提出哪个问题。对于每个算法，该项目将证明相应的严格的计算和统计问题实例相关的性能保证，而不是最坏情况下的性能保证。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。