EAGER: Declarative Crowdsourcing

EAGER：声明式众包

• 批准号: 1251827
• 负责人: Neoklis Polyzotis
• 金额: $ 19.99万
• 依托单位: University of California-Santa Cruz
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-10-01 至 2014-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1251827&HistoricalAwards=false
• 关键词: EAGER; Declarative; Crowdsourcing

A variety of applications are increasingly relying on crowdsourcing services, such as Amazon Mechanical Turk or CrowdFlower, in order to access human computation at a large scale and solve problems that cannot be tackled using only machine computation. Despite the surge in crowdsourcing platforms, it remains very difficult and error-prone to employ crowdsourcing within an application: existing services mostly expose a procedural interface to post individual human-computation tasks, and provide little support (if any) for task coordination, reward management, or clean-up of the obtained answers. As a result, a large fraction of the application logistics is devoted to orchestrating and optimizing the interaction with the crowdsourcing service. This project explores the novel paradigm of declarative crowdsourcing through the development of the Deco database system. Deco models and offers support for accessing the collective knowledge of the crowd by posing declarative queries over a relational-like database. The project explores methods to mitigate the effect of "noisy" human workers who provide data of low quality and to model the resulting uncertainty in the answers returned by Deco. The two problems are tightly coupled with a tradeoff among the latency to contact human workers, the expense to recruit them and the quality of the data they provide. Handling this tradeoff in the context of query optimization is one of the key technical challenges addressed by the project. The project represents a high-risk research effort, as it targets non-trivial problems that are inherent in the usage of crowdsourcing in practice. The corresponding high payoff is that the results of this research provide a robust and principled foundation for declarative crowdsourcing, thus enabling a wide variety of applications to incorporate crowdsourcing as a core component of their software stack. Moreover, this project identifies desirable features of crowdsourcing services in order to support this novel declarative interface, thereby providing valuable guidance for the design of next-generation crowdsourcing platforms. Finally, the project provides training to students and the opportunity to engage in the emerging research area that lies in the intersection of databases and crowdsourcing. Details for the project can be found at the project web site (http://db.cs.ucsc.edu/deco).

各种应用程序越来越多地依赖于众包服务，例如Amazon Mechanical Turk或CrowdFlower，以便大规模访问人类计算并解决仅使用机器计算无法解决的问题。尽管众包平台激增，但在应用程序中使用众包仍然非常困难和容易出错：现有的服务大多公开了一个程序接口来发布单个人工计算任务，并且很少为任务协调，奖励管理或清理所获得的答案提供支持（如果有的话）。因此，很大一部分应用程序物流都致力于协调和优化与众包服务的交互。这个项目通过Deco数据库系统的开发探索了声明式众包的新范式。Deco通过在类似关系的数据库上提出声明性查询，为访问群体的集体知识建模并提供支持。该项目探索了减轻提供低质量数据的“嘈杂”人类工作人员的影响的方法，并对Deco返回的答案中产生的不确定性进行建模。这两个问题与联系人类工人的延迟，招聘他们的费用以及他们提供的数据质量之间的权衡紧密相关。在查询优化的上下文中处理这种折衷是该项目解决的关键技术挑战之一。该项目是一项高风险的研究工作，因为它针对的是实践中使用众包所固有的重要问题。相应的高回报是，这项研究的结果为声明式众包提供了一个强大而有原则的基础，从而使各种各样的应用程序能够将众包作为其软件堆栈的核心组件。此外，该项目确定了众包服务的理想功能，以支持这种新颖的声明式接口，从而为下一代众包平台的设计提供了有价值的指导。最后，该项目为学生提供培训，并有机会参与数据库和众包交叉的新兴研究领域。该项目的详细情况可在项目网站（http：//db.cs.ucsc.edu/deco）上查阅。