EAGER: Declarative Crowdsourcing

各种各样的应用程序越来越依赖于众包服务，比如Amazon Mechanical Turk或CrowdFlower，以便大规模地访问人类计算并解决仅使用机器计算无法解决的问题。尽管众包平台激增，但在应用程序中使用众包仍然非常困难且容易出错：现有的服务大多公开一个过程接口来发布单个的人工计算任务，并且对任务协调、奖励管理或获得的答案的清理提供很少的支持（如果有的话）。因此，应用程序物流的很大一部分用于编排和优化与众包服务的交互。该项目通过Deco数据库系统的开发，探索了声明性众包的新范式。Deco通过在类似关系的数据库上提出声明性查询，对人群的集体知识进行建模并提供访问支持。该项目探索了减轻“嘈杂”的人类工作人员提供低质量数据的影响的方法，并对Deco返回的答案中产生的不确定性进行建模。这两个问题与联系人力的延迟、招聘人力的费用和他们提供的数据质量之间的权衡紧密相连。在查询优化上下文中处理这种权衡是该项目要解决的关键技术挑战之一。该项目代表了高风险的研究工作，因为它针对的是在实践中使用众包所固有的非琐碎问题。相应的高回报是，这项研究的结果为声明性众包提供了一个强大而有原则的基础，从而使各种各样的应用程序能够将众包作为其软件堆栈的核心组件。此外，为了支持这种新颖的声明式接口，本项目确定了众包服务的理想特性，从而为下一代众包平台的设计提供了有价值的指导。最后，该项目为学生提供培训，并有机会参与数据库和众包交叉的新兴研究领域。该项目的详细信息可在该项目的网站（http://db.cs.ucsc.edu/deco）上找到。

A variety of applications are increasingly relying on crowdsourcing services, such as Amazon Mechanical Turk or CrowdFlower, in order to access human computation at a large scale and solve problems that cannot be tackled using only machine computation. Despite the surge in crowdsourcing platforms, it remains very difficult and error-prone to employ crowdsourcing within an application: existing services mostly expose a procedural interface to post individual human-computation tasks, and provide little support (if any) for task coordination, reward management, or clean-up of the obtained answers. As a result, a large fraction of the application logistics is devoted to orchestrating and optimizing the interaction with the crowdsourcing service. This project explores the novel paradigm of declarative crowdsourcing through the development of the Deco database system. Deco models and offers support for accessing the collective knowledge of the crowd by posing declarative queries over a relational-like database. The project explores methods to mitigate the effect of "noisy" human workers who provide data of low quality and to model the resulting uncertainty in the answers returned by Deco. The two problems are tightly coupled with a tradeoff among the latency to contact human workers, the expense to recruit them and the quality of the data they provide. Handling this tradeoff in the context of query optimization is one of the key technical challenges addressed by the project. The project represents a high-risk research effort, as it targets non-trivial problems that are inherent in the usage of crowdsourcing in practice. The corresponding high payoff is that the results of this research provide a robust and principled foundation for declarative crowdsourcing, thus enabling a wide variety of applications to incorporate crowdsourcing as a core component of their software stack. Moreover, this project identifies desirable features of crowdsourcing services in order to support this novel declarative interface, thereby providing valuable guidance for the design of next-generation crowdsourcing platforms. Finally, the project provides training to students and the opportunity to engage in the emerging research area that lies in the intersection of databases and crowdsourcing. Details for the project can be found at the project web site (http://db.cs.ucsc.edu/deco).