EAGER: Declarative Crowdsourcing

各种应用程序越来越依赖众包服务，如亚马逊机械土耳其或CrowdFlow，以便大规模访问人类计算，并解决仅使用机器计算无法解决的问题。尽管众包平台激增，但在应用程序中使用众包仍然非常困难和容易出错：现有服务大多公开一个程序接口来发布单个人工计算任务，并且几乎没有为任务协调、奖励管理或获得的答案的清理提供支持(如果有的话)。因此，很大一部分应用程序物流都用于编排和优化与众包服务的交互。该项目通过开发Deco数据库系统，探索了声明性众包的新范例。DECO通过在类似关系的数据库上提出声明性查询来建模并提供对访问公众集体知识的支持。该项目探索了一些方法，以减轻“嘈杂”的人类工作人员提供的低质量数据的影响，并对Deco返回的答案中产生的不确定性进行建模。这两个问题与联系人类工人的延迟、招聘他们的费用以及他们提供的数据质量之间的权衡密切相关。在查询优化的上下文中处理这种权衡是该项目解决的关键技术挑战之一。该项目代表了一项高风险的研究工作，因为它针对的是在实践中使用众包所固有的重大问题。相应的高回报是，这项研究的结果为声明性众包提供了强大和原则性的基础，从而使各种应用程序能够将众包作为其软件堆栈的核心组件。此外，该项目确定了众包服务的理想功能，以支持这种新颖的声明式界面，从而为下一代众包平台的设计提供有价值的指导。最后，该项目为学生提供培训，并提供参与数据库和众包交叉的新兴研究领域的机会。有关该项目的详细资料，请浏览该项目的网站(http://db.cs.ucsc.edu/deco).

A variety of applications are increasingly relying on crowdsourcing services, such as Amazon Mechanical Turk or CrowdFlower, in order to access human computation at a large scale and solve problems that cannot be tackled using only machine computation. Despite the surge in crowdsourcing platforms, it remains very difficult and error-prone to employ crowdsourcing within an application: existing services mostly expose a procedural interface to post individual human-computation tasks, and provide little support (if any) for task coordination, reward management, or clean-up of the obtained answers. As a result, a large fraction of the application logistics is devoted to orchestrating and optimizing the interaction with the crowdsourcing service. This project explores the novel paradigm of declarative crowdsourcing through the development of the Deco database system. Deco models and offers support for accessing the collective knowledge of the crowd by posing declarative queries over a relational-like database. The project explores methods to mitigate the effect of "noisy" human workers who provide data of low quality and to model the resulting uncertainty in the answers returned by Deco. The two problems are tightly coupled with a tradeoff among the latency to contact human workers, the expense to recruit them and the quality of the data they provide. Handling this tradeoff in the context of query optimization is one of the key technical challenges addressed by the project. The project represents a high-risk research effort, as it targets non-trivial problems that are inherent in the usage of crowdsourcing in practice. The corresponding high payoff is that the results of this research provide a robust and principled foundation for declarative crowdsourcing, thus enabling a wide variety of applications to incorporate crowdsourcing as a core component of their software stack. Moreover, this project identifies desirable features of crowdsourcing services in order to support this novel declarative interface, thereby providing valuable guidance for the design of next-generation crowdsourcing platforms. Finally, the project provides training to students and the opportunity to engage in the emerging research area that lies in the intersection of databases and crowdsourcing. Details for the project can be found at the project web site (http://db.cs.ucsc.edu/deco).