CAREER: Streamlining Task Deployment on Crowdsourcing Platforms

众包利用在线基础设施，利用人类未被充分开发和丰富的异质知识和认知池来解决各种任务，否则这些任务被认为是机器单独难以解决的。众包系统建立在私人或公共平台上，是部署各种需要人工智能的任务的流行手段。在此类平台上部署任务需要确定适当的部署策略，以满足请求者提供的部署参数，作为质量、延迟和成本的阈值，还需要分析可用于承担已部署任务的劳动力。到目前为止，任务部署仍然是一个艰苦的手动过程，因为请求者在决定如何组织员工、以什么方式组织以及以什么结构来满足部署参数方面几乎没有帮助。因此，请求者和工作人员大多被限制在一个平台上，因为不存在跨平台部署流程的轻松可移植性。该项目研究位于众包生态系统中多个利益相关者之间的中间层，通过允许轻松灵活地指定部署限制和目标，然后基于这些规范推荐部署策略，来帮助请求者部署众包任务。该系统的开发实现了部署过程跨平台的可移植性和可重用性。为实现这些目标，本项目开发了一个包含三个集成组件的中间件系统：(1)部署策略推荐引擎被优化以适应生态系统中的多个利益相关者，并负责对批量请求进行建模和推荐部署策略。(2)劳动力分析引擎分析可用的劳动力并提供给推荐引擎，因为部署的任务将由员工承担。该引擎的输出是对工人偏好或人为因素的估计，例如工人的可用性，作为精确(离散)或不精确(间隔或概率分布函数)的信息。(3)结果聚合模块对部署的任务质量进行评估，然后反馈给其他两个引擎进行重新调整。它得到了全自动或混合算法的支持，这些算法很少涉及机器算法中的人类智能。斯隆的开发计划包括原则性建模、严格的算法设计、声明性框架开发、在多个真实平台中的部署和集成。斯隆的不同组成部分被赋予了多目标离散优化和计算几何算法，以及适应于机器学习的多方面建模技术。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Crowdsourcing leverages online infrastructure to tap an under-explored and richly heterogeneous pool of human knowledge and cognition for solving a variety of tasks that are otherwise considered hard for machines to solve alone. Crowdsourcing systems are built on private or public platforms and are a popular means of deploying a variety of tasks that require human intelligence. Task deployment on such platforms requires identifying appropriate deployment strategies to satisfy deployment parameters, provided by requesters as thresholds on quality, latency, and cost, and also requires analysis of the workforce that is available to undertake the deployed tasks. To date, task deployment remains a painstakingly manual process, as there is little to no help for requesters in deciding how to organize the workforce, in what style, and in what structure to satisfy deployment parameters. Consequently, requesters and workers are mostly confined to one platform, as there is no easy portability of deployment processes across platforms. This project investigates a middle layer that sits between multiple stakeholders in a crowdsourcing ecosystem to aid requesters in deploying crowdsourcing tasks by allowing easy and flexible specification of deployment constraints and goals, and then recommending deployment strategies based on those specifications. Development of this system thus enables the portability and reuse of deployment processes across platforms.To achieve these goals, this project develops a middleware system called SLOAN (Scalable, decLarative, Optimization-driven, Adaptive, and uNified) with three integrated components: (1) The Deployment Strategy Recommendation Engine is optimized to accommodate multi-stakeholders in the ecosystem, and is responsible for modeling and recommending deployment strategies to a batch of requests. (2) The Workforce Analytics Engine analyzes the available workforce and feeds to the Recommendation Engine, as the deployed tasks are to be undertaken by the workers. The outputs of this engine are estimations of workers' preferences or human factors, such as availability of the workers, as precise (discrete), or imprecise (intervals or probability distribution functions) information. (3) The Result Aggregation Module estimates the quality of the deployed tasks, and then feeds to the other two engines for readjustment. It is empowered by fully automated or hybrid algorithms that sparingly involve human intelligence inside machine algorithms. The development plan of SLOAN involves principled modeling, rigorous algorithm design, declarative framework development, deployment and integration inside multiple real world platforms. Different components of SLOAN are empowered with multi-objective discrete optimization and computational geometric algorithms, as well as multi-faceted modeling techniques adapted from machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.