CAREER: Streamlining Task Deployment on Crowdsourcing Platforms

众包利用在线基础设施，挖掘尚未开发且丰富多样的人类知识和认知库，以解决各种被认为很难由机器单独解决的任务。众包系统建立在私人或公共平台上，是部署各种需要人工智能的任务的流行手段。在这样的平台上进行任务部署需要确定适当的部署策略，以满足由请求者提供的部署参数（作为质量、延迟和成本的阈值），还需要分析可用于执行部署任务的劳动力。到目前为止，任务部署仍然是一个艰苦的手动过程，因为在决定如何组织工作人员、以何种风格以及以何种结构来满足部署参数方面，请求者几乎没有帮助。因此，请求者和工作者大多被限制在一个平台上，因为部署过程不容易跨平台移植。该项目研究了众包生态系统中位于多个利益相关者之间的中间层，通过允许简单灵活的部署约束和目标规范来帮助请求者部署众包任务，然后根据这些规范推荐部署策略。因此，该系统的开发可以实现跨平台部署过程的可移植性和重用。为了实现这些目标，该项目开发了一个名为SLOAN（可伸缩的、声明式的、优化驱动的、自适应的和统一的）的中间件系统，其中包含三个集成组件：(1)部署策略推荐引擎经过优化以适应生态系统中的多个涉众，并负责建模和向一批请求推荐部署策略。(2)劳动力分析引擎分析可用的劳动力，并将其馈送给推荐引擎，因为部署的任务将由工人承担。该引擎的输出是对工人偏好或人为因素的估计，例如工人的可用性，作为精确（离散）或不精确（间隔或概率分布函数）的信息。(3)结果聚合模块对部署任务的质量进行估计，然后反馈给其他两个引擎进行重新调整。它是由完全自动化或混合算法授权的，这些算法很少涉及机器算法中的人类智能。SLOAN的开发计划包括原则性建模、严格的算法设计、声明性框架开发、在多个真实世界平台内的部署和集成。斯隆的不同组件具有多目标离散优化和计算几何算法，以及适应机器学习的多方面建模技术。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Crowdsourcing leverages online infrastructure to tap an under-explored and richly heterogeneous pool of human knowledge and cognition for solving a variety of tasks that are otherwise considered hard for machines to solve alone. Crowdsourcing systems are built on private or public platforms and are a popular means of deploying a variety of tasks that require human intelligence. Task deployment on such platforms requires identifying appropriate deployment strategies to satisfy deployment parameters, provided by requesters as thresholds on quality, latency, and cost, and also requires analysis of the workforce that is available to undertake the deployed tasks. To date, task deployment remains a painstakingly manual process, as there is little to no help for requesters in deciding how to organize the workforce, in what style, and in what structure to satisfy deployment parameters. Consequently, requesters and workers are mostly confined to one platform, as there is no easy portability of deployment processes across platforms. This project investigates a middle layer that sits between multiple stakeholders in a crowdsourcing ecosystem to aid requesters in deploying crowdsourcing tasks by allowing easy and flexible specification of deployment constraints and goals, and then recommending deployment strategies based on those specifications. Development of this system thus enables the portability and reuse of deployment processes across platforms.To achieve these goals, this project develops a middleware system called SLOAN (Scalable, decLarative, Optimization-driven, Adaptive, and uNified) with three integrated components: (1) The Deployment Strategy Recommendation Engine is optimized to accommodate multi-stakeholders in the ecosystem, and is responsible for modeling and recommending deployment strategies to a batch of requests. (2) The Workforce Analytics Engine analyzes the available workforce and feeds to the Recommendation Engine, as the deployed tasks are to be undertaken by the workers. The outputs of this engine are estimations of workers' preferences or human factors, such as availability of the workers, as precise (discrete), or imprecise (intervals or probability distribution functions) information. (3) The Result Aggregation Module estimates the quality of the deployed tasks, and then feeds to the other two engines for readjustment. It is empowered by fully automated or hybrid algorithms that sparingly involve human intelligence inside machine algorithms. The development plan of SLOAN involves principled modeling, rigorous algorithm design, declarative framework development, deployment and integration inside multiple real world platforms. Different components of SLOAN are empowered with multi-objective discrete optimization and computational geometric algorithms, as well as multi-faceted modeling techniques adapted from machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.