Octopus: A Framework for Cost-Quality-Time Optimization in Crowdsourcing
Octopus: A Framework for Cost-Quality-Time Optimization in Crowdsourcing
复制标题
Octopus：众包中成本-质量-时间优化的框架
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Mausam
中科院分区：
文献类型：
--
作者：
Karan Goel;Shreya Rajpal;Mausam
我们推出了 Octopus，一种人工智能代理，可以在微型众包市场上共同平衡三个相互冲突的任务目标——工作质量、总成本和完成时间。以前的控制代理主要关注成本质量或成本时间权衡，而不是直接一致地控制这三者。三目标优化的简单表述是棘手的； Octopus 采用分层 POMDP 方法，由三个不同的组件负责设置每个任务的报酬、选择下一个任务以及控制任务级别的质量。我们证明，在实际实验中，Octopus 的性能显着优于现有的最先进方法。我们还在 Amazon Mechanical Turk 上部署了 Octopus，展示了它在现实世界的动态环境中管理任务的能力。
We present Octopus, an AI agent to jointly balance three conflicting task objectives on a micro-crowdsourcing marketplace – the quality of work, total cost incurred, and time to completion. Previous control agents have mostly focused on cost-quality, or cost-time tradeoffs, but not on directly controlling all three in concert. A naive formulation of three-objective optimization is intractable; Octopus takes a hierarchical POMDP approach, with three different components responsible for setting the pay per task, selecting the next task, and controlling task-level quality. We demonstrate that Octopus significantly outperforms existing state-of-the-art approaches on real experiments. We also deploy Octopus on Amazon Mechanical Turk, showing its ability to manage tasks in a real-world, dynamic setting.