CHS: Small: Scientific Design of Interactive Human Computation Systems

人类计算系统是一种智能系统，它组织人类手动执行当前计算系统难以解决的计算过程，收集通常无法自动获得的常识性知识，或标记数据。将它们设计为娱乐功能，有望克服工人对货币补偿的需求，激励人群工人提供数据和解决方案，以换取非货币奖励。然而，早期的尝试通常没有发挥出它们的潜力，这在很大程度上是因为我们不了解设计决策将如何影响人类员工的表现和参与度。该项目将以三种方式发展人类计算系统的设计科学：(1)控制实验，研究现代商业社交和移动应用程序中常见的机制如何适应人类计算。(2)智能系统，可以自动对不同系统设计对人类工人的影响进行大规模实验。(3)研究系统自动学习的设计知识如何通过智能工具来支持人类软件设计人员。这项研究将产生(a)新的人类计算设计模式，(b)研究人类计算系统组成部分的新方法，(c)对这些模式如何影响人类群体工作人员绩效和参与的新理解，以及(d)开发包含智能创造力支持的设计工具的新方法。更好地理解设计考虑因素对用户的影响将使设计、开发和部署有效的系统变得更容易、更快。该研究将开发最优的实验设计算法，自动生成人类计算系统的变化，并对其对人类用户的影响进行测试，从而形成软件机制对群体工作人员绩效和参与度影响的贝叶斯模型。

Human computation systems are intelligent systems that organize humans to manually carry out computational processes that are too hard for current computational systems to solve, collect commonsense knowledge typically not available automatically, or label data. Designing them to serve recreational functions promises to overcome the need for worker monetary compensation, incentivizing crowd workers to generate data and solutions in exchange for non-monetary rewards. However, early attempts have generally not lived up to their potential, in large part because we do not understand how design decisions will impact human worker performance and engagement. This project will develop a science of design for human computation systems, in three ways: (1) Controlled experiments on how mechanics common to modern commercial social and mobile applications can be adapted to human computation. (2) An intelligent system that can automatically conduct large-scale experiments on the effects of different system designs on human workers. (3) Investigation of how the design knowledge automatically learned by the system can be used to support human software designers via intelligent tools.This research will result in (a) novel human computation design patterns, (b) new methodologies for studying the constituent parts of human computation systems, (c) new understanding of how those patterns affect human crowd worker performance and engagement, and (d) new approaches to the development of design tools that incorporate intelligent creativity support. An improved comprehension of the effects of design considerations on users will make it easier and quicker to design, develop, and deploy effective systems. The research will develop optimal experimental design algorithms that automatically generate variations of human computation systems and conduct tests of their effects on human users, resulting in a Bayesian model of the effects of software mechanics on crowd worker performance and engagement.