The Influence of Crowd Type and Task Complexity on Crowdsourced Work Quality
The Influence of Crowd Type and Task Complexity on Crowdsourced Work Quality
复制标题
人群类型和任务复杂度对众包工作质量的影响
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Motomichi Toyama
中科院分区：
文献类型：
--
作者：
R. M. Borromeo;Thomas Laurent;Motomichi Toyama
随着众包使用的普及，确保众包工作质量的必要性被放大。虽然众包中的质量控制已经得到了广泛的研究，但现有的机制仍有可能得到改进，以考虑到影响质量的其他因素。然而，由于众包依赖于人，因此很难识别和考虑影响质量的所有因素。在这项研究中，我们对人群类型和任务复杂性对工作质量的影响进行了初步调查，方法是将简单和复杂版本的数据提取任务众包给有偿和无偿人群。然后，我们根据其与黄金标准数据集的相似性来衡量结果的质量。我们的实验表明，无论任务类型如何，无偿群体都会产生高质量的结果，而有偿群体在简单任务中会产生更好的结果。我们打算扩展我们的工作，整合现有的质量控制机制，并对更多不同的人群成员进行更多的实验。
As the use of crowdsourcing spreads, the need to ensure the quality of crowdsourced work is magnified. While quality control in crowdsourcing has been widely studied, established mechanisms may still be improved to take into account other factors that affect quality. However, since crowdsourcing relies on humans, it is difficult to identify and consider all factors affecting quality. In this study, we conduct an initial investigation on the effect of crowd type and task complexity on work quality by crowdsourcing a simple and more complex version of a data extraction task to paid and unpaid crowds. We then measure the quality of the results in terms of its similarity to a gold standard data set. Our experiments show that the unpaid crowd produces results of high quality regardless of the type of task while the paid crowd yields better results in simple tasks. We intend to extend our work to integrate existing quality control mechanisms and perform more experiments with more varied crowd members.