The shape of and solutions to the MTurk quality crisis
The shape of and solutions to the MTurk quality crisis
复制标题
DOI：
10.1017/psrm.2020.6
复制
发表时间：
2020-10-01
影响因子：
3.9
通讯作者：
Winter, Nicholas J. G.
中科院分区：
文献类型：
--
作者：
Kennedy, Ryan;Clifford, Scott;Winter, Nicholas J. G.
亚马逊的Mechanical Turk被广泛用于数据收集;然而，由于使用虚拟私人服务器以欺诈方式获取研究数据，数据质量可能会下降。不幸的是，我们对这种欺诈的规模和后果知之甚少，社会科学家发现和预防这种欺诈的工具也不发达。我们首先分析了38项研究，表明这种欺诈行为并不新鲜，但最近有所增加。然后，我们表明，这些欺诈的受访者提供特别低质量的数据，可以削弱治疗效果。最后，我们提供了两个解决方案：一个易于使用的应用程序，用于识别现有数据集中的欺诈行为，以及一种在Qualtrics调查中阻止欺诈受访者的方法。
Amazon's Mechanical Turk is widely used for data collection; however, data quality may be declining due to the use of virtual private servers to fraudulently gain access to studies. Unfortunately, we know little about the scale and consequence of this fraud, and tools for social scientists to detect and prevent this fraud are underdeveloped. We first analyze 38 studies and show that this fraud is not new, but has increased recently. We then show that these fraudulent respondents provide particularly low-quality data and can weaken treatment effects. Finally, we provide two solutions: an easy-to-use application for identifying fraud in the existing datasets and a method for blocking fraudulent respondents in Qualtrics surveys.