Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
复制标题
DOI：
10.1016/j.jesp.2017.01.006
复制
发表时间：
2017-05-01
影响因子：
3.5
通讯作者：
Acquisti, Alessandro
中科院分区：
文献类型：
--
作者：
Peer, Eyal;Brandimarte, Laura;Acquisti, Alessandro
Amazon Mechanical Turk (MTurk) 作为在线研究平台的成功是有代价的：MTurk 受到人口补充速度放缓和参与者日益不天真的困扰。最近，出现了许多替代平台，提供类似于 MTurk 的功能，但提供了接触新的、更幼稚的人群的机会。在调查了几个选项后，我们实证检验了两个这样的平台：CrowdFlower (CF) 和 Prolific Academic (ProA)。在两项研究中，我们发现与 MTurk 参与者相比，两个平台上的参与者都更加天真且不那么不诚实。在这三个平台中，CF 提供了最佳的响应率，但 CF 参与者未能通过更多的注意力检查问题，并且没有重现 ProA 和 MTurk 上复制的已知效果。此外，ProA 参与者生成的数据质量高于 CF，与 MTurk 相当。 ProA 和 CF 参与者也比 MTurk 参与者更加多样化。 (C) 2017 Elsevier Inc. 保留所有权利。
The success of Amazon Mechanical Turk (MTurk) as an online research platform has come at a price: MTurk has suffered from slowing rates of population replenishment, and growing participant non-naivety. Recently, a number of alternative platforms have emerged, offering capabilities similar to MTurk but providing access to new and more naive populations. After surveying several options, we empirically examined two such platforms, CrowdFlower (CF) and Prolific Academic (ProA). In two studies, we found that participants on both platforms were more naive and less dishonest compared to MTurk participants. Across the three platforms, CF provided the best response rate, but CF participants failed more attention-check questions and did not reproduce known effects replicated on ProA and MTurk. Moreover, ProA participants produced data quality that was higher than CF's and comparable to MTurk's. ProA and CF participants were also much more diverse than participants from MTurk. (C) 2017 Elsevier Inc. All rights reserved.