Comparing the similarity of responses received from studies in Amazon's Mechanical Turk to studies conducted online and with direct recruitment.
Comparing the similarity of responses received from studies in Amazon's Mechanical Turk to studies conducted online and with direct recruitment.
复制标题
DOI：
10.1371/journal.pone.0121595
复制
发表时间：
2015
期刊：
影响因子：
3.7
通讯作者：
Zawieska K
中科院分区：
文献类型：
--
作者：
Bartneck C;Duenser A;Moltchanova E;Zawieska K
基于计算机和互联网的问卷调查已经成为人机交互研究和其他相关领域（如心理学和社会学）的标准工具。亚马逊的土耳其机器人（AMT）服务是一种招募参与者并进行特定类型实验的新方法。这项研究比较了通过AMT招募的参与者与通过在线论坛招募的参与者或直接在大学校园招募的参与者是否给出了不同的回答。此外，我们比较了在AMT内进行的研究与通过AMT招募参与者但使用外部在线问卷服务进行的研究相比，是否会产生不同的反应。本研究结果表明，通过AMT招募的参与者获得的结果与通过校园或在线论坛招募的参与者获得的结果存在统计学差异。然而，我们确实认为，这种差异是如此之小，以至于没有实际后果。在AMT内进行研究与使用在线问卷服务进行研究之间没有显著差异。与在校园和在线论坛条件下获得的结果相比，直接从AMT中获得的结果没有显著差异。这可能表明，对于招募参与者和进行研究而言，AMT是一种可行且经济的选择，因为与其他常用方法相比，使用AMT建立和运行研究通常需要更少的精力和时间。我们讨论了我们的发现以及使用AMT进行实证研究的局限性。
Computer and internet based questionnaires have become a standard tool in Human-Computer Interaction research and other related fields, such as psychology and sociology. Amazon’s Mechanical Turk (AMT) service is a new method of recruiting participants and conducting certain types of experiments. This study compares whether participants recruited through AMT give different responses than participants recruited through an online forum or recruited directly on a university campus. Moreover, we compare whether a study conducted within AMT results in different responses compared to a study for which participants are recruited through AMT but which is conducted using an external online questionnaire service. The results of this study show that there is a statistical difference between results obtained from participants recruited through AMT compared to the results from the participant recruited on campus or through online forums. We do, however, argue that this difference is so small that it has no practical consequence. There was no significant difference between running the study within AMT compared to running it with an online questionnaire service. There was no significant difference between results obtained directly from within AMT compared to results obtained in the campus and online forum condition. This may suggest that AMT is a viable and economical option for recruiting participants and for conducting studies as setting up and running a study with AMT generally requires less effort and time compared to other frequently used methods. We discuss our findings as well as limitations of using AMT for empirical studies.
登录
查看更多内容
影响因子：
5.4
作者：
Berinsky, Adam J.;Huber, Gregory A.;Lenz, Gabriel S.
通讯作者：
Lenz, Gabriel S.
影响因子：
12.6
作者：
Buhrmester, Michael;Kwang, Tracy;Gosling, Samuel D.
通讯作者：
Gosling, Samuel D.
影响因子：
3.7
作者：
Hultsch, DF;MacDonald, SWS;Dixon, RA
通讯作者：
Dixon, RA
影响因子：
2.3
作者：
Horton, John J.;Rand, David G.;Zeckhauser, Richard J.
通讯作者：
Zeckhauser, Richard J.
影响因子：
2
作者：
Hedeker, D
通讯作者：
Hedeker, D