Amazon Mechanical Turk in Organizational Psychology: An Evaluation and Practical Recommendations
Amazon Mechanical Turk in Organizational Psychology: An Evaluation and Practical Recommendations
复制标题
DOI：
10.1007/s10869-016-9458-5
复制
发表时间：
2017-08-01
影响因子：
4.8
通讯作者：
Sliter, Michael
中科院分区：
文献类型：
--
作者：
Cheung, Janelle H.;Burns, Deanna K.;Sliter, Michael
在组织心理学研究界，亚马逊土耳其机器人是一个越来越受欢迎的数据源。本文对MTurk进行了评估，并为使用MTurk的研究人员提供了一套实用的建议。我们提出了与使用MTurk和有效性推断的潜在威胁相关的方法学问题的评估。基于我们的评估，我们也提供了一套建议，以加强有效性推断使用MTurk样本。尽管MTurk样本可以克服一些重要的有效性问题，但研究人员必须根据其研究目标考虑其他限制。研究人员应该根据我们在评估中讨论的不同问题仔细评估MTurk样本的适当性和质量。对于MTurk是否适合进行研究，并没有一个放之四海而皆准的答案。答案取决于研究问题和采用的数据收集和分析程序。数据的质量不是由数据源本身决定的，而是由研究人员在研究设计、数据收集和数据分析阶段做出的决定决定的。当前的论文扩展了文献评估MTurk在一个更全面的方式比以前的评论。过去的综述论文主要关注内部效度和外部效度，而对统计结论和结构效度的关注较少，而统计结论和结构效度对研究结果的准确推断同样重要。本文还提供了一组在使用MTurk时解决有效性问题的实用建议。
Amazon Mechanical Turk is an increasingly popular data source in the organizational psychology research community. This paper presents an evaluation of MTurk and provides a set of practical recommendations for researchers using MTurk.We present an evaluation of methodological concerns related to the use of MTurk and potential threats to validity inferences. Based on our evaluation, we also provide a set of recommendations to strengthen validity inferences using MTurk samples.Although MTurk samples can overcome some important validity concerns, there are other limitations researchers must consider in light of their research objectives. Researchers should carefully evaluate the appropriateness and quality of MTurk samples based on the different issues we discuss in our evaluation.There is not a one-size-fits-all answer to whether MTurk is appropriate for a research study. The answer depends on the research questions and the data collection and analytic procedures adopted. The quality of the data is not defined by the data source per se, but rather the decisions researchers make during the stages of study design, data collection, and data analysis.The current paper extends the literature by evaluating MTurk in a more comprehensive manner than in prior reviews. Past review papers focused primarily on internal and external validity, with less attention paid to statistical conclusion and construct validity-which are equally important in making accurate inferences about research findings. This paper also provides a set of practical recommendations in addressing validity concerns when using MTurk.