Conducting behavioral research on Amazon's Mechanical Turk
Conducting behavioral research on Amazon's Mechanical Turk
复制标题
DOI：
10.3758/s13428-011-0124-6
复制
发表时间：
2012-03-01
影响因子：
5.4
通讯作者：
Suri, Siddharth
中科院分区：
文献类型：
--
作者：
Mason, Winter;Suri, Siddharth
亚马逊的 Mechanical Turk 是一个在线劳动力市场，申请者可以在这里发布工作，工人可以选择从事哪些工作来获取报酬。本文的中心目的是演示如何使用该网站进行行为研究，并降低可以从该平台受益的研究人员的进入门槛。我们描述了适用于跨学科的各种类型的研究和实验的通用技术。我们首先讨论在 Mechanical Turk 上进行实验的一些优点，例如轻松访问大型、稳定且多样化的主题库、实验成本低廉以及开发理论和执行实验之间的更快迭代。虽然进行行为研究的其他方法可能在上述一个或多个轴上与 Mechanical Turk 相当甚至更好，但我们将证明，当作为一个整体时， Mechanical Turk 对于许多研究人员来说可能是一个有用的工具。我们将讨论工人的行为与专家和实验室受试者的行为相比如何。然后我们将说明在 Mechanical Turk 上放置任务的机制，包括招募受试者、执行任务以及审查提交的工作。我们还为研究人员在该平台上进行研究时可能遇到的常见问题提供解决方案，包括进行同步实验的技术、确保高质量工作的方法、如何保持数据私密性以及如何维护代码安全。
Amazon's Mechanical Turk is an online labor market where requesters post jobs and workers choose which jobs to do for pay. The central purpose of this article is to demonstrate how to use this Web site for conducting behavioral research and to lower the barrier to entry for researchers who could benefit from this platform. We describe general techniques that apply to a variety of types of research and experiments across disciplines. We begin by discussing some of the advantages of doing experiments on Mechanical Turk, such as easy access to a large, stable, and diverse subject pool, the low cost of doing experiments, and faster iteration between developing theory and executing experiments. While other methods of conducting behavioral research may be comparable to or even better than Mechanical Turk on one or more of the axes outlined above, we will show that when taken as a whole Mechanical Turk can be a useful tool for many researchers. We will discuss how the behavior of workers compares with that of experts and laboratory subjects. Then we will illustrate the mechanics of putting a task on Mechanical Turk, including recruiting subjects, executing the task, and reviewing the work that was submitted. We also provide solutions to common problems that a researcher might face when executing their research on this platform, including techniques for conducting synchronous experiments, methods for ensuring high-quality work, how to keep data private, and how to maintain code security.