Online panels in social science research: Expanding sampling methods beyond Mechanical Turk
Online panels in social science research: Expanding sampling methods beyond Mechanical Turk
复制标题
DOI：
10.3758/s13428-019-01273-7
复制
发表时间：
2019-10-01
影响因子：
5.4
通讯作者：
Litman, Leib
中科院分区：
文献类型：
--
作者：
Chandler, Jesse;Rosenzweig, Cheskie;Litman, Leib
亚马逊土耳其机器人（MTurk）被行为科学家广泛用于招募研究参与者。MTurk提供了传统的学生主题库的优势，但它也有重要的局限性。特别是，MTurk人口很少，而且可能被过度使用，行为科学家感兴趣的一些群体代表性不足，难以招募。在这里，我们研究了在线研究小组是否可以避免这些限制。具体来说，我们比较了样本组成，数据质量（测量的效果大小，内部可靠性和注意力检查），和非天真的参与者从MTurk和总理面板招募的在线研究小组的聚合。Prime Panel的参与者在年龄、家庭组成、宗教信仰、教育和政治态度方面更加多样化。Prime Panels参与者还报告了对经典方案的较少暴露，并产生了较大的效应量，但仅在筛选出几名筛选任务失败的参与者之后。我们的结论是，在线研究小组提供了一个独特的研究机会，但有一些重要的权衡。
Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of interest to behavioral scientists are underrepresented and difficult to recruit. Here we examined whether online research panels can avoid these limitations. Specifically, we compared sample composition, data quality (measured by effect sizes, internal reliability, and attention checks), and the non-naivete of participants recruited from MTurk and Prime Panels-an aggregate of online research panels. Prime Panels participants were more diverse in age, family composition, religiosity, education, and political attitudes. Prime Panels participants also reported less exposure to classic protocols and produced larger effect sizes, but only after screening out several participants who failed a screening task. We conclude that online research panels offer a unique opportunity for research, yet one with some important trade-offs.