Validating the demographic, political, psychological, and experimental results obtained from a new source of online survey respondents
Validating the demographic, political, psychological, and experimental results obtained from a new source of online survey respondents
复制标题
DOI：
10.1177/2053168018822174
复制
发表时间：
2019-01-28
影响因子：
2.7
通讯作者：
McClellan, Oliver A.
中科院分区：
文献类型：
--
作者：
Coppock, Alexander;McClellan, Oliver A.
研究人员越来越多地将在线便利样本作为调查回复的来源，这些样本收集起来既简单又便宜。随着对这些来源的依赖越来越大，人们对便利样本的使用也越来越担心，尤其是亚马逊的土耳其机器人。我们区分“外部效度”和理论相关性，后者是任何数据收集策略的更重要的理由。我们探索了一个替代来源的在线便利样本，清醒的Fulfilling交换，并评估其适用于在线调查实验研究。我们的出发点是Berinsky，Huber和伦茨在2012年的研究，该研究在受访者特征和治疗效果估计方面将亚马逊的Mechanical Turk与美国国家概率样本进行了比较。我们使用Lucid平台上的大样本调查回复复制了这些相同的分析。我们的研究结果表明，Lucid的人口统计学和实验结果与美国国家基准吻合良好，但旨在消除关于平价医疗法案的“死亡小组”谣言的实验性治疗除外。我们的结论是，从Lucid平台招募的受试者构成了一个适合评估许多社会科学理论的样本，可以作为目前在Mechanical Turk或其他类似平台上进行研究的许多学者的替代品。
Researchers have increasingly turned to online convenience samples as sources of survey responses that are easy and inexpensive to collect. As reliance on these sources has grown, so too have concerns about the use of convenience samples in general and Amazon's Mechanical Turk in particular. We distinguish between "external validity" and theoretical relevance, with the latter being the more important justification for any data collection strategy. We explore an alternative source of online convenience samples, the Lucid Fulcrum Exchange, and assess its suitability for online survey experimental research. Our point of departure is the 2012 study by Berinsky, Huber, and Lenz that compares Amazon's Mechanical Turk to US national probability samples in terms of respondent characteristics and treatment effect estimates. We replicate these same analyses using a large sample of survey responses on the Lucid platform. Our results indicate that demographic and experimental findings on Lucid track well with US national benchmarks, with the exception of experimental treatments that aim to dispel the "death panel" rumor regarding the Affordable Care Act. We conclude that subjects recruited from the Lucid platform constitute a sample that is suitable for evaluating many social scientific theories, and can serve as a drop-in replacement for many scholars currently conducting research on Mechanical Turk or other similar platforms.