Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
复制标题
DOI：
10.1017/psrm.2018.10
复制
发表时间：
2019-07-01
影响因子：
3.9
通讯作者：
Coppock, Alexander
中科院分区：
文献类型：
--
作者：
Coppock, Alexander
调查实验治疗效果评估在多大程度上适用于其他人群和背景？对便民样本进行的调查实验经常受到批评，理由是受试者与一般公众有足够的差异，使这类实验的结果在更广泛的范围内缺乏信息。然而，在存在适度的治疗效果异质性的情况下，这种担忧可能会得到缓解。我提供了一系列15个复制实验的证据，证明从方便样品(如亚马逊的机械土耳其人)获得的结果与从国家样品获得的结果相似。无论是这些实验中采用的治疗方法对许多受试者类型都有相似的反应，还是方便性和国家样本在治疗效果调节剂方面没有太大差异。使用有限的实验内异质性的证据，我证明了前者很可能是这样的。尽管样本中的背景特征差异很大，但这些实验中发现的效果似乎相对均匀。
To what extent do survey experimental treatment effect estimates generalize to other populations and contexts? Survey experiments conducted on convenience samples have often been criticized on the grounds that subjects are sufficiently different from the public at large to render the results of such experiments uninformative more broadly. In the presence of moderate treatment effect heterogeneity, however, such concerns may be allayed. I provide evidence from a series of 15 replication experiments that results derived from convenience samples like Amazon's Mechanical Turk are similar to those obtained from national samples. Either the treatments deployed in these experiments cause similar responses for many subject types or convenience and national samples do not differ much with respect to treatment effect moderators. Using evidence of limited within-experiment heterogeneity, I show that the former is likely to be the case. Despite a wide diversity of background characteristics across samples, the effects uncovered in these experiments appear to be relatively homogeneous.