On the Transportability of Laboratory Results
On the Transportability of Laboratory Results
复制标题
DOI：
10.1177/0049124119826151
复制
发表时间：
2021-08-01
影响因子：
6.3
通讯作者：
Keuschnigg, Marc
中科院分区：
文献类型：
--
作者：
Bader, Felix;Baumeister, Bastian;Keuschnigg, Marc
实验室研究结果到原始实施之外的其他实例的“可移植性”需要观察到的行为率和估计的治疗效果对特定研究环境和所研究样本的变化的稳健性。在基于公平、信任和互惠的激励游戏的四项研究中，我们评估了（1）实验室结果对本地招募的学生受试者池的敏感性，（2）在线收集的行为数据以及在不同匿名条件下在实验室中收集的行为数据的可比性，（3）基于学生的结果对更广泛人群的普遍性，以及（4）通过在 Amazon Mechanical Turk 进行复制，实验室结果在不同研究背景下的稳定性。对于使用激励游戏作为亲社会行为测量工具的实验室设计类别，我们发现行为率和决策情况之间的确切行为差异不会超出特定的实现范围。最明显的是，从标准参与者池中获得的数据与从更广泛的人群中获得的数据存在显着差异。这破坏了使用基于经验的实验室研究来建立人类行为的描述性参数。相比之下，游戏之间行为差异的方向对于样本和设置的变化非常稳健。此外，我们没有发现任何证据表明匿名效应或模式效应可能会导致实验室测量产生偏差。这些结果强调了实验室实验在理论驱动设计中建立普遍因果效应的能力。
The "transportability" of laboratory findings to other instances than the original implementation entails the robustness of rates of observed behaviors and estimated treatment effects to changes in the specific research setting and in the sample under study. In four studies based on incentivized games of fairness, trust, and reciprocity, we evaluate (1) the sensitivity of laboratory results to locally recruited student-subject pools, (2) the comparability of behavioral data collected online and, under varying anonymity conditions, in the laboratory, (3) the generalizability of student-based results to the broader population, and (4) with a replication at Amazon Mechanical Turk, the stability of laboratory results across research contexts. For the class of laboratory designs using incentivized games as measurement instruments of prosocial behavior, we find that rates of behavior and the exact behavioral differences between decision situations do not transport beyond specific implementations. Most clearly, data obtained from standard participant pools differ significantly from those from the broader population. This undermines the use of empirically motivated laboratory studies to establish descriptive parameters of human behavior. Directions of the behavioral differences between games, in contrast, are remarkably robust to changes in samples and settings. Moreover, we find no evidence for either anonymity effects nor mode effects potentially biasing laboratory measurement. These results underscore the capacity of laboratory experiments to establish generalizable causal effects in theory-driven designs.