Approaches toward External Validity of Randomized Controlled Trials in the Social Sciences

社会科学家和决策者一直对是否以及如何将随机对照试验（rct）的结果推广到新的人群、地点和环境感兴趣。外部有效性的问题是基于证据的决策的基础，因为研究人群和地点往往不同于现实世界的人群和地点，干预可能会扩大规模。本研究发展了统计方法来提高社会科学中随机对照试验的外部效度。该研究在方法论上有两个贡献。首先是设计rct外部效度的新框架。具体来说，这个项目开发了一种算法来选择实验地点，这样研究人员就可以可靠地估计可推广的因果效应。选址是必要的，因为社会科学的实验结果在不同的地方往往是异质的，有偏见的实验地点的选择可能导致较低的普遍性和可复制性。第二个贡献是一个统计工具来量化对外部效度偏差的稳健性。这种外部稳健性的新措施允许研究人员评估外部效度，即使在rct没有外部效度考虑的情况下进行。这两种方法相辅相成，共同为提高社会科学中随机对照试验的外部效度提供了统一的途径。该项目由科学之科学：发现、传播和影响计划共同资助；问责制、制度和行为项目；以及综合活动办公室。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Social scientists and decision makers have been interested in whether and how well they can generalize the results of randomized controlled trials (RCTs) to new populations, places, and contexts. This question of external validity is a foundation of evidence-based decision making because study populations and sites are often different from the real-world populations and sites to which an intervention might be scaled up. This research develops statistical approaches to improve the external validity of RCTs in the social sciences. The research makes two methodological contributions. The first is a new framework to design RCTs for external validity. Specifically, this project develops an algorithm to select experimental sites such that researchers can credibly estimate generalizable causal effects. Site selection is essential because experimental results in the social sciences are often heterogeneous across places, and biased selection of experimental sites can lead to low generalizability and replicability. The second contribution is a statistical tool to quantify the robustness to external validity bias. This new measure of external robustness allows researchers to evaluate external validity even when RCTs were conducted without external validity considerations. These two methods complement each other and together provide a unified pipeline to improve the external validity of RCTs in the social sciences.The project is co-funded by the Science of Science: Discovery, Communications, and Impact program; the Accountability, Institutions and Behavior program; and the Office of Integrative Activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.