Approaches toward External Validity of Randomized Controlled Trials in the Social Sciences

社会科学家和决策者一直感兴趣的是，他们是否以及如何将随机对照试验（RCT）的结果推广到新的人群、地点和环境。这个外部有效性问题是循证决策的基础，因为研究人群和研究中心通常与干预措施可能扩大的现实人群和研究中心不同。本研究发展了统计方法，以提高社会科学中RCT的外部效度。该研究作出了两个方法上的贡献。第一个是设计外部效度RCT的新框架。具体而言，该项目开发了一种算法来选择实验地点，以便研究人员可以合理地估计可推广的因果效应。选址是必不可少的，因为社会科学的实验结果往往是异质性的地方，和有偏见的选择实验地点可能会导致低的普遍性和可复制性。第二个贡献是一个统计工具来量化外部效度偏差的鲁棒性。这种新的外部稳健性测量方法允许研究人员评估外部效度，即使在没有外部效度考虑的情况下进行RCT。这两种方法相互补充，共同提供了一个统一的管道，以提高社会科学中RCT的外部效度。该项目由科学的科学：发现，交流和影响计划，问责制，机构和行为计划，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的学术价值和更广泛的影响审查标准。

Social scientists and decision makers have been interested in whether and how well they can generalize the results of randomized controlled trials (RCTs) to new populations, places, and contexts. This question of external validity is a foundation of evidence-based decision making because study populations and sites are often different from the real-world populations and sites to which an intervention might be scaled up. This research develops statistical approaches to improve the external validity of RCTs in the social sciences. The research makes two methodological contributions. The first is a new framework to design RCTs for external validity. Specifically, this project develops an algorithm to select experimental sites such that researchers can credibly estimate generalizable causal effects. Site selection is essential because experimental results in the social sciences are often heterogeneous across places, and biased selection of experimental sites can lead to low generalizability and replicability. The second contribution is a statistical tool to quantify the robustness to external validity bias. This new measure of external robustness allows researchers to evaluate external validity even when RCTs were conducted without external validity considerations. These two methods complement each other and together provide a unified pipeline to improve the external validity of RCTs in the social sciences.The project is co-funded by the Science of Science: Discovery, Communications, and Impact program; the Accountability, Institutions and Behavior program; and the Office of Integrative Activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.