Approaches toward External Validity of Randomized Controlled Trials in the Social Sciences

社会科学家和决策者一直感兴趣的是，他们是否能够以及在多大程度上将随机对照试验(RCT)的结果推广到新的人群、地点和背景。外部有效性问题是基于证据的决策的基础，因为研究人群和地点往往与干预可能扩大到的真实世界人口和地点不同。本研究发展统计方法以提高随机对照研究在社会科学领域的外部效度。本研究在方法论上有两点贡献。第一个是设计RCT外部效度的新框架。具体地说，该项目开发了一种选择实验点的算法，以便研究人员能够可信地估计可概括的因果效应。选址是至关重要的，因为社会科学的实验结果往往在不同的地方是不同的，而偏颇地选择实验点可能会导致推广能力和可复制性较低。第二个贡献是量化对外部有效性偏差的稳健性的统计工具。这种新的外部稳健性测量允许研究人员评估外部效度，即使在进行随机对照试验时没有考虑外部效度。这两种方法相辅相成，共同提供了一个统一的渠道，以提高随机对照试验在社会科学中的外部有效性。该项目由科学科学：发现、交流和影响计划；责任、机构和行为计划；以及综合活动办公室共同资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Social scientists and decision makers have been interested in whether and how well they can generalize the results of randomized controlled trials (RCTs) to new populations, places, and contexts. This question of external validity is a foundation of evidence-based decision making because study populations and sites are often different from the real-world populations and sites to which an intervention might be scaled up. This research develops statistical approaches to improve the external validity of RCTs in the social sciences. The research makes two methodological contributions. The first is a new framework to design RCTs for external validity. Specifically, this project develops an algorithm to select experimental sites such that researchers can credibly estimate generalizable causal effects. Site selection is essential because experimental results in the social sciences are often heterogeneous across places, and biased selection of experimental sites can lead to low generalizability and replicability. The second contribution is a statistical tool to quantify the robustness to external validity bias. This new measure of external robustness allows researchers to evaluate external validity even when RCTs were conducted without external validity considerations. These two methods complement each other and together provide a unified pipeline to improve the external validity of RCTs in the social sciences.The project is co-funded by the Science of Science: Discovery, Communications, and Impact program; the Accountability, Institutions and Behavior program; and the Office of Integrative Activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.