"Improved Methodologies for Field Experiments: Maximizing Statistical Power While Promoting Replication."

该项目提出了分析随机对照试验(RCT)数据的新方法，该方法现在越来越多地应用于经济学和其他社会科学的某些领域。研究人员经常希望使用随机对照试验来检验多个不同的假设。然而，进行这种数据分析需要仔细考虑统计问题；特别是，使用相同的数据进行多个不同的统计分析可能会增加误报的机会。因此，进行随机对照研究的经济学家越来越多地指定分析前计划；也就是说，他们正在指定他们计划如何在开始分析之前分析数据。然而，这些计划确实限制了研究人员在开始统计分析之前重复使用数据和从数据中学习的能力。研究团队将为研究人员开发一种潜在的机制，让他们在不事先指定每个假设的情况下，从数据中学习。他们将使用机器学习和概率论的概念，为如何在给定的RCT实验中最大化功率提供更好的框架。因此，研究结果可以在重要方面改善数据分析。研究小组有证据表明，在实践中，分析前计划包括对许多甚至数百个假设的测试。因此，当前的分析前计划设计很可能因经济利益的许多影响而动力不足。该团队将扩展和应用生物统计学的技术，包括把关、顺序测试和FDR控制，以设计分析前计划。他们还将开发一种分裂样本方法，研究人员对数据集的一部分进行探索性分析，并使用对该部分数据的估计与可能机制的理论相结合，在登记将在其余数据部分进行测试的假设子集之前完善他们的假设。

This project proposes new methods for analyzing data from randomized controlled trials (RCTs), which are now increasingly used in some areas of economics and other social sciences. Researchers often want to use an RCT to test multiple different hypotheses. However, carrying out this kind of data analysis requires careful consideration of statistical issues; in particular, there is a chance that using the same data to run multiple different statistical analyses will increase the chance of false positives. As a result, economists conducting RCTs are increasingly specifying pre-analysis plans; that is, they are specifying how they plan to analyze the data before beginning the analysis. These plans, however, do limit researchers' ability to reuse data and learn from the data before beginning the statistical analysis. The research team will develop a potential mechanism for researchers to learn from the data without specifying every hypothesis in advance. They will use concepts from machine-learning as well as probability theory to provide a better framework for how to maximize power within a given RCT experiment. The results could therefore improve data analysis in important ways.The research team has evidence that in practice preanalysis plans include tests for many, even hundreds, of hypotheses. As a result current preanalysis plan designs are likely to be underpowered for many effects of economic interest. The team will extend and apply techniques from biostatistics, including gatekeeping, sequential testing, and FDR control to the design of preanalysis plans. They will also develop a split-sample approach in which researchers conduct exploratory analysis on one part of the data set, and, using estimates from that part of the data combined with theory over likely mechanisms, refine their hypotheses before registering a subset of hypotheses to be tested on the remaining part of the data.