"Improved Methodologies for Field Experiments: Maximizing Statistical Power While Promoting Replication."

该项目提出了分析随机对照试验（rct）数据的新方法，随机对照试验现在越来越多地用于经济学和其他社会科学的一些领域。研究人员通常希望使用随机对照试验来检验多个不同的假设。然而，进行这种数据分析需要仔细考虑统计问题；特别是，使用相同的数据运行多个不同的统计分析有可能增加误报的机会。因此，进行随机对照试验的经济学家越来越多地指定分析前计划；也就是说，他们在开始分析之前指定了他们计划如何分析数据。然而，这些计划确实限制了研究人员在开始统计分析之前重用数据和从数据中学习的能力。研究小组将开发一种潜在的机制，使研究人员可以在不事先指定每个假设的情况下从数据中学习。他们将使用机器学习和概率论的概念，为如何在给定的随机对照试验中最大化功率提供更好的框架。因此，研究结果可以在重要方面改进数据分析。研究小组有证据表明，在实践中，预分析计划包括对许多甚至数百种假设的测试。因此，当前的预分析方案设计很可能对经济利益的许多影响缺乏动力。该团队将扩展和应用生物统计学技术，包括把关、顺序测试和FDR控制，以设计预分析计划。他们还将开发一种分离样本方法，在这种方法中，研究人员对数据集的一部分进行探索性分析，并使用来自该部分数据的估计与可能机制的理论相结合，完善他们的假设，然后将假设子集注册到数据的其余部分进行测试。

This project proposes new methods for analyzing data from randomized controlled trials (RCTs), which are now increasingly used in some areas of economics and other social sciences. Researchers often want to use an RCT to test multiple different hypotheses. However, carrying out this kind of data analysis requires careful consideration of statistical issues; in particular, there is a chance that using the same data to run multiple different statistical analyses will increase the chance of false positives. As a result, economists conducting RCTs are increasingly specifying pre-analysis plans; that is, they are specifying how they plan to analyze the data before beginning the analysis. These plans, however, do limit researchers' ability to reuse data and learn from the data before beginning the statistical analysis. The research team will develop a potential mechanism for researchers to learn from the data without specifying every hypothesis in advance. They will use concepts from machine-learning as well as probability theory to provide a better framework for how to maximize power within a given RCT experiment. The results could therefore improve data analysis in important ways.The research team has evidence that in practice preanalysis plans include tests for many, even hundreds, of hypotheses. As a result current preanalysis plan designs are likely to be underpowered for many effects of economic interest. The team will extend and apply techniques from biostatistics, including gatekeeping, sequential testing, and FDR control to the design of preanalysis plans. They will also develop a split-sample approach in which researchers conduct exploratory analysis on one part of the data set, and, using estimates from that part of the data combined with theory over likely mechanisms, refine their hypotheses before registering a subset of hypotheses to be tested on the remaining part of the data.