Multiple Testing in Econometrics: Theory and Applications

本项目在多重测试上追求三个不同的项目：1.考虑同时检验零假设的问题。解决这个问题的经典方法是要求控制FWER（Family Error Rate），即即使是一个错误拒绝的概率。不幸的是，当s较大时，这种过程检测错误零假设的能力可能非常有限。出于这个原因，在这种情况下通常优选放松对FWER的控制。在与Joseph Romano和Michael Wolf的联合工作中，该项目开发了解决此问题的方法，该方法渐进地控制错误发现率（FDR），即错误拒绝的拒绝分数的期望值（在没有拒绝的情况下定义为0）。与现有的FDR控制方法不同，该项目开发的方法在确定拒绝哪些零假设时，将有关检验统计量的联合分布的信息结合起来，从而能够更好地检测错误的零假设。该项目通过模拟研究和应用于对冲基金的评估来说明这一属性。有关项目评估的文献通常侧重于估计项目对所有个人或所有接受治疗的个人的平均效果。当然，即使这些量为零，也可能是这样的情况，即以观测协变量的某个值为条件的平均效应是非零的。确定协变量的值，这是真的可能是实质性的利益，特别是有兴趣的政策制定者在扩展程序或治疗到其他人群。3.在发展经济学中，在发展中国家进行的项目随机试验越来越受欢迎。在许多情况下，该计划涉及单一治疗，但有多个感兴趣的结果。这方面的一个突出例子是PROGRESA，这是墨西哥于1998年开始的一项大规模持续减贫方案。现有的研究发现，当单独考虑每一个结果时，PROGRESA对大量不同的结果都有影响。然而，这些发现中的许多可能是由于错误的拒绝，导致对该计划效果的夸大。研究人员与李秀亨和Joanne Yoong一起，通过考虑测试的多样性，重新评估了PROGRESA对这些不同结果的影响。更广泛的影响：上述项目不仅对经济学家有用，而且对广泛学科的研究人员也有用。FDR的控制已经在许多应用中提出，包括教育研究、微阵列数据分析、模型选择和植物育种等。因此，如第一个项目中所述，开发更强大的FDR控制方法将在对冲基金评估之外的许多应用中使用。治疗效果的估计不仅是许多经济学问题的核心，正如经济学中大量文献所证明的那样，而且也是生物统计学中的许多问题。例如，新药物或疗法的功效可能随着患者的可观察协变量而变化。确定新药或治疗对这些观察到的协变量的哪些值有效可能是有意义的，因此第二个项目的结果将是相关的。最后，随机试验当然不仅在发展经济学中很常见，而且在所有科学领域都很常见。因此，在第三个项目中开发的PROGRESA评价方法将与任何此类随机试验相关，前提是有一个以上的关注结局。

This project pursues three different projects on multiple testing:1. Consider the problem of testing s null hypotheses simultaneously. The classical approach to such a problem is to require control of the Familywise Error Rate (FWER), the probability of even one false rejection. Unfortunately, when s is large, the ability of such a procedure to detect false null hypotheses may be very limited. For this reason, it is often preferred in such situations to relax control of the FWER. In joint work with Joseph Romano and Michael Wolf, this project develops methods for this problem that asymptotically control the false discovery rate (FDR), the expected value of the fraction of rejections that are false rejections (defined to be 0 in the case of no rejections). Unlike existing methods for control of the FDR, the methods developed by this project incorporate information about the joint distribution of the test statistics when determining which null hypotheses to reject and, thus, are better able to detect false null hypotheses. The project illustrates this property via a simulation study and an application to the evaluation of hedge funds.2. The literature on program evaluation typically focuses on estimation of the average effect of a program for all individuals or for all treated individuals. Of course, even if these quantities are zero, it may be the case that the average effect conditional on some value of observed covariates is nonzero. Identifying the values of the covariates for which this is true may be of substantive interest, especially to policy makers interested in extending the program or treatment to other populations. 3. Randomized trials of programs in developing countries have become increasingly popular within development economics. In many cases, the program involves a single treatment, but there are multiple outcomes of interest. A prominent example of this is PROGRESA, a large-scale, on-going poverty reduction program in Mexico started in 1998. Existing studies have found that PROGRESA has an effect on a large number of different outcomes when each outcome is considered individually. Many of these findings may, however, be due to false rejections, leading to an overstatement of the effect of the program. Together with Soohyung Lee and Joanne Yoong, the investigator reevaluates the impact of PROGRESA on these different outcomes by accounting for the multiplicity of tests under consideration.Broader Impact: The projects described above will be useful not only to economists, but to researchers in a wide array of disciplines. Control of the FDR has been suggested in numerous applications, including, among others, educational studies, analysis of microarray data, model selection, and plant breeding. For this reason, the development of more powerful methods for control of the FDR, as described in the first project, will be of use in many applications beyond the evaluation of hedge funds. The estimation of treatment effects lies at the center of not only many economic questions, as evidenced by its large literature within economics, but also many questions in biostatistics. For example, the efficacy of a new drug or therapy may vary with the observable covariates of the patients. It may be of interest to determine for which values of these observed covariates the new drug or therapy is effective, so the results of the second project will therefore be relevant. Finally, randomized trials are, of course, common not only in development economics, but in all parts of the sciences. Hence, the methodology developed in the third project for the evaluation of PROGRESA will be relevant in any such randomized trial provided that there is more than one outcome of interest.