Multiple Testing in Econometrics: Theory and Applications

本课题在多重检验方面进行了三个不同的项目：1.考虑S零假设的同时检验问题。解决这类问题的经典方法是要求控制家庭误码率(FWER)，即即使是一次错误拒绝的概率。不幸的是，当S个头很大时，这样的程序检测虚假零假设的能力可能非常有限。出于这个原因，在这种情况下，放松对FWER的控制通常是可取的。在与Joseph Romano和Michael Wolf的联合工作中，该项目为这个问题开发了渐近控制错误发现率(FDR)的方法，错误发现率是错误拒绝的比例的期望值(在没有拒绝的情况下定义为0)。与现有的控制FDR的方法不同，该项目开发的方法在确定拒绝哪些零假设时纳入了关于测试统计的联合分布的信息，因此能够更好地检测错误的零假设。该项目通过模拟研究和在对冲基金评估中的应用来说明这一性质。有关计划评估的文献通常侧重于评估一个计划对所有个人或所有接受治疗的个人的平均效果。当然，即使这些量为零，也可能是以观察到的协变量的某个值为条件的平均效应不为零的情况。确定这一点成立的协变量的值可能具有实质性的意义，特别是对于有兴趣将该计划或治疗扩大到其他人群的政策制定者。3.发展中国家项目的随机试验在发展经济学中越来越受欢迎。在许多情况下，该计划只涉及一种治疗，但有多种令人感兴趣的结果。一个突出的例子是Progresa，这是墨西哥于1998年开始的大规模、持续的减贫计划。现有的研究发现，当单独考虑每种结果时，Progresa对大量不同的结果有影响。然而，这些发现中的许多可能是由于虚假的拒绝，导致了对该计划效果的夸大。这位研究人员与李秀雄和Joanne Yoong一起，通过考虑多种测试来重新评估Progresa对这些不同结果的影响。更广泛的影响：上述项目不仅对经济学家有用，而且对范围广泛的学科的研究人员也有用。在教育研究、微阵列数据分析、模型选择和植物育种等方面，已经提出了对FDR进行控制的许多应用。因此，如第一个项目所述，开发更强大的控制FDR的方法，将在对冲基金评估之外的许多应用中发挥作用。治疗效果的估计不仅是许多经济问题的中心，经济学中的大量文献证明了这一点，而且生物统计学中的许多问题也是如此。例如，一种新药或疗法的疗效可能因患者可观察到的协变量而异。确定新药或疗法对这些观察到的协变量中的哪些值有效可能是有意义的，因此第二个项目的结果将是相关的。最后，当然，随机试验不仅在发展经济学中很常见，而且在科学的各个领域都很常见。因此，在第三个项目中制定的评估Progresa的方法将适用于任何这样的随机试验，只要有一个以上的相关结果。

This project pursues three different projects on multiple testing:1. Consider the problem of testing s null hypotheses simultaneously. The classical approach to such a problem is to require control of the Familywise Error Rate (FWER), the probability of even one false rejection. Unfortunately, when s is large, the ability of such a procedure to detect false null hypotheses may be very limited. For this reason, it is often preferred in such situations to relax control of the FWER. In joint work with Joseph Romano and Michael Wolf, this project develops methods for this problem that asymptotically control the false discovery rate (FDR), the expected value of the fraction of rejections that are false rejections (defined to be 0 in the case of no rejections). Unlike existing methods for control of the FDR, the methods developed by this project incorporate information about the joint distribution of the test statistics when determining which null hypotheses to reject and, thus, are better able to detect false null hypotheses. The project illustrates this property via a simulation study and an application to the evaluation of hedge funds.2. The literature on program evaluation typically focuses on estimation of the average effect of a program for all individuals or for all treated individuals. Of course, even if these quantities are zero, it may be the case that the average effect conditional on some value of observed covariates is nonzero. Identifying the values of the covariates for which this is true may be of substantive interest, especially to policy makers interested in extending the program or treatment to other populations. 3. Randomized trials of programs in developing countries have become increasingly popular within development economics. In many cases, the program involves a single treatment, but there are multiple outcomes of interest. A prominent example of this is PROGRESA, a large-scale, on-going poverty reduction program in Mexico started in 1998. Existing studies have found that PROGRESA has an effect on a large number of different outcomes when each outcome is considered individually. Many of these findings may, however, be due to false rejections, leading to an overstatement of the effect of the program. Together with Soohyung Lee and Joanne Yoong, the investigator reevaluates the impact of PROGRESA on these different outcomes by accounting for the multiplicity of tests under consideration.Broader Impact: The projects described above will be useful not only to economists, but to researchers in a wide array of disciplines. Control of the FDR has been suggested in numerous applications, including, among others, educational studies, analysis of microarray data, model selection, and plant breeding. For this reason, the development of more powerful methods for control of the FDR, as described in the first project, will be of use in many applications beyond the evaluation of hedge funds. The estimation of treatment effects lies at the center of not only many economic questions, as evidenced by its large literature within economics, but also many questions in biostatistics. For example, the efficacy of a new drug or therapy may vary with the observable covariates of the patients. It may be of interest to determine for which values of these observed covariates the new drug or therapy is effective, so the results of the second project will therefore be relevant. Finally, randomized trials are, of course, common not only in development economics, but in all parts of the sciences. Hence, the methodology developed in the third project for the evaluation of PROGRESA will be relevant in any such randomized trial provided that there is more than one outcome of interest.