Theory and Methods for Multiple Testing and Inference

主要研究人员：Joseph P Romano命题：多重测试建议的理论和方法ID：DMS-0404979摘要本研究建议的主要目标是发展多重测试和推理问题的理论和方法论。处理多重性的一个经典方法是要求决策规则控制家庭错误率(FWER)，即拒绝至少一个真实假设的概率。但当测试数量很多时，对FWER的控制是如此严格，以至于替代假设几乎没有机会被发现。作为回应，Benjamini和Hochberg的错误发现率(FDR)得到了广泛的应用，研究者将考虑替代度量，如拒绝k个或更多真实假设的概率，或直接基于实际错误发现比例(FDP)的度量。对于每一种差错控制措施，都希望构建在最弱的可能假设下显示差错控制的过程。受差错控制的约束，这些程序在检测可选假设的能力方面应该是有效的。用来开发不依赖于不切实际或不可验证的模型假设的方法的主要方法将是使用引导、次抽样和其他计算机密集型方法。这些工具提供了可行的方法来获得有效的分布近似，而几乎不假设产生数据的随机机制。正如重抽样在单一推理的问题上取得了巨大的成功一样，它的使用也可以卓有成效地扩展到多重推理的问题上。虽然这种方法已经得到了一些成功的使用，但其全部潜力目前还没有实现，显然，在未来几年内将提出更有效和更广泛适用的方法。Bootstrap和相关方法的强大之处在于，可以捕获单个测试统计的联合依赖结构，因此方法不需要过于保守。对这种新方法的追求将从理论、计算和实践的角度进行调查。值得注意的是，当假设的数量与样本量相比很大时，研究人员将解决多重推理问题，方向性错误的公开问题，以及控制FDR的有效技术的构建，以及其他错误措施。实际上，任何科学实验都是为了回答关于调查过程的问题，这些问题通常可以正式地转化为一组假设。只有一个例外，即只考虑一种假设。此外，由于“数据窥探”(或“数据挖掘”)的影响，还出现了其他推理问题。统计学家面临着一个挑战，即如何解释复杂数据分析产生的所有可能的错误，以便得出的任何推断或有趣的结论都可以可靠地视为真实的结构，而不是随机数据的伪像。虽然处理同时推理数据问题的统计方法的历史可以追溯到至少半个世纪以前，但大多数经典技术通常依赖于强有力的假设，或者它们效率低下。在计算机和信息时代的推动下，人们对更可靠、更有效的多重测试方法的需求越来越大。例如，目前生物技术和基因组学中的方法产生了DNA微阵列实验，其中必须同时分析细胞中数千个基因的表达水平。类似的问题也出现在图像处理领域，如神经成像和计量经济学。现在，遇到由兆字节组成的信息的数据并不少见。因此，统计学家面临着新的挑战，即设计不是基于强有力的假设的技术，并且能够在存在大量数据的情况下有效地处理多样性问题。

Principal Investigator: Joseph P RomanoProposal Title: Theory and methods for Multiple Testing Proposal Id: DMS - 0404979AbstractThe main goal of this research proposal is the development of theory and methodology for problems in multiple testing and inference. A classical approach to dealing with multiplicity is to require decision rules that control the familywise error rate (FWER), the probability of rejecting at least one true hypothesis. But when the number of tests is large, control of the FWER is so stringent that alternative hypotheses have little chance of being detected. In response, the false discovery rate (FDR) of Benjamini and Hochberg has gained wide use.Alternative measures, such as the probability of rejecting k or more true hypotheses, or ones based directly of the actual false discovery proportion (FDP) will be considered by the investigator. For each measure of error control, it is desired to construct procedures that exhibit error control under the weakest possible assumptions. Subject to error control, the procedures should be efficient in their ability to detect alternative hypotheses. The main approach used to develop methods that do not rely on unrealistic or unverifiable model assumptions will be the use of the bootstrap, subsampling, and other computer-intensive methods. These tools offer viable approaches to obtaining valid distributional approximations while assuming very little about the stochastic mechanism generating the data. Just as resampling has been enormously successful in the case of questions of a single inference, its use can be extended fruitfully to questions of multiple inferences.While such an approach has been used with some success, its full potential is currently unrealized and it is clear that efficient and more broadly applicable methods will be advanced in the next few years. The power of the bootstrap and related methods is that the joint dependency structure of the individual test statistics can be captured so that methods need not be overly conservative. The pursuit of such new methodology will be investigated from theoretical, computational and practical points of view. Notably, the investigator will address the multiple inference problem when the number of hypotheses is large compared with sample size, the open problem of directional errors, as well as the construction of efficient techniques that control the FDR, as well as other measures of error.Virtually any scientific experiment sets out to answer questions about the process under investigation, which often can be translated formally into a set of hypotheses. It is the exception that a single hypothesis is considered. Moreover, due to effects of "data snooping" (or "data mining"), other inference questions arise as well.The statistician is then faced with the challenge of accounting for all possible errors resulting from a complex data analysis, so that any resulting inferences or interesting conclusions can reliably be viewed as real structure rather than artifacts of random data. While the history of statistical methods that deal with problems of simultaneous inference data back at least half a century, most of the classical techniques typically rely on strong assumptions, or they are inefficient. Driven by the advent of computers and the information age, there has been a growing demand for more reliable and efficient methods for multiple testing. For example, current methods in biotechnology and genomics generate DNA microarray experiments, where expression levels in cells for thousands of genes must be analyzed simultaneously. Similar problems arise in image processing, such as neuroimaging, and econometrics. It is now not uncommon to encounter data consisting of megabytes of information. Thus, the statistician is faced with new challenges of devising techniques that are not based on strong assumptions and can effectively deal with problems of multiplicity in the presence of vast amounts of data.