CAREER: Online Multiple Hypothesis Testing: A Comprehensive Treatment

随着时间的推移，在技术和制药行业中测试大量假设是很常见的。作为后一种情况的一个例子，假设一个实验室正在试图开发一种治疗阿尔茨海默氏症等疾病的方法。这是一种复杂的疾病，不太可能找到一种对每个人都有效的单一疗法。更有可能的是，对这种药物的研究将持续数年，甚至数十年，每隔几个月，一种新药可能会通过临床试验来测试其疗效。当我们在测试某种药物是否比安慰剂更好时，我们不知道将来还要测试多少种药物（假设），但我们知道早期测试的结果。这是在线多假设检验所考虑的设置，这个项目的主题-随着时间的推移，大量的假设序列以在线方式进行测试，我们希望确保在这个过程中不会有太多的错误发现。一个错误的发现不仅会导致错误的希望，还会浪费数百万美元用于后续临床试验，并可能给患者带来更糟糕的结果。该项目旨在开发新的方法来测试这样一系列假设，以便在任何时候都能控制某些常见的错误度量。针对本科生和研究生的培训部分将通过计划中的跨学科教程/研讨会为新研究人员提供跨学科教育，并与K-12学生进行外联。离线多重测试的方法非常丰富，有大量的方法可以控制各种各样的错误度量，事实上，PI最近对文献做出了重大贡献。相比之下，在线多重测试文献较少开发。这项资助采取了全面和综合的方法，这将导致整个错误度量范围的新方法：全局零测试，家庭明智的错误率，错误发现率，错误覆盖率，以及错误发现比例的同时控制。PI已经在其中一些方面开展了初步工作。我们还将在R中开发一个公共软件包，沿着相关文档，以便更容易地吸收和应用这些方法。所有的方法都将伴随着严格的理论保证，这将是可取的，在上述制药应用。这一奖项反映了NSF的法定使命，并已被认为是值得支持的，通过评估使用基金会的知识价值和更广泛的影响审查标准。

It is common in the technological and pharmaceutical industries to test a large sequences of hypotheses over time. As an example in the latter case, suppose a lab is trying to develop a cure for a disease like Alzheimer's. This is a complex disease for which it is unlikely to find a single cure that works for everyone. It is much more likely that research on the drug will continue for years, if not decades, and every few months a new drug may be tested for its efficacy using a clinical trial. When we are testing whether a particular drug is any better than a placebo, we have no idea how many more drugs (hypotheses) we will test in the future, but we do know the results of the earlier tests. This is the setup considered by online multiple hypothesis testing, the topic of this project --- a large sequence of hypotheses are tested over time in an online fashion, and we would like to ensure that there are not too many false discoveries in this process just due to chance. A false discovery results not just in false hopes, but in millions of wasted dollars in follow up clinical trials, and possibly worse outcomes for patients. This project aims to develop novel methodology to test such a sequence of hypotheses so that certain common error metrics are controlled at any time. The training component for undergraduate and graduate students will prepare new researchers with inter-disciplinary education via the planned cross-disciplinary tutorials/workshops, and outreach to K-12 students.The methodology in offline multiple testing is rich, with a plethora of methods that control a wide variety of error metrics, and in fact the PI has contributed significantly to the literature recently. In contrast, the online multiple testing literature is less developed. This grant takes a holistic and comprehensive approach, that will result in new methods for a whole spectrum of error metrics: global null testing, family wise error rate, false discovery rate, false coverage rate, and simultaneous control of the false discovery proportion. The PI already has preliminary work on some of these fronts. We will also develop a public software package in R along with associated documentation to enable the easier assimilation and application of these methods. All methods will be accompanied by rigorous theoretical guarantees, is would be desirable in the aforementioned pharmaceutical application.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.