Interactive Methods for Data-Adaptive Multiple Testing

在现代应用中，非常大和复杂的数据集通常是在没有考虑特定研究问题的情况下收集的。相反，明确的目标是探索数据，寻找新的见解，发现它们可能不会被发现的关系和结构，然后报告这些发现的推论。该项目为交互式多重测试提供了一系列迭代算法框架，其中融合了探索性和验证性分析。这些框架具有极强的适应性：在过程的每一步，算法向做出指导过程的数据驱动决策的分析师透露更多数据。本项目的起点是PI最近在ADTER和STAR算法方面的工作，这些算法提供了灵活而强大的框架，用于控制FDR，同时利用辅助信息(ADAPT)或对拒绝集(STAR)实施约束(STAR)。给定每个p值的预测值，Adapt允许分析师使用任何机器学习方法从数据中交互地估计贝叶斯最优p值权重，同时可证明地控制有限样本FDR。STAR同样允许通用的交互建模，同时还保证拒绝集满足自然的结构约束，例如回归中的层次原则。这两种方法都通过可选的停止参数来保证FDR控制，为分析师提供了几乎完全的灵活性，以自适应地对数据进行建模。PI将致力于四个项目，通过开发新的交互方法和应用适应当前的科学问题来继续这一研究计划。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In modern applications, very large and complex data sets are routinely collected without a specific research question in mind. Rather, the express goal is to explore the data in search of novel insights, discovering relationships and structure they may not expected to be found, then report inferences for those findings. This project provides a series of iterative algorithmic frameworks for interactive multiple testing, which blend exploratory and confirmatory analyses. The frameworks are adaptive in an unusually strong sense: at each step of the procedure, the algorithm reveals more data to the analyst who makes data-driven decisions that guide the procedure.The starting point of this project is the PI's recent work on the AdaPT and STAR algorithms, which give flexible and powerful frameworks for controlling the FDR while exploiting side information (AdaPT) or enforcing constraints on the rejection set (STAR). Given predictor for each p-value, AdaPT lets analysts interactively estimate Bayes-optimal p-value weights from the data using any machine learning method, while provably controlling finite-sample FDR. The STAR likewise allows for generic interactive modeling while also guaranteeing that the rejection set satisfies natural structural constraints such as hierarchy principles in regression. Both methods guarantee FDR control by means of optional stopping arguments, giving near-total flexibility to the analyst to adaptively model the data. The PI will work on four projects that continue this research program by developing new interactive methods and applying AdaPT to current scientific problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.