Interactive Methods for Data-Adaptive Multiple Testing

在现代应用中，通常在没有特定研究问题的情况下收集非常大且复杂的数据集。相反，明确的目标是探索数据以寻找新的见解，发现他们可能意想不到的关系和结构，然后报告这些发现的推论。该项目为交互式多重测试提供了一系列迭代算法框架，其中混合了探索性和验证性分析。这些框架在非常强的意义上是自适应的：在过程的每一步，算法向做出数据驱动决策的分析人员揭示更多的数据，这些决策指导过程。该项目的起点是PI最近在AdaPT和STAR算法上的工作，它们提供了灵活而强大的框架，用于在利用侧信息（AdaPT）或对拒绝集（STAR）实施约束的同时控制FDR。给定每个p值的预测器，AdaPT允许分析师使用任何机器学习方法从数据中交互式地估计贝叶斯最优p值权重，同时可证明控制有限样本FDR。STAR同样允许通用的交互式建模，同时也保证拒绝集满足自然结构约束，如回归中的层次原则。这两种方法都通过可选的停止参数保证了FDR控制，为分析人员提供了几乎完全的灵活性来自适应地对数据建模。PI将开展四个项目，通过开发新的互动方法和将AdaPT应用于当前的科学问题来继续这项研究计划。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In modern applications, very large and complex data sets are routinely collected without a specific research question in mind. Rather, the express goal is to explore the data in search of novel insights, discovering relationships and structure they may not expected to be found, then report inferences for those findings. This project provides a series of iterative algorithmic frameworks for interactive multiple testing, which blend exploratory and confirmatory analyses. The frameworks are adaptive in an unusually strong sense: at each step of the procedure, the algorithm reveals more data to the analyst who makes data-driven decisions that guide the procedure.The starting point of this project is the PI's recent work on the AdaPT and STAR algorithms, which give flexible and powerful frameworks for controlling the FDR while exploiting side information (AdaPT) or enforcing constraints on the rejection set (STAR). Given predictor for each p-value, AdaPT lets analysts interactively estimate Bayes-optimal p-value weights from the data using any machine learning method, while provably controlling finite-sample FDR. The STAR likewise allows for generic interactive modeling while also guaranteeing that the rejection set satisfies natural structural constraints such as hierarchy principles in regression. Both methods guarantee FDR control by means of optional stopping arguments, giving near-total flexibility to the analyst to adaptively model the data. The PI will work on four projects that continue this research program by developing new interactive methods and applying AdaPT to current scientific problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.