Adaptive novelty detection with false discovery rate guarantee
Adaptive novelty detection with false discovery rate guarantee
复制标题
具有错误发现率保证的自适应新颖性检测
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Étienne Roquain
中科院分区：
文献类型：
--
作者：
Ariane Marandon;Lihua Lei;D. Mary;Étienne Roquain
本文研究了半监督新颖性检测问题，其中研究人员可以使用一组“典型”测量。受多重测试和共形推理最新进展的推动，我们提出了 AdaDetect，这是一种灵活的方法，能够封装任何概率分类算法，并控制有限样本中检测到的新事物的错误发现率 (FDR)，除了可交换性之外，无需任何分布假设。与通常致力于预先指定的 p 值函数的经典 FDR 控制程序相比，AdaDetect 以数据自适应方式学习转换，将力量集中在区分内部值和异常值的方向上。受多重测试文献的启发，我们进一步提出了 AdaDetect 的变体，它们可以适应空值的比例，同时保持有限样本 FDR 控制。这些方法在合成数据集和现实世界数据集上进行了说明，包括在天体物理学中的应用。
This paper studies the semi-supervised novelty detection problem where a set of"typical"measurements is available to the researcher. Motivated by recent advances in multiple testing and conformal inference, we propose AdaDetect, a flexible method that is able to wrap around any probabilistic classification algorithm and control the false discovery rate (FDR) on detected novelties in finite samples without any distributional assumption other than exchangeability. In contrast to classical FDR-controlling procedures that are often committed to a pre-specified p-value function, AdaDetect learns the transformation in a data-adaptive manner to focus the power on the directions that distinguish between inliers and outliers. Inspired by the multiple testing literature, we further propose variants of AdaDetect that are adaptive to the proportion of nulls while maintaining the finite-sample FDR control. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.