Asymmetric Error Control Under Imperfect Supervision: A Label-Noise-Adjusted Neyman–Pearson Umbrella Algorithm
Asymmetric Error Control Under Imperfect Supervision: A Label-Noise-Adjusted Neyman–Pearson Umbrella Algorithm
复制标题
DOI：
10.1080/01621459.2021.2016423
复制
发表时间：
2021-12
影响因子：
3.7
通讯作者：
Shu Yao;Bradley Rava;Xin Tong;Gareth M. James
中科院分区：
文献类型：
--
作者：
Shu Yao;Bradley Rava;Xin Tong;Gareth M. James
摘要 数据中的标签噪声长期以来一直是监督学习应用中的一个重要问题，因为它影响许多广泛使用的分类方法的有效性。最近，医学诊断和网络安全等重要的现实应用重新引起了人们对 Neyman-Pearson (NP) 分类范式的兴趣，该范式将更严重的错误类型（例如 I 类错误）限制在首选水平下，同时最小化其他类型的错误（例如 II 类错误）。然而，对于标签噪声下的NP范式的研究却很少。令人有些惊讶的是，即使常见的 NP 分类器在训练阶段忽略标签噪声，它们仍然能够以高概率控制 I 类错误。然而，他们付出的代价是 I 类错误的过度保守，从而导致功效显着下降（即 1 - II 类错误）。假设领域专家提供了腐败严重程度的下限，我们提出了第一个理论支持的算法，该算法将大多数最先进的分类方法应用于 NP 范式下的训练标签噪声。由此产生的分类器不仅能够以高概率将 I 类错误控制在所需水平以下，而且还提高了功效。
Abstract Label noise in data has long been an important problem in supervised learning applications as it affects the effectiveness of many widely used classification methods. Recently, important real-world applications, such as medical diagnosis and cybersecurity, have generated renewed interest in the Neyman–Pearson (NP) classification paradigm, which constrains the more severe type of error (e.g., the Type I error) under a preferred level while minimizing the other (e.g., the Type II error). However, there has been little research on the NP paradigm under label noise. It is somewhat surprising that even when common NP classifiers ignore the label noise in the training stage, they are still able to control the Type I error with high probability. However, the price they pay is excessive conservativeness of the Type I error and hence a significant drop in power (i.e., 1 - Type II error). Assuming that domain experts provide lower bounds on the corruption severity, we propose the first theory-backed algorithm that adapts most state-of-the-art classification methods to the training label noise under the NP paradigm. The resulting classifiers not only control the Type I error with high probability under the desired level but also improve power.