Open Category Detection with PAC Guarantees
Open Category Detection with PAC Guarantees
复制标题
DOI：
--
复制
发表时间：
2018-07
期刊：
影响因子：
--
通讯作者：
Si Liu;Risheek Garrepalli;Thomas G. Dietterich;Alan Fern;Dan Hendrycks
中科院分区：
文献类型：
--
作者：
Si Liu;Risheek Garrepalli;Thomas G. Dietterich;Alan Fern;Dan Hendrycks
开放类别检测是检测属于训练数据中不存在的类别或类的“外来”测试实例的问题。在许多应用中，可靠地检测这种外星人是确保测试集预测的安全性和准确性的核心。不幸的是，在一般假设下，没有算法为它们检测外星人的能力提供理论上的保证。此外，虽然有开放类别检测的算法，但很少有经验结果直接报告外星人的检测率。因此，在我们对开放类别检测的理解中存在着显著的理论和经验差距。在本文中，我们通过研究一种简单但实用的开放类别检测变体来解决这一差距。在我们的设置中，我们被提供了一个只包含感兴趣的目标类别的“干净的”训练集，以及一个包含少量外来例子的未标记的“受污染的”训练集。在已知$\α$上界的假设下，我们开发了一种算法，该算法具有PAC风格的外星人检测率保证，同时以最小化错误警报为目标。在人工合成数据集和标准基准数据集上的实验结果证明了该算法的有效性，并为进一步的改进提供了一个基线。
Open category detection is the problem of detecting "alien" test instances that belong to categories or classes that were not present in the training data. In many applications, reliably detecting such aliens is central to ensuring the safety and accuracy of test set predictions. Unfortunately, there are no algorithms that provide theoretical guarantees on their ability to detect aliens under general assumptions. Further, while there are algorithms for open category detection, there are few empirical results that directly report alien detection rates. Thus, there are significant theoretical and empirical gaps in our understanding of open category detection. In this paper, we take a step toward addressing this gap by studying a simple, but practically-relevant variant of open category detection. In our setting, we are provided with a "clean" training set that contains only the target categories of interest and an unlabeled "contaminated" training set that contains a fraction $\alpha$ of alien examples. Under the assumption that we know an upper bound on $\alpha$, we develop an algorithm with PAC-style guarantees on the alien detection rate, while aiming to minimize false alarms. Empirical results on synthetic and standard benchmark datasets demonstrate the regimes in which the algorithm can be effective and provide a baseline for further advancements.