Training Uncertainty-Aware Classifiers with Conformalized Deep Learning
Training Uncertainty-Aware Classifiers with Conformalized Deep Learning
复制标题
DOI：
10.48550/arxiv.2205.05878
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Bat-Sheva Einbinder;Yaniv Romano;Matteo Sesia;Yanfei Zhou
中科院分区：
文献类型：
--
作者：
Bat-Sheva Einbinder;Yaniv Romano;Matteo Sesia;Yanfei Zhou
深度神经网络是检测数据中隐藏模式并利用它们进行预测的强大工具，但它们并不是为了理解不确定性和估计可靠的概率而设计的。特别是，他们往往过于自信。我们开始，以解决这个问题的背景下，多类分类，开发一种新的训练算法，生产模型更可靠的不确定性估计，而不牺牲预测能力。这个想法是通过最小化损失函数来减轻过度自信，该损失函数受到共形推理的启发，通过仔细利用保持数据来量化模型的不确定性。合成和真实的数据的实验表明，该方法可以导致更小的共形预测集与更高的条件覆盖率，与保持数据的精确校准后，相比，国家的最先进的替代品。
Deep neural networks are powerful tools to detect hidden patterns in data and leverage them to make predictions, but they are not designed to understand uncertainty and estimate reliable probabilities. In particular, they tend to be overconfident. We begin to address this problem in the context of multi-class classification by developing a novel training algorithm producing models with more dependable uncertainty estimates, without sacrificing predictive power. The idea is to mitigate overconfidence by minimizing a loss function, inspired by advances in conformal inference, that quantifies model uncertainty by carefully leveraging hold-out data. Experiments with synthetic and real data demonstrate this method can lead to smaller conformal prediction sets with higher conditional coverage, after exact calibration with hold-out data, compared to state-of-the-art alternatives.