When and How Mixup Improves Calibration
When and How Mixup Improves Calibration
复制标题
DOI：
--
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Linjun Zhang;Zhun Deng;Kenji Kawaguchi;James Y. Zou
中科院分区：
文献类型：
--
作者：
Linjun Zhang;Zhun Deng;Kenji Kawaguchi;James Y. Zou
在许多机器学习应用中，模型提供准确捕获其预测不确定性的置信度分数是很重要的。虽然现代学习方法在预测准确性方面取得了巨大成功，但生成校准的置信度分数仍然是一个重大挑战。Mixup是一种流行而简单的数据增强技术，基于对训练示例进行凸组合，已被经验发现可以显着提高不同应用程序的置信度校准。然而，Mixup何时以及如何帮助校准仍然是一个谜。在本文中，我们从理论上证明，Mixup提高校准\textit {高维}设置通过调查自然的统计模型。有趣的是，Mixup的校准优势随着模型容量的增加而增加。我们支持我们的理论与常见的架构和数据集上的实验。此外，我们还研究了Mixup如何在半监督学习中改进校准。虽然合并未标记的数据有时会使模型校准得更少，但添加Mixup训练可以缓解这个问题，并可证明可以改善校准。我们的分析为理解混淆和校准提供了新的见解和框架。
In many machine learning applications, it is important for the model to provide confidence scores that accurately capture its prediction uncertainty. Although modern learning methods have achieved great success in predictive accuracy, generating calibrated confidence scores remains a major challenge. Mixup, a popular yet simple data augmentation technique based on taking convex combinations of pairs of training examples, has been empirically found to significantly improve confidence calibration across diverse applications. However, when and how Mixup helps calibration is still a mystery. In this paper, we theoretically prove that Mixup improves calibration in \textit{high-dimensional} settings by investigating natural statistical models. Interestingly, the calibration benefit of Mixup increases as the model capacity increases. We support our theories with experiments on common architectures and datasets. In addition, we study how Mixup improves calibration in semi-supervised learning. While incorporating unlabeled data can sometimes make the model less calibrated, adding Mixup training mitigates this issue and provably improves calibration. Our analysis provides new insights and a framework to understand Mixup and calibration.