How Does Mixup Help With Robustness and Generalization?
How Does Mixup Help With Robustness and Generalization?
复制标题
DOI：
--
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Linjun Zhang;Zhun Deng;Kenji Kawaguchi;Amirata Ghorbani;James Y. Zou
中科院分区：
文献类型：
--
作者：
Linjun Zhang;Zhun Deng;Kenji Kawaguchi;Amirata Ghorbani;James Y. Zou
Mixup是一种流行的数据增强技术，它基于对示例及其标签的凸组合。这种简单的技术已被证明可以大大提高训练模型的鲁棒性和泛化性。然而，为什么会出现这样的改善还不太清楚。在本文中，我们提供理论分析来证明在训练中使用Mixup如何有助于模型的鲁棒性和泛化。对于鲁棒性，我们证明最小化混合损失对应于近似最小化对抗损失的上界。这解释了为什么通过Mixup训练获得的模型对几种对抗性攻击（如快速梯度符号方法（FGSM））具有鲁棒性。为了推广，我们证明了混合增强对应于一种特定类型的数据自适应正则化，它减少了过拟合。我们的分析为理解Mixup提供了新的见解和框架。
Mixup is a popular data augmentation technique based on taking convex combinations of pairs of examples and their labels. This simple technique has been shown to substantially improve both the robustness and the generalization of the trained model. However, it is not well-understood why such improvement occurs. In this paper, we provide theoretical analysis to demonstrate how using Mixup in training helps model robustness and generalization. For robustness, we show that minimizing the Mixup loss corresponds to approximately minimizing an upper bound of the adversarial loss. This explains why models obtained by Mixup training exhibits robustness to several kinds of adversarial attacks such as Fast Gradient Sign Method (FGSM). For generalization, we prove that Mixup augmentation corresponds to a specific type of data-adaptive regularization which reduces overfitting. Our analysis provides new insights and a framework to understand Mixup.