Mitigating robust overfitting via self-residual-calibration regularization
Mitigating robust overfitting via self-residual-calibration regularization
复制标题
通过自残差校准正则化减轻稳健过度拟合
DOI：
10.1016/j.artint.2023.103877
复制
发表时间：
2023
影响因子：
14.4
通讯作者：
Satoh Shin'ichi
中科院分区：
文献类型：
--
作者：
Liu Hong;Zhong Zhun;Sebe Nicu;Satoh Shin'ichi
对抗训练中的过拟合问题近年来引起了人工智能和机器学习领域研究人员的兴趣。为了解决这个问题，本文首先评估了几种校准方法在各种鲁棒模型上的防御性能。我们的分析和实验揭示了两个有趣的特性：1)校准良好的鲁棒模型会降低鲁棒模型的置信度；2)自然图像和对抗图像的可信度之间存在权衡。这些新特性为设计一种简单而有效的正则化（称为自残差校准（SRC））提供了直接的见解。提出的SRC计算与真值标签对应的对抗特征和自然特征之间的绝对残差。此外，我们利用弹球损失最小化它们之间的分位数残差，从而产生更鲁棒的正则化。大量的实验表明，我们的SRC可以有效地缓解过拟合问题，同时提高最先进模型的鲁棒性。重要的是，SRC是各种正则化方法的补充。当与他们结合在一起时，我们能够在AutoAttack基准排行榜上获得顶级表现。
Overfitting in adversarial training has attracted the interest of researchers in the community of artificial intelligence and machine learning in recent years. To address this issue, in this paper we begin by evaluating the defense performances of several calibration methods on various robust models. Our analysis and experiments reveal two intriguing properties:1) a well-calibrated robust model is decreasing the confidence of robust model; 2) there is a trade-off between the confidences of natural and adversarial images. These new properties offer a straightforward insight into designing a simple but effective regularization, called Self-Residual-Calibration (SRC). The proposed SRC calculates the absolute residual between adversarial and natural logit features corresponding to the ground-truth labels. Furthermore, we utilize the pinball loss to minimize the quantile residual between them, resulting in more robust regularization. Extensive experiments indicate that our SRC can effectively mitigate the overfitting problem while improving the robustness of state-of-the-art models. Importantly, SRC is complementary to various regularization methods. When combined with them, we are capable of achieving the top-rank performance on the AutoAttack benchmark leaderboard.
影响因子：
22.7
作者：
Goodfellow, Ian;McDaniel, Patrick;Papernot, Nicolas
通讯作者：
Papernot, Nicolas