Self-Progressing Robust Training
Self-Progressing Robust Training
复制标题
DOI：
10.1609/aaai.v35i8.16874
复制
发表时间：
2020-12
期刊：
影响因子：
--
通讯作者：
Minhao Cheng;Pin-Yu Chen;Sijia Liu;Shiyu Chang;Cho-Jui Hsieh;Payel Das
中科院分区：
文献类型：
--
作者：
Minhao Cheng;Pin-Yu Chen;Sijia Liu;Shiyu Chang;Cho-Jui Hsieh;Payel Das
在新的甚至是敌对的环境下增强模型的健壮性是建立可信的机器学习系统的重要里程碑。目前的健壮性训练方法，如对抗性训练，在模型训练过程中显式地使用一种“攻击”(例如，L范数有界扰动)来生成对抗性样本，以提高对抗性。在本文中，我们采取了不同的视角，提出了一种新的框架萌芽，自进步式稳健训练。在模型训练过程中，Sprout通过我们提出的参数化标签平滑技术逐步调整训练标签的分布，使得训练不会产生攻击，并且更具可扩展性。我们还使用基于邻域风险最小化的一般公式来激励萌发，该公式包含了许多稳健的训练方法作为特例。与目前最先进的对抗性训练方法(PGD-L-INFINTY和TRATES)相比，在L-INFTY范数有界攻击和各种不变性测试下，Sprout具有更好的性能和更好的可伸缩性。我们的结果为可扩展、有效和攻击无关的健壮训练方法提供了新的线索。
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems. Current robust training methods such as adversarial training explicitly uses an ``attack'' (e.g., l_infty-norm bounded perturbation) to generate adversarial examples during model training for improving adversarial robustness. In this paper, we take a different perspective and propose a new framework SPROUT, self-progressing robust training. During model training, SPROUT progressively adjusts training label distribution via our proposed parametrized label smoothing technique, making training free of attack generation and more scalable. We also motivate SPROUT using a general formulation based on vicinity risk minimization, which includes many robust training methods as special cases. Compared with state-of-the-art adversarial training methods (PGD-l_infty and TRADES) under l_infty-norm bounded attacks and various invariance tests, SPROUT consistently attains superior performance and is more scalable to large neural networks. Our results shed new light on scalable, effective and attack-independent robust training methods.