Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
复制标题
DOI：
--
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Aditi Raghunathan;Sang Michael Xie;Fanny Yang;John C. Duchi;Percy Liang
中科院分区：
文献类型：
--
作者：
Aditi Raghunathan;Sang Michael Xie;Fanny Yang;John C. Duchi;Percy Liang
对抗训练通过扰动来增加训练集，以提高鲁棒误差（在最坏情况下的扰动），但它通常会导致标准误差的增加（在未扰动的测试输入上）。以前对这种权衡的解释依赖于假设类中没有预测因子具有低标准和鲁棒误差的假设。在这项工作中，我们精确地描述了线性回归中的最佳线性预测具有零标准误差和鲁棒误差时，增广对标准误差的影响。特别是，我们表明，标准误差可能会增加，即使当增广扰动有无噪声的最佳线性预测的意见。然后，我们证明了最近提出的鲁棒自训练（ESTA）估计提高了鲁棒误差，而不牺牲无噪声线性回归的标准误差。从经验上讲，对于神经网络，我们发现使用不同对抗性训练方法的训练可以改善CIFAR-10中随机和对抗性旋转以及对抗性$\ell_\infty$扰动的标准误差和鲁棒误差。
Adversarial training augments the training set with perturbations to improve the robust error (over worst-case perturbations), but it often leads to an increase in the standard error (on unperturbed test inputs). Previous explanations for this tradeoff rely on the assumption that no predictor in the hypothesis class has low standard and robust error. In this work, we precisely characterize the effect of augmentation on the standard error in linear regression when the optimal linear predictor has zero standard and robust error. In particular, we show that the standard error could increase even when the augmented perturbations have noiseless observations from the optimal linear predictor. We then prove that the recently proposed robust self-training (RST) estimator improves robust error without sacrificing standard error for noiseless linear regression. Empirically, for neural networks, we find that RST with different adversarial training methods improves both standard and robust error for random and adversarial rotations and adversarial $\ell_\infty$ perturbations in CIFAR-10.