Unlabeled Data Improves Adversarial Robustness
Unlabeled Data Improves Adversarial Robustness
复制标题
DOI：
--
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Y. Carmon;Aditi Raghunathan;Ludwig Schmidt;Percy Liang;John C. Duchi
中科院分区：
文献类型：
--
作者：
Y. Carmon;Aditi Raghunathan;Ludwig Schmidt;Percy Liang;John C. Duchi
我们从理论上和经验上证明，对抗鲁棒性可以从半监督学习中显着受益。理论上，我们重新审视 Schmidt 等人的简单高斯模型。这显示了标准分类和稳健分类之间的样本复杂性差距。我们证明，未标记的数据弥补了这一差距：简单的半监督学习程序（自训练）使用实现高标准精度所需的相同数量的标签来实现高鲁棒精度。根据经验，我们使用来自 8000 万张微小图像的 50 万张未标记图像来增强 CIFAR-10，并使用强大的自我训练，在以下方面超越最先进的鲁棒精度超过 5 个点：(i) 通过对抗性训练对多种强攻击的 $\ell_\infty$ 鲁棒性，以及 (ii) 通过随机平滑认证 $\ell_2$ 和 $\ell_\infty$ 鲁棒性。在 SVHN 上，添加数据集自己的额外训练集并删除标签可提供 4 到 10 点的增益，与使用额外标签的增益相差不到 1 点。
We demonstrate, theoretically and empirically, that adversarial robustness can significantly benefit from semisupervised learning. Theoretically, we revisit the simple Gaussian model of Schmidt et al. that shows a sample complexity gap between standard and robust classification. We prove that unlabeled data bridges this gap: a simple semisupervised learning procedure (self-training) achieves high robust accuracy using the same number of labels required for achieving high standard accuracy. Empirically, we augment CIFAR-10 with 500K unlabeled images sourced from 80 Million Tiny Images and use robust self-training to outperform state-of-the-art robust accuracies by over 5 points in (i) $\ell_\infty$ robustness against several strong attacks via adversarial training and (ii) certified $\ell_2$ and $\ell_\infty$ robustness via randomized smoothing. On SVHN, adding the dataset's own extra training set with the labels removed provides gains of 4 to 10 points, within 1 point of the gain from using the extra labels.