Pruning Adversarially Robust Neural Networks without Adversarial Examples-猫眼课题宝

DOI：

10.1109/icdm54844.2022.00120

发表时间：

2022-10

期刊：

2022 IEEE International Conference on Data Mining (ICDM)

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

T. Jian;Zifeng Wang;Yanzhi Wang;Jennifer G. Dy;Stratis Ioannidis

对抗性剪枝可以压缩模型，同时保持鲁棒性。当前的方法需要在修剪过程中访问对抗性示例。这极大地影响了训练效率。此外，随着新的对抗性攻击和训练方法的快速发展，对抗性剪枝方法需要进行相应的修改以跟上。在这项工作中，我们提出了一种新颖的框架来修剪先前训练的鲁棒神经网络，同时保持对抗性鲁棒性，而无需进一步生成对抗性示例。我们利用并发的自蒸馏和剪枝来保留原始模型中的知识，并通过希尔伯特-施密特信息瓶颈对剪枝后的模型进行正则化。我们全面评估了我们提出的框架，并在针对 MNIST、CIFAR-10 和 CIFAR-100 数据集针对五种最先进的攻击进行修剪训练的架构时，展示了其在对抗鲁棒性和效率方面的卓越性能。

Adversarial pruning compresses models while preserving robustness. Current methods require access to adversarial examples during pruning. This significantly hampers training efficiency. Moreover, as new adversarial attacks and training methods develop at a rapid rate, adversarial pruning methods need to be modified accordingly to keep up. In this work, we propose a novel framework to prune a previously trained robust neural network while maintaining adversarial robustness, without further generating adversarial examples. We leverage concurrent self-distillation and pruning to preserve knowledge in the original model as well as regularizing the pruned model via the Hilbert-Schmidt Information Bottleneck. We comprehensively evaluate our proposed framework and show its superior performance in terms of both adversarial robustness and efficiency when pruning architectures trained on the MNIST, CIFAR-10, and CIFAR-100 datasets against five state-of-the-art attacks..

Pruning Adversarially Robust Neural Networks without Adversarial Examples

文献摘要

相似文献