DNR: A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNs
DNR: A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNs
复制标题
DOI：
10.1145/3394885.3431542
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Souvik Kundu;M. Nazemi;P. Beerel;M. Pedram
中科院分区：
文献类型：
--
作者：
Souvik Kundu;M. Nazemi;P. Beerel;M. Pedram
本文提出了一种动态网络重布线（DNR）方法来生成修剪深度神经网络（DNN）模型，该模型对对抗性攻击具有鲁棒性，同时在干净图像上保持较高的准确性。特别是，公开的DNR方法基于使用混合损失函数的统一约束优化公式，该混合损失函数将超高模型压缩与鲁棒对抗训练相结合。该训练策略基于混合损失函数计算的每层归一化动量动态调整层间连通性。与现有需要多次训练迭代的鲁棒修剪框架相比，本文提出的学习策略只需一次训练迭代即可实现总体目标修剪比，并且可以调优以支持不规则和结构化通道修剪。为了评估DNR的优点，在CIFAR-10和CIFAR-100上使用了两种被广泛接受的模型，即VGG16和ResNet-18，在Tiny-ImageNet上使用了VGG16。与基线未压缩模型相比，DNR在所有数据集上提供了超过20倍的压缩，并且在干净或对抗分类精度上没有显着下降。此外，我们的实验表明，DNR始终发现压缩模型具有更好的干净和对抗性图像分类性能，而不是通过最先进的替代方案实现的。我们的模型和测试代码可在https://github.com/ksouvik52/DNR_ASP_DAC2021上获得。
This paper presents a dynamic network rewiring (DNR) method to generate pruned deep neural network (DNN) models that are robust against adversarial attacks yet maintain high accuracy on clean images. In particular, the disclosed DNR method is based on a unified constrained optimization formulation using a hybrid loss function that merges ultra-high model compression with robust adversarial training. This training strategy dynamically adjusts inter-layer connectivity based on per-layer normalized momentum computed from the hybrid loss function. In contrast to existing robust pruning frameworks that require multiple training iterations, the proposed learning strategy achieves an overall target pruning ratio with only a single training iteration and can be tuned to support both irregular and structured channel pruning. To evaluate the merits of DNR, experiments were performed with two widely accepted models, namely VGG16 and ResNet-18, on CIFAR-10, CIFAR-100 as well as with VGG16 on Tiny-ImageNet. Compared to the baseline un-compressed models, DNR provides over 20× compression on all the datasets with no significant drop in either clean or adversarial classification accuracy. Moreover, our experiments show that DNR consistently finds compressed models with better clean and adversarial image classification performance than what is achievable through state-of-the-art alternatives. Our models and test codes are available at https://github.com/ksouvik52/DNR_ASP_DAC2021.