Deadwooding: Robust Global Pruning for Deep Neural Networks
Deadwooding: Robust Global Pruning for Deep Neural Networks
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Sawinder Kaur;Ferdinando Fioretto;Asif Salekin
中科院分区：
文献类型：
--
作者：
Sawinder Kaur;Ferdinando Fioretto;Asif Salekin
深度神经网络逼近高度复杂函数的能力是其成功的关键。然而，这一好处是以较大的模型尺寸为代价的，这对其在资源受限环境中的部署提出了挑战。剪枝是一种用于限制此问题的有效技术，但通常以降低精确度和对手的健壮性为代价。本文针对这些不足，提出了一种新的全局剪枝技术DeadWoding，它利用拉格朗日对偶方法来鼓励模型的稀疏性，同时保持精度和稳健性。由此得到的模型在稳健性和准确性方面明显优于最先进的研究。
The ability of Deep Neural Networks to approximate highly complex functions is key to their success. This benefit, however, comes at the expense of a large model size, which challenges its deployment in resource-constrained environments. Pruning is an effective technique used to limit this issue, but often comes at the cost of reduced accuracy and adversarial robustness. This paper addresses these shortcomings and introduces Deadwooding, a novel global pruning technique that exploits a Lagrangian Dual method to encourage model sparsity while retaining accuracy and ensuring robustness. The resulting model is shown to significantly outperform the state-of-the-art studies in measures of robustness and accuracy.