Network Pruning via Annealing and Direct Sparsity Control
Network Pruning via Annealing and Direct Sparsity Control
复制标题
DOI：
10.1109/ijcnn52387.2021.9533741
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Yangzi Guo;Yiyuan She;Adrian Barbu
中科院分区：
文献类型：
--
作者：
Yangzi Guo;Yiyuan She;Adrian Barbu
人工神经网络（ANN），特别是深度卷积神经网络，目前非常流行，并已被证明成功地为许多视觉问题提供了非常可靠的解决方案。然而，深度神经网络的使用受到其密集的计算和内存成本的广泛阻碍。在本文中，我们提出了一种新的有效的网络修剪框架，适用于非结构化和结构化的通道级修剪。我们提出的方法收紧稀疏性约束，逐渐删除网络参数或过滤器通道的基础上的标准和时间表。网络大小在整个迭代过程中不断下降的吸引人的事实使其适合于修剪任何未经训练或预先训练的网络。因为我们的方法使用$L_{0}$约束而不是$L_{1}$惩罚，所以它不会在训练参数或滤波器通道中引入任何偏差。此外，$L_{0}$约束使得在网络修剪过程中直接指定所需的稀疏级别变得容易。最后，在大量的合成和真实的视觉数据集上进行了实验验证，结果表明，与其他现有的网络剪枝方法相比，该方法具有更好的性能。
Artificial neural networks (ANNs) especially deep convolutional neural networks are very popular these days and have been proved to successfully offer quite reliable solutions to many vision problems. However, the use of deep neural networks is widely impeded by their intensive computational and memory cost. In this paper, we propose a novel efficient network pruning framework that is suitable for both non-structured and structured channel-level pruning. Our proposed method tightens a sparsity constraint by gradually removing network parameters or filter channels based on a criterion and a schedule. The attractive fact that the network size keeps dropping throughout the iterations makes it suitable for the pruning of any untrained or pre-trained network. Because our method uses a $L_{0}$ constraint instead of the $L_{1}$ penalty, it does not introduce any bias in the training parameters or filter channels. Furthermore, the $L_{0}$ constraint makes it easy to directly specify the desired sparsity level during the network pruning process. Finally, experimental validation on extensive synthetic and real vision datasets show that the proposed method obtains better or competitive performance compared to other states of art network pruning methods.