Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
复制标题
DOI：
--
复制
发表时间：
2017-09
期刊：
影响因子：
--
通讯作者：
Huan Wang;Qiming Zhang;Yuehai Wang;Haoji Hu
中科院分区：
文献类型：
--
作者：
Huan Wang;Qiming Zhang;Yuehai Wang;Haoji Hu
尽管深卷积神经网络(CNN)在各种计算机视觉任务中表现出了更好的性能，但它的应用受到存储和计算的显著增加的限制。在CNN简化技术中，参数剪枝是一种很有前途的方法，它的目的是在不大幅降低原始精度的情况下减少各层的权重。本文提出了一种新的渐进参数剪枝方法，称为结构化概率剪枝(SPP)，它以概率的方式有效地剪枝卷积层的权重。具体地说，与现有的确定性剪枝方法不同，SPP为每个权重引入了剪枝概率，并通过从剪枝概率中采样来指导剪枝。基于训练过程的重要性标准，设计了一种增加和减少剪枝概率的机制。实验表明，在加速4倍的情况下，SPP在AlexNet和VGG-16上的分类精度分别是AlexNet和VGG-16的0.3%和0.8%。此外，SPP可以直接用于加速多分支CNN网络，如ResNet，而不需要特定的适配。我们的2倍加速ResNet-50在ImageNet上仅损失前5名的0.8%的准确率。利用AlexNet在Flower-102数据集上进一步验证了该方法在迁移学习任务上的有效性。
Although deep Convolutional Neural Network (CNN) has shown better performance in various computer vision tasks, its application is restricted by a significant increase in storage and computation. Among CNN simplification techniques, parameter pruning is a promising approach which aims at reducing the number of weights of various layers without intensively reducing the original accuracy. In this paper, we propose a novel progressive parameter pruning method, named Structured Probabilistic Pruning (SPP), which effectively prunes weights of convolutional layers in a probabilistic manner. Specifically, unlike existing deterministic pruning approaches, where unimportant weights are permanently eliminated, SPP introduces a pruning probability for each weight, and pruning is guided by sampling from the pruning probabilities. A mechanism is designed to increase and decrease pruning probabilities based on importance criteria for the training process. Experiments show that, with 4x speedup, SPP can accelerate AlexNet with only 0.3% loss of top-5 accuracy and VGG-16 with 0.8% loss of top-5 accuracy in ImageNet classification. Moreover, SPP can be directly applied to accelerate multi-branch CNN networks, such as ResNet, without specific adaptations. Our 2x speedup ResNet-50 only suffers 0.8% loss of top-5 accuracy on ImageNet. We further prove the effectiveness of our method on transfer learning task on Flower-102 dataset with AlexNet.