REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation
REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation
复制标题
DOI：
10.1587/transinf.2020edp7049
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Koji Kamma;T. Wada
中科院分区：
文献类型：
--
作者：
Koji Kamma;T. Wada
摘要为了提高卷积神经网络模型的推理速度，提出了一种基于重构误差感知的剪枝方法(REAP)，以减少卷积神经网络模型的冗余度。在REAP中，我们包括以下步骤：1)剪枝每个卷积层中其他信道的输出可以重构的冗余信道；2)用最小二乘法更新剩余信道的权值，以补偿剪枝带来的误差。这就是我们如何压缩和加速最初很大、很慢、几乎没有降级的模型。REAP保持模型性能的能力为我们节省了大量的时间和人力来重新训练剪枝后的模型。REAP的挑战在于选择要修剪的信道的计算成本。对于频道的选择，我们需要解决大量的最小二乘问题。我们开发了一种基于双正交系的ffi算法来获得这些最小二乘问题的解。在实验中，我们证明了REAP能够以更小的模型性能进行剪枝，而不是现有的几种方法，包括以前最先进的方法。fice。
SUMMARY This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an e ffi cient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.