MIXP: Efficient Deep Neural Networks Pruning for Further FLOPs Compression via Neuron Bond
MIXP: Efficient Deep Neural Networks Pruning for Further FLOPs Compression via Neuron Bond
复制标题
DOI：
10.1109/ijcnn52387.2021.9533522
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Bin Hu;Tianming Zhao;Yucheng Xie;Yan Wang;Xiaonan Guo;Jerry Q. Cheng;Yingying Chen
中科院分区：
文献类型：
--
作者：
Bin Hu;Tianming Zhao;Yucheng Xie;Yan Wang;Xiaonan Guo;Jerry Q. Cheng;Yingying Chen
神经元网络修剪可以有效地压缩预训练的 CNN，以便在低端边缘设备上部署。然而，很少有工作专注于降低剪枝和推理的计算成本。我们发现现有的剪枝方法通常会删除参数而没有进行细粒度的影响分析，从而很难获得最优解决方案。这项工作开发了一种新颖的混合剪枝机制 MIXP，它可以有效降低 CNN 的计算成本，同时保持高权重压缩比和模型精度。我们建议去除神经元键，这样可以有效减少 CNN 中的卷积计算量和权重大小。我们还设计了一个影响因子来细粒度地分析神经元键和权重的重要性，使得 MIXP 能够通过很少的再训练迭代实现精确的剪枝。使用 MNIST、CIFAR-10 和 ImageNet 数据集进行的实验表明，与现有的剪枝方法相比，MIXP 在四种广泛使用的 CNN 上可以实现显着更少的 FLOP 和重新训练迭代。
Neuron networks pruning is effective in compressing pre-trained CNNs for their deployment on low-end edge devices. However, few works have focused on reducing the computational cost of pruning and inference. We find that existing pruning methods usually remove parameters without fine-grained impact analysis, making it hard to achieve an optimal solution. This work develops a novel mixture pruning mechanism, MIXP, which can effectively reduce the computational cost of CNNs while maintaining a high weight compression ratio and model accuracy. We propose to remove neuron bond that can effectively reduce convolution computations and weight size in CNNs. We also design an influence factor to analyze the importance of neuron bonds and weights in a fine-grained way so that MIXP could achieve precise pruning with few retraining iterations. Experiments with MNIST, CIFAR-10, and ImageNet datasets demonstrate that MIXP could achieve significantly fewer FLOPs and retraining iterations on four widely-used CNNs than existing pruning methods.