Shallowing Deep Networks: Layer-wise Pruning based on Feature Representations
Shallowing Deep Networks: Layer-wise Pruning based on Feature Representations
复制标题
DOI：
10.1109/tpami.2018.2874634
复制
发表时间：
2019-12-01
影响因子：
23.6
通讯作者：
Zhao, Qi
中科院分区：
文献类型：
--
作者：
Chen, Shi;Zhao, Qi
卷积神经网络（CNN）最近的激增在各种应用中取得了成功。然而，这些成功伴随着计算成本和计算资源需求的显著增加，这严重阻碍了在计算能力有限的设备上利用复杂的CNN。在这项工作中，我们提出了一种基于特征表示的分层修剪方法，旨在将复杂的CNN减少到具有同等性能的更紧凑的CNN。与以往基于权重信息进行连接或过滤器修剪的参数修剪方法不同，我们的方法通过调查卷积层中学习的特征来确定冗余参数，并且修剪过程在层级别上操作。实验结果表明，该方法能够显著降低计算开销，在不同数据集上，剪枝后的模型性能与原始模型相当甚至更好.
Recent surge of Convolutional Neural Networks (CNNs) has brought successes among various applications. However, these successes are accompanied by a significant increase in computational cost and the demand for computational resources, which critically hampers the utilization of complex CNNs on devices with limited computational power. In this work, we propose a feature representation based layer-wise pruning method that aims at reducing complex CNNs to more compact ones with equivalent performance. Different from previous parameter pruning methods that conduct connection-wise or filter-wise pruning based on weight information, our method determines redundant parameters by investigating the features learned in the convolutional layers and the pruning process is operated at a layer level. Experiments demonstrate that the proposed method is able to significantly reduce computational cost and the pruned models achieve equivalent or even better performance compared to the original models on various datasets.