Harmonious Coexistence of Structured Weight Pruning and Ternarization for Deep Neural Networks
Harmonious Coexistence of Structured Weight Pruning and Ternarization for Deep Neural Networks
复制标题
DOI：
10.1609/aaai.v34i04.6138
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Li Yang;Zhezhi He;Deliang Fan
中科院分区：
文献类型：
--
作者：
Li Yang;Zhezhi He;Deliang Fan
深度卷积神经网络（Deep convolutional neural network， DNN）已被广泛应用于许多计算机视觉任务中。然而，其庞大的模型尺寸和较高的计算复杂度阻碍了其在资源有限的嵌入式系统（如FPGA和mGPU）中的广泛部署。权值修剪和量化是采用最广泛的两种模型压缩技术，分别通过引入权值稀疏性（即强制部分权值为零）和将权值量化为有限位宽值来压缩DNN模型。虽然有一些研究试图将权值修剪和量化结合起来，但我们仍然观察到权值修剪和量化之间的不协调，特别是当使用更激进的压缩方案（例如，结构化修剪和低位宽量化）时。本文以FPGA为测试计算平台，以处理单元（Processing Elements， PE）为基本并行计算单元，首先提出了一种基于处理单元的结构化剪枝方案，该方案在考虑处理单元结构的基础上引入了权值稀疏化。此外，我们将其与优化的权重三元化方法集成，该方法将权重量化为三元值（{-1,0,+1}），从而将DNN中的主要卷积操作从乘法累加（MAC）转换为仅加法，以及将原始模型（从32位浮点数压缩为2位三元表示）压缩至少16倍。在此基础上，提出了一种自适应阈值加权惩罚裁剪（WPC）技术，研究并解决了PE-wise结构化剪枝与三元化的共存问题。我们的实验表明，我们提出的技术融合可以在ImageNet数据集上实现最佳的最先进的~ 21× pe的结构化压缩率，而ResNet-18的精度仅下降1.74%/0.94% （top-1/top-5）。
Deep convolutional neural network (DNN) has demonstrated phenomenal success and been widely used in many computer vision tasks. However, its enormous model size and high computing complexity prohibits its wide deployment into resource limited embedded system, such as FPGA and mGPU. As the two most widely adopted model compression techniques, weight pruning and quantization compress DNN model through introducing weight sparsity (i.e., forcing partial weights as zeros) and quantizing weights into limited bit-width values, respectively. Although there are works attempting to combine the weight pruning and quantization, we still observe disharmony between weight pruning and quantization, especially when more aggressive compression schemes (e.g., Structured pruning and low bit-width quantization) are used. In this work, taking FPGA as the test computing platform and Processing Elements (PE) as the basic parallel computing unit, we first propose a PE-wise structured pruning scheme, which introduces weight sparsification with considering of the architecture of PE. In addition, we integrate it with an optimized weight ternarization approach which quantizes weights into ternary values ({-1,0,+1}), thus converting the dominant convolution operations in DNN from multiplication-and-accumulation (MAC) to addition-only, as well as compressing the original model (from 32-bit floating point to 2-bit ternary representation) by at least 16 times. Then, we investigate and solve the coexistence issue between PE-wise Structured pruning and ternarization, through proposing a Weight Penalty Clipping (WPC) technique with self-adapting threshold. Our experiment shows that the fusion of our proposed techniques can achieve the best state-of-the-art ∼21× PE-wise structured compression rate with merely 1.74%/0.94% (top-1/top-5) accuracy degradation of ResNet-18 on ImageNet dataset.