PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices
PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices
复制标题
DOI：
10.1609/aaai.v34i04.5954
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
Xiaolong Ma;Fu-Ming Guo;Wei Niu;Xue Lin;Jian Tang;Kaisheng Ma;Bin Ren;Yanzhi Wang
中科院分区：
文献类型：
--
作者：
Xiaolong Ma;Fu-Ming Guo;Wei Niu;Xue Lin;Jian Tang;Kaisheng Ma;Bin Ren;Yanzhi Wang
深度神经网络（DNN）上的模型压缩技术已被广泛认为是在各种平台上实现加速的有效方法，而DNN权重修剪是一种简单有效的方法。目前有两种主流的剪枝方法，代表了两个极端的剪枝规则：非结构化的，细粒度的剪枝可以实现高稀疏性和准确性，但不是硬件友好的;结构化的，粗粒度的剪枝利用硬件有效的结构在剪枝，但遭受准确性下降时，剪枝率高。在本文中，我们介绍PCONV，包括一个新的稀疏维，细粒度修剪模式内的粗粒度结构。PCONV包括两种类型的稀疏性，稀疏卷积模式（SCP），其从卷积内核修剪生成，以及连接稀疏性，其从卷积间核修剪生成。本质上，SCP由于其特殊的视觉特性而提高了准确性，并且连接稀疏性提高了修剪率，同时保持了过滤器计算的平衡工作量。为了部署PCONV，我们开发了一个新的编译器辅助DNN推理框架，并实时执行PCONV模型，而不会影响精度，这在以前的工作中是无法实现的。我们的实验结果表明，PCONV优于三种最先进的端到端DNN框架，TensorFlow-Lite，TVM和Alibaba移动的神经网络，加速分别高达39.2倍，11.4倍和6.3倍，并且没有准确性损失。移动的设备可以在大规模DNN上实现实时推理。
Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method. There are currently two mainstreams of pruning methods representing two extremes of pruning regularity: non-structured, fine-grained pruning can achieve high sparsity and accuracy, but is not hardware friendly; structured, coarse-grained pruning exploits hardware-efficient structures in pruning, but suffers from accuracy drop when the pruning rate is high. In this paper, we introduce PCONV, comprising a new sparsity dimension, – fine-grained pruning patterns inside the coarse-grained structures. PCONV comprises two types of sparsities, Sparse Convolution Patterns (SCP) which is generated from intra-convolution kernel pruning and connectivity sparsity generated from inter-convolution kernel pruning. Essentially, SCP enhances accuracy due to its special vision properties, and connectivity sparsity increases pruning rate while maintaining balanced workload on filter computation. To deploy PCONV, we develop a novel compiler-assisted DNN inference framework and execute PCONV models in real-time without accuracy compromise, which cannot be achieved in prior work. Our experimental results show that, PCONV outperforms three state-of-art end-to-end DNN frameworks, TensorFlow-Lite, TVM, and Alibaba Mobile Neural Network with speedup up to 39.2 ×, 11.4 ×, and 6.3 ×, respectively, with no accuracy loss. Mobile devices can achieve real-time inference on large-scale DNNs.