PermDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices
PermDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices
复制标题
DOI：
10.1109/micro.2018.00024
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
Chunhua Deng;Siyu Liao;Yi Xie;K. Parhi;Xuehai Qian;Bo Yuan
中科院分区：
文献类型：
--
作者：
Chunhua Deng;Siyu Liao;Yi Xie;K. Parhi;Xuehai Qian;Bo Yuan
深度神经网络（DNN）已经成为最重要和最流行的人工智能（AI）技术。模型大小的增长对底层计算平台提出了关键的能源效率挑战。因此，模型压缩成为一个至关重要的问题。然而，目前的方法受到各种缺点的限制。具体而言，网络稀疏化方法遭受不规则性，启发式性质和大的索引开销。另一方面，最近的基于结构化矩阵的方法（即，CirCNN）受限于相对复杂的算术计算（即，FFT）、较不灵活的压缩比以及不能充分利用输入稀疏性。为了解决这些缺点，本文提出了PermDNN，这是一种使用置换对角矩阵生成和执行硬件友好的结构化稀疏DNN模型的新方法。与非结构化稀疏化方法相比，PermDNN消除了索引开销，非启发式压缩效果和耗时的重新训练的缺点。与循环结构强加方法相比，PermDNN具有计算复杂度更低、压缩比灵活、算术运算简单、输入稀疏性得到充分利用等优点。我们提出了PermDNN架构，一个多处理单元（PE）全连接（FC）层目标计算引擎。整个体系结构具有高度的可扩展性和灵活性，因此可以支持具有不同模型配置的不同应用的需求。我们使用CMOS 28 nm工艺实现了32-PE设计。与EIE相比，PermDNN在不同工作负载下的吞吐量提高了3.3倍~ 4.8倍，面积效率提高了5.9倍~ 8.5倍，能源效率提高了2.8倍~ 4.0倍。与CirCNN相比，PermDNN的吞吐量提高了11.51倍，能效提高了3.89倍。
Deep neural network (DNN) has emerged as the most important and popular artificial intelligent (AI) technique. The growth of model size poses a key energy efficiency challenge for the underlying computing platform. Thus, model compression becomes a crucial problem. However, the current approaches are limited by various drawbacks. Specifically, network sparsification approach suffers from irregularity, heuristic nature and large indexing overhead. On the other hand, the recent structured matrix-based approach (i.e., CirCNN) is limited by the relatively complex arithmetic computation (i.e., FFT), less flexible compression ratio, and its inability to fully utilize input sparsity. To address these drawbacks, this paper proposes PermDNN, a novel approach to generate and execute hardware-friendly structured sparse DNN models using permuted diagonal matrices. Compared with unstructured sparsification approach, PermDNN eliminates the drawbacks of indexing overhead, non-heuristic compression effects and time-consuming retraining. Compared with circulant structure-imposing approach, PermDNN enjoys the benefits of higher reduction in computational complexity, flexible compression ratio, simple arithmetic computation and full utilization of input sparsity. We propose PermDNN architecture, a multi-processing element (PE) fully-connected (FC) layer-targeted computing engine. The entire architecture is highly scalable and flexible, and hence it can support the needs of different applications with different model configurations. We implement a 32-PE design using CMOS 28nm technology. Compared with EIE, PermDNN achieves 3.3x~4.8x higher throughout, 5.9x~8.5x better area efficiency and 2.8x~4.0x better energy efficiency on different workloads. Compared with CirCNN, PermDNN achieves 11.51x higher throughput and 3.89x better energy efficiency.