Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression
Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression
复制标题
DOI：
10.1109/tcsi.2021.3120312
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Zhuang Shao;Xiaoliang Chen;Li Du;Lei Chen;Yuan Du;Weihao Zhuang;Huadong Wei;Chenjia Xie;Zhongfeng Wang
中科院分区：
文献类型：
--
作者：
Zhuang Shao;Xiaoliang Chen;Li Du;Lei Chen;Yuan Du;Weihao Zhuang;Huadong Wei;Chenjia Xie;Zhongfeng Wang
现有的深度卷积神经网络（CNN）在网络推理过程中会产生大量的层间特征数据。为了保证嵌入式系统的实时处理，需要大容量的片上存储器来缓冲层间特征图。在本文中，我们提出了一个有效的硬件加速器与层间特征压缩技术，以显着减少所需的片上存储器的大小和片外存储器访问带宽。该加速器通过使用硬件实现的8 × 8离散余弦变换（DCT）将存储的数据变换到频域来压缩层间特征图。在DCT之后通过量化去除高频分量。利用稀疏矩阵压缩来进一步压缩层间特征图。片上内存分配方案的设计，以支持动态配置的特征地图缓冲区大小和暂存器大小，根据不同的网络层的要求。硬件加速器将压缩，解压缩和CNN加速结合到一个计算流中，实现最小的压缩和处理延迟。在FPGA平台上实现了一个原型加速器，并在台积电28纳米COMS技术合成。它通过增加轻硬件面积开销，实现了403 GOPS峰值吞吐量和1.4\times\sim 3.3\times $层间特征图缩减，使其成为智能物联网设备的有前途的硬件加速器。
Existing deep convolutional neural networks (CNNs) generate massive interlayer feature data during network inference. To maintain real-time processing in embedded systems, large on-chip memory is required to buffer the interlayer feature maps. In this paper, we propose an efficient hardware accelerator with an interlayer feature compression technique to significantly reduce the required on-chip memory size and off-chip memory access bandwidth. The accelerator compresses interlayer feature maps through transforming the stored data into frequency domain using hardware-implemented $8\times 8$ discrete cosine transform (DCT). The high-frequency components are removed after the DCT through quantization. Sparse matrix compression is utilized to further compress the interlayer feature maps. The on-chip memory allocation scheme is designed to support dynamic configuration of the feature map buffer size and scratch pad size according to different network-layer requirements. The hardware accelerator combines compression, decompression, and CNN acceleration into one computing stream, achieving minimal compressing and processing delay. A prototype accelerator is implemented on an FPGA platform and also synthesized in TSMC 28-nm COMS technology. It achieves 403GOPS peak throughput and $1.4\times \sim 3.3\times $ interlayer feature map reduction by adding light hardware area overhead, making it a promising hardware accelerator for intelligent IoT devices.