SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation
SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation
复制标题
DOI：
10.1109/isca45697.2020.00082
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Yang Zhao;Xiaohan Chen;Yue Wang;Chaojian Li;Haoran You;Y. Fu;Yuan Xie;Zhangyang Wang;Yingyan Lin
中科院分区：
文献类型：
--
作者：
Yang Zhao;Xiaohan Chen;Yue Wang;Chaojian Li;Haoran You;Y. Fu;Yuan Xie;Zhangyang Wang;Yingyan Lin
我们推出了 SmartExchange，这是一种算法-硬件协同设计框架，可以用成本较高的内存存储/访问来换取成本较低的计算，从而实现深度神经网络 (DNN) 的节能推理。我们开发了一种新颖的算法来实施特别有利的 DNN 权重结构，其中每个分层权重矩阵可以存储为一个小基矩阵和一个大稀疏系数矩阵的乘积，该矩阵的非零元素都是 2 的幂。据我们所知，该算法是第一个将稀疏化或剪枝、分解和量化三种主流模型压缩思想集成到一个统一框架中的公式。由此产生的稀疏且易于量化的 DNN 大大降低了数据移动和权重存储方面的能耗。除此之外，我们还进一步设计了专用加速器，以充分利用 SmartExchange 强制权重来提高能源效率和延迟性能。大量实验表明：1）在算法层面，SmartExchange 在基于九个模型和四个数据集的各种消融研究中优于最先进的压缩技术，包括仅稀疏或剪枝、分解和量化； 2) 在硬件层面，当以七个 DNN 模型（包括四个标准 DNN、两个紧凑型 DNN 模型和一个分割模型）和三个数据集为基准进行基准测试时，SmartExchange 与四个最先进的 DNN 加速器相比，可以将能源效率提高高达 6.7 美元，并将延迟降低高达 19.2 美元。
We present SmartExchange, an algorithm-hardware co-design framework to trade higher-cost memory storage/access for lower-cost computation, for energy-efficient inference of deep neural networks (DNNs). We develop a novel algorithm to enforce a specially favorable DNN weight structure, where each layerwise weight matrix can be stored as the product of a small basis matrix and a large sparse coefficient matrix whose non-zero elements are all power-of-2. To our best knowledge, this algorithm is the first formulation that integrates three mainstream model compression ideas: sparsification or pruning, decomposition, and quantization, into one unified framework. The resulting sparse and readily-quantized DNN thus enjoys greatly reduced energy consumption in data movement as well as weight storage. On top of that, we further design a dedicated accelerator to fully utilize the SmartExchange-enforced weights to improve both energy efficiency and latency performance. Extensive experiments show that 1) on the algorithm level, SmartExchange outperforms state-of-the-art compression techniques, including merely sparsification or pruning, decomposition, and quantization, in various ablation studies based on nine models and four datasets; and 2) on the hardware level, SmartExchange can boost the energy efficiency by up to $6.7 \times$ and reduce the latency by up to $19.2 \times$ over four state-of-the-art DNN accelerators, when benchmarked on seven DNN models (including four standard DNNs, two compact DNN models, and one segmentation model) and three datasets.