Boosting Deep Neural Network Efficiency with Dual-Module Inference-猫眼课题宝

DOI：

发表时间：

2020-07

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Liu Liu-Liu;Lei Deng;Zhaodong Chen;Yuke Wang;Shuangchen Li;Jingwei Zhang;Yihua Yang;Zhenyu Gu;Yufei Ding;Yuan Xie

在机器学习任务中使用深度神经网络（DNN）有望提供高质量的结果，但由于DNN的内存限制和计算限制执行模式，因此满足严格的延迟要求和能量限制具有挑战性。我们提出了一个大-小双模块推理，动态跳过不必要的内存访问和计算，以加速DNN推理。利用非线性激活函数的抗噪声特性，我们建议使用一个轻量级的小模块来近似原始DNN层，称为大模块，以计算更具抗噪声能力的不敏感区域的激活。因此，可以减少昂贵的存储器访问和大模块的计算，因为结果仅在敏感区域中计算。对于内存受限的模型，如递归神经网络（RNN），我们的方法可以将整体内存访问平均减少40%，并在基于CPU的商品服务器平台上实现1.54倍至1.75倍的加速，对模型质量的影响可以忽略不计。此外，我们的方法可以将卷积神经网络（CNN）等计算约束模型的操作减少3.02倍，准确率仅下降0.5%。

Using deep neural networks (DNNs) in machine learning tasks is promising in delivering high-quality results but challenging to meet stringent latency requirements and energy constraints because of the memory-bound and the compute-bound execution pattern of DNNs. We propose a big-little dual-module inference to dynamically skip unnecessary memory accesses and computations to accelerate DNN inference. Leveraging the noise-resilient feature of nonlinear activation functions, we propose to use a lightweight little module that approximates the original DNN layer, termed as the big module, to compute activations of the insensitive region that are more noise-resilient. Hence, the expensive memory accesses and computations of the big module can be reduced as the results are only calculated in the sensitive region. For memory-bound models such as recurrent neural networks (RNNs), our method can reduce the overall memory accesses by 40% on average and achieve 1.54x to 1.75x speedup on a commodity CPU-based server platform with a negligible impact on model quality. In addition, our method can reduce the operations of the compute-bound models such as convolutional neural networks (CNNs) by 3.02x, with only a 0.5% accuracy drop.

Boosting Deep Neural Network Efficiency with Dual-Module Inference

文献摘要

被引文献

相似文献