RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition
RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition
复制标题
DOI：
10.1109/dac18072.2020.9218499
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Peiyan Dong;Siyue Wang;Wei Niu;Chengming Zhang;Sheng Lin;Z. Li;Yifan Gong;Bin Ren;X. Lin;Yanzhi Wang;Dingwen Tao
中科院分区：
文献类型：
--
作者：
Peiyan Dong;Siyue Wang;Wei Niu;Chengming Zhang;Sheng Lin;Z. Li;Yifan Gong;Bin Ren;X. Lin;Yanzhi Wang;Dingwen Tao
基于递归神经网络（RNNs）的自动语音识别在智能手机等移动设备上具有重要的应用前景。然而，以前的RNN压缩技术要么由于不规则而遭受硬件性能开销，要么由于保留硬件友好性的规则而遭受显著的精度损失。在这项工作中，我们提出了RTMobile，它利用了一种新的基于块的修剪方法和编译器优化来加速移动设备上的RNN推理。我们提出的RTMobile是第一个可以在移动平台上实现实时RNN推理的工作。实验结果表明，RTMobile在推理精度和时间上都明显优于现有的RNN硬件加速方法。与FPGA相比，在GRU上使用Adreno 640嵌入式GPU的RTMobile在保持相同推理时间的前提下，能效提高了40倍。
Recurrent neural networks (RNNs) based automatic speech recognition has nowadays become promising and important on mobile devices such as smart phones. However, previous RNN compression techniques either suffer from hardware performance overhead due to irregularity or significant accuracy loss due to the preserved regularity for hardware friendliness. In this work, we propose RTMobile that leverages both a novel block-based pruning approach and compiler optimizations to accelerate RNN inference on mobile devices. Our proposed RTMobile is the first work that can achieve real-time RNN inference on mobile platforms. Experimental results demonstrate that RTMobile can significantly outperform existing RNN hardware acceleration methods in terms of both inference accuracy and time. Compared with prior work on FPGA, RTMobile using Adreno 640 embedded GPU on GRU can improve the energy-efficiency by 40× while maintaining the same inference time.