E-RNN: Design Optimization for Efficient Recurrent Neural Networks in FPGAs
E-RNN: Design Optimization for Efficient Recurrent Neural Networks in FPGAs
复制标题
DOI：
10.1109/hpca.2019.00028
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Zhe Li;Caiwen Ding;Siyue Wang;Wujie Wen;Youwei Zhuo;Chang Liu;Qinru Qiu;Wenyao Xu;X. Lin;Xuehai Qian;Yanzhi Wang
中科院分区：
文献类型：
--
作者：
Zhe Li;Caiwen Ding;Siyue Wang;Wujie Wen;Youwei Zhuo;Chang Liu;Qinru Qiu;Wenyao Xu;X. Lin;Xuehai Qian;Yanzhi Wang
递归神经网络(RNN)对于与时间序列相关的应用变得越来越重要，这些应用需要高效和实时的实现。两种主要类型是长短期记忆(LSTM)和门控循环单位(GRU)网络。实时、高效、准确的硬件RNN实现是一项具有挑战性的任务，因为RNN对不精确累加具有很高的敏感性，而且需要特殊的激活函数实现。前人工作的一个关键缺陷是缺乏一个系统的RNN模型和硬件实现的设计优化框架，特别是当块大小(或压缩比)需要与RNN类型、层大小等联合优化时。本文采用基于块循环矩阵的框架，提出了适用于自动语音识别(ASR)应用的高效RNN(E-RNN)框架。总体目标是在满足精度要求的情况下提高性能/能效。我们使用交替方向乘法器(ADMM)技术来进行更精确的块循环训练，并提出了两个设计探索，以提供块大小的指导和减少RNN训练次数。基于这两个观察结果，我们将E-RNN分解为两个阶段：第一阶段确定RNN模型以减少精度要求下的计算量和存储量；第二阶段给出RNN模型的硬件实现，包括处理单元设计/优化、量化、激活实现等。在实际的FPGA部署上的实验结果表明，在相同的精度下，E-RNN的能效比ESE提高了37.4倍，与C-LSTM相比提高了2倍以上。
Recurrent Neural Networks (RNNs) are becoming increasingly important for time series-related applications which require efficient and real-time implementations. The two major types are Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks. It is a challenging task to have real-time, efficient, and accurate hardware RNN implementations because of the high sensitivity to imprecision accumulation and the requirement of special activation function implementations. A key limitation of the prior works is the lack of a systematic design optimization framework of RNN model and hardware implementations, especially when the block size (or compression ratio) should be jointly optimized with RNN type, layer size, etc. In this paper, we adopt the block-circulant matrix-based framework, and present the Efficient RNN (E-RNN) framework for FPGA implementations of the Automatic Speech Recognition (ASR) application. The overall goal is to improve performance/energy efficiency under accuracy requirement. We use the alternating direction method of multipliers (ADMM) technique for more accurate block-circulant training, and present two design explorations providing guidance on block size and reducing RNN training trials. Based on the two observations, we decompose E-RNN in two phases: Phase I on determining RNN model to reduce computation and storage subject to accuracy requirement, and Phase II on hardware implementations given RNN model, including processing element design/optimization, quantization, activation implementation, etc. Experimental results on actual FPGA deployments show that E-RNN achieves a maximum energy efficiency improvement of 37.4$\times$ compared with ESE, and more than 2$\times$ compared with C-LSTM, under the same accuracy.