An FPGA Implementation of Stochastic Computing-Based LSTM
An FPGA Implementation of Stochastic Computing-Based LSTM
复制标题
DOI：
10.1109/iccd46524.2019.00014
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Guy Maor;Xiaoming Zeng;Zhendong Wang;Yang Hu
中科院分区：
文献类型：
--
作者：
Guy Maor;Xiaoming Zeng;Zhendong Wang;Yang Hu
长短期记忆（Long Short Term Memory，LSTM）作为一种特殊类型的递归神经网络（Recurrent Neural Networks，RNN），能够处理序列数据，在精度上有很大的提高，在图像/视频识别和语音识别中得到了广泛的应用。然而，LSTM通常具有高计算复杂度，并且在实现时可能导致高硬件成本和功耗。随着物联网（IoT）和移动的/边缘计算的发展，大量资源有限的移动的和边缘设备被广泛部署，这进一步加剧了这种情况。最近，随机计算（SC）已经应用于神经网络（NN）（例如，卷积神经网络（CNN）结构，以提高功率效率。本质上，SC可以有效地简化基本算术电路（例如，乘法），并降低硬件成本和功耗。因此，本文将SC引入LSTM，创造性地提出了一种基于SC的LSTM架构设计，以节省硬件成本和功耗。更重要的是，本文成功地实现了现场可编程门阵列（FPGA）的设计，并评估其性能的MNIST数据集。评估结果表明，SC-LSTM设计工作平稳，与基线二进制LSTM实现相比，可以显著降低73.24%的功耗，而不会造成太大的精度损失。在未来，SC可以在广泛的物联网和移动的/边缘应用中节省硬件成本并降低功耗。
As a special type of recurrent neural networks (RNN), Long Short Term Memory (LSTM) is capable of processing sequential data with a great improvement in accuracy and is widely applied in image/video recognition and speech recognition. However, LSTM typically possesses high computational complexity and may cause high hardware cost and power consumption when being implemented. With the development of Internet of Things (IoT) and mobile/edge computation, lots of mobile and edge devices with limited resources are widely deployed, which further exacerbates the situation. Recently, Stochastic Computing (SC) has been applied in to neural networks (NN) (e.g., convolution neural networks, CNN) structure to improve power efficiency. Essentially, SC can effectively simplify the fundamental arithmetic circuits (e.g., multiplication), and reduce the hardware cost and power consumption. Therefore, this paper introduces SC into LSTM and creatively proposes an SC-based LSTM architecture design to save the hardware cost and power consumption. More importantly, the paper successfully implements the design on a Field Programmable Gate Array (FPGA) and evaluates its performance on the MNIST dataset. The evaluation results show that the SC-LSTM design works smoothly and can significantly reduce power consumption by 73.24% compared to the baseline binary LSTM implementation without much accuracy loss. In the future, SC can potentially save hardware cost and reduce power consumption in a wide range of IoT and mobile/edge applications.