A Training-Efficient Hybrid-Structured Deep Neural Network With Reconfigurable Memristive Synapses
A Training-Efficient Hybrid-Structured Deep Neural Network With Reconfigurable Memristive Synapses
复制标题
具有可重构忆阻突触的训练高效的混合结构深度神经网络
DOI：
10.1109/tvlsi.2019.2942267
复制
发表时间：
2020
影响因子：
2.8
通讯作者：
Y. Yi
中科院分区：
文献类型：
--
作者：
Kangjun Bai;Qiyuan An;Lingjia Liu;Y. Yi
神经形态计算发展的持续成功极大地推动了今天的人工智能的发展。深度神经网络(DNN)是一种类似大脑的机器学习结构，它依赖于密集的向量矩阵计算，在数据密集型应用中具有非凡的性能。最近，非易失性存储器(NVM)交叉开关阵列在神经网络设计中独树一帜地发挥了其固有的向量矩阵计算和并行计算能力。在本文中，我们设计和制造了一种混合结构的DNN(混合DNN)，结合了空间深度(Space)和时间深度(Time)的深度学习特性。我们的混合DNN采用分层信息处理方式的记忆突触和基于延迟的尖峰神经网络(SNN)模块作为读出层。实验结果表明，该系统具有较高的计算并行性和能效，硬件实现成本较低，适合于低功耗嵌入式应用。从混沌时间序列的预测基准来看，与最先进的DNN设计相比，我们的混合DNN的预测误差减少了1.16倍-13.77倍。此外，在手写数字分类和语音数字识别任务上，我们的混合DNN分别获得了99.03%和99.63%的测试准确率。
The continued success in the development of neuromorphic computing has immensely pushed today’s artificial intelligence forward. Deep neural networks (DNNs), a brainlike machine learning architecture, rely on the intensive vector–matrix computation with extraordinary performance in data-extensive applications. Recently, the nonvolatile memory (NVM) crossbar array uniquely has unvailed its intrinsic vector–matrix computation with parallel computing capability in neural network designs. In this article, we design and fabricate a hybrid-structured DNN (hybrid-DNN), combining both depth-in-space (spatial) and depth-in-time (temporal) deep learning characteristics. Our hybrid-DNN employs memristive synapses working in a hierarchical information processing fashion and delay-based spiking neural network (SNN) modules as the readout layer. Our fabricated prototype in 130-nm CMOS technology along with experimental results demonstrates its high computing parallelism and energy efficiency with low hardware implementation cost, making the designed system a candidate for low-power embedded applications. From chaotic time-series forecasting benchmarks, our hybrid-DNN exhibits $1.16\times $ – $13.77\times $ reduction on the prediction error compared to the state-of-the-art DNN designs. Moreover, our hybrid-DNN records 99.03% and 99.63% testing accuracy on the handwritten digit classification and the spoken digit recognition tasks, respectively.