Spike-Train Level Direct Feedback Alignment: Sidestepping Backpropagation for On-Chip Training of Spiking Neural Nets
Spike-Train Level Direct Feedback Alignment: Sidestepping Backpropagation for On-Chip Training of Spiking Neural Nets
复制标题
DOI：
10.3389/fnins.2020.00143
复制
发表时间：
2020-03-13
影响因子：
4.3
通讯作者：
Li, Peng
中科院分区：
文献类型：
--
作者：
Lee, Jeongjun;Zhang, Renqian;Li, Peng
尖峰神经网络（SNNS）提出了一个有希望的计算模型，并启用了基于事件驱动的超低功率神经形态硬件。然而，培训SNN以达到常规深人造神经网络（ANN）的相同性能，尤其是使用错误反向传播（BP）算法，由于固有的复杂动力学和尖峰神经元的非差异性尖峰活动而构成了重大挑战。在本文中，我们介绍了第一项有关实现竞争性尖峰训练水平反向传播（BP）等算法的研究，以实现对SNN的芯片培训。我们提出了一种新颖的Spike-Train水平直接反馈对齐（ST-DFA）算法，该算法比BP更具生物性和硬件友好性。探索了算法和硬件协调和有效的在线神经信号计算，以实现ST-DFA的芯片实现。在Xilinx ZC706 FPGA板上，提议的硬件有效的ST-DFA显示出出色的性能与真实世界语音和图像分类应用程序的高架权衡。带有片上ST-DFA培训的SNN神经处理器显示，MNIST数据集的竞争性分类精度为96.27％，分别减少了4倍输入分辨率和84.88％的竞争性分类精度，而具有挑战性的16扬声器Ti46语音语料库分别为84.88％。与最先进的BP算法HM2-BP的硬件实施相比，拟议的ST-DFA的设计将功能资源降低了76.7％，向后培训潜伏期降低了31.6％，同时优雅地交易分类性能。
Spiking neural networks (SNNs) present a promising computing model and enable bio-plausible information processing and event-driven based ultra-low power neuromorphic hardware. However, training SNNs to reach the same performances of conventional deep artificial neural networks (ANNs), particularly with error backpropagation (BP) algorithms, poses a significant challenge due to inherent complex dynamics and non-differentiable spike activities of spiking neurons. In this paper, we present the first study on realizing competitive spike-train level backpropagation (BP) like algorithms to enable on-chip training of SNNs. We propose a novel spike-train level direct feedback alignment (ST-DFA) algorithm, which is much more bio-plausible and hardware friendly than BP. Algorithm and hardware co-optimization and efficient online neural signal computation are explored for on-chip implementation of ST-DFA. On the Xilinx ZC706 FPGA board, the proposed hardware-efficient ST-DFA shows excellent performance vs. overhead tradeoffs for real-world speech and image classification applications. SNN neural processors with on-chip ST-DFA training show competitive classification accuracy of 96.27% for the MNIST dataset with 4x input resolution reduction and 84.88% for the challenging 16-speaker TI46 speech corpus, respectively. Compared to the hardware implementation of the state-of-the-art BP algorithm HM2-BP, the design of the proposed ST-DFA reduces functional resources by 76.7% and backward training latency by 31.6% while gracefully trading off classification performance.