Approximating Back-propagation for a Biologically Plausible Local Learning Rule in Spiking Neural Networks
Approximating Back-propagation for a Biologically Plausible Local Learning Rule in Spiking Neural Networks
复制标题
DOI：
10.1145/3354265.3354275
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Amar Shrestha;Haowen Fang;Qing Wu;Qinru Qiu
中科院分区：
文献类型：
--
作者：
Amar Shrestha;Haowen Fang;Qing Wu;Qinru Qiu
使用尖峰的异步事件驱动计算和通信有助于实现尖峰神经网络 (SNN)，使其在专用神经形态硬件上实现大规模并行、极其节能且高度鲁棒。然而，缺乏统一的鲁棒学习算法将SNN限制在精度较低的浅层网络中。然而，人工神经网络 (ANN) 具有反向传播算法，可以利用梯度下降来训练局部鲁棒通用函数逼近器的网络。但反向传播算法在生物学上既不合理，也不适合神经形态实现，因为它需要：1）单独的后向和前向传递，2）可微神经元，3）高精度传播误差，4）前馈权重和后向传递的权重矩阵的相干副本，以及5）非局部权重更新。因此，我们提出了完全使用尖峰神经元的反向传播算法的近似，并将其扩展到局部权重更新规则，该规则类似于生物学上合理的学习规则尖峰时序相关可塑性（STDP）。这将使错误能够通过尖峰神经元进行传播，从而为 SNN 提供更具生物学合理性和神经形态实现友好的反向传播算法。我们在各种传统和非传统基准上测试了所提出的算法，并取得了有竞争力的结果。
Asynchronous event-driven computation and communication using spikes facilitate the realization of spiking neural networks (SNN) to be massively parallel, extremely energy efficient and highly robust on specialized neuromorphic hardware. However, the lack of a unified robust learning algorithm limits the SNN to shallow networks with low accuracies. Artificial neural networks (ANN), however, have the backpropagation algorithm which can utilize gradient descent to train networks which are locally robust universal function approximators. But backpropagation algorithm is neither biologically plausible nor neuromorphic implementation friendly because it requires: 1) separate backward and forward passes, 2) differentiable neurons, 3) high-precision propagated errors, 4) coherent copy of weight matrices at feedforward weights and the backward pass, and 5) non-local weight update. Thus, we propose an approximation of the backpropagation algorithm completely with spiking neurons and extend it to a local weight update rule which resembles a biologically plausible learning rule spike-timing-dependent plasticity (STDP). This will enable error propagation through spiking neurons for a more biologically plausible and neuromorphic implementation friendly backpropagation algorithm for SNNs. We test the proposed algorithm on various traditional and non-traditional benchmarks with competitive results.