Skip-Connected Self-Recurrent Spiking Neural Networks With Joint Intrinsic Parameter and Synaptic Weight Training
Skip-Connected Self-Recurrent Spiking Neural Networks With Joint Intrinsic Parameter and Synaptic Weight Training
复制标题
DOI：
10.1162/neco_a_01393
复制
发表时间：
2020-10
影响因子：
2.9
通讯作者：
Wenrui Zhang;Peng Li
中科院分区：
文献类型：
--
作者：
Wenrui Zhang;Peng Li
摘要是一类重要的尖峰神经网络（SNN），经常性尖峰神经网络（RSNN）具有很大的计算能力，并且已广泛用于处理诸如音频和文本之类的顺序数据。但是，大多数RSNN都遇到了两个问题。首先，由于缺乏建筑指导，通常会采用随机反复的连接性，这不能保证良好的性能。其次，对RSNN的培训通常是具有挑战性的，可以实现的模型精度。为了解决这些问题，我们提出了一种新型的RSNN，跳过连接的自我旋转SNNS（SCSR-SNN）。 SCSR-SNN中的复发是通过在尖峰神经元中添加自我连接来引入的。具有自我连接的SNN可以实现与更复杂的RSNN相似的经常性行为，而由于网络的大多数馈电本性，因此可以更直接地计算出误差梯度。网络动力学通过非粘合层之间的跳过连接丰富。此外，我们提出了一种新的反向传播方法（BP）方法，即返回的内在可塑性（BIP），以通过训练固有模型参数进一步提高SCSR-SNN的性能。与根据神经元活动调整神经元内固有参数的标准内在可塑性规则不同，提出的BIP方法根据突触重量训练的良好定义全局损耗功能的反向传播误差梯度优化了内在参数。基于具有挑战性的语音，神经形态语音和神经形态图像数据集，与其他类型的RSNN相比，通过最新的BP方法培训的其他类型的RSNN，拟议的SCSR-SNN可以提高高达2.85％的性能。
Abstract As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. First, due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. Second, training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNN, skip-connected self-recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs, while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method, backpropagated intrinsic plasticity (BIP), to boost the performance of ScSr-SNNs further by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Based on challenging speech, neuromorphic speech, and neuromorphic image data sets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.