Direct Training for Spiking Neural Networks: Faster, Larger, Better
Direct Training for Spiking Neural Networks: Faster, Larger, Better
复制标题
DOI：
10.1609/aaai.v33i01.33011311
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
Yujie Wu;Lei Deng;Guoqi Li;Jun Zhu;Luping Shi
中科院分区：
文献类型：
--
作者：
Yujie Wu;Lei Deng;Guoqi Li;Jun Zhu;Luping Shi
尖峰神经网络(SNN)能够在新兴的神经形态硬件上实现高能效，正受到越来越多的关注。然而现在，由于缺乏有效的学习算法和高效的编程框架，SNN并没有表现出与人工神经网络(ANN)相媲美的性能。我们从两个方面来解决这个问题：(1)提出了一种神经元归一化技术来调整神经元的选择性，并开发了一种针对深层SNN的直接学习算法。(2)通过缩小码率编码窗口和将泄漏积分-火灾(LIF)模型转换为显式迭代版本，提出了一种基于Pytorch的大规模SNN训练实现方法。通过这种方式，我们能够训练深度SNN，加速比达到几十倍。结果，我们在神经形态数据集(N-MNIST和DVSCIFAR10)上获得了比已报道的工作更好的准确性，而在非尖峰数据集上获得了与现有神经网络和预先训练的SNN相当的精度(CIFAR10)。据我们所知，这是第一个在CIFAR10上高性能地直接训练深度SNN的工作，其高效的实现为挖掘SNN的潜力提供了一种新的途径。
Spiking neural networks (SNNs) that enables energy efficient implementation on emerging neuromorphic hardware are gaining more attention. Yet now, SNNs have not shown competitive performance compared with artificial neural networks (ANNs), due to the lack of effective learning algorithms and efficient programming frameworks. We address this issue from two aspects: (1) We propose a neuron normalization technique to adjust the neural selectivity and develop a direct learning algorithm for deep SNNs. (2) Via narrowing the rate coding window and converting the leaky integrate-and-fire (LIF) model into an explicitly iterative version, we present a Pytorch-based implementation method towards the training of large-scale SNNs. In this way, we are able to train deep SNNs with tens of times speedup. As a result, we achieve significantly better accuracy than the reported works on neuromorphic datasets (N-MNIST and DVSCIFAR10), and comparable accuracy as existing ANNs and pre-trained SNNs on non-spiking datasets (CIFAR10). To our best knowledge, this is the first work that demonstrates direct training of deep SNNs with high performance on CIFAR10, and the efficient implementation provides a new way to explore the potential of SNNs.