Learning Local Representation by Gradient-Isolated Memorizing of Spiking Neural Network
Learning Local Representation by Gradient-Isolated Memorizing of Spiking Neural Network
复制标题
DOI：
10.1109/hpcc-dss-smartcity-dependsys57074.2022.00123
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Man Wu;Zheng Chen;Yunpeng Yao
中科院分区：
文献类型：
--
作者：
Man Wu;Zheng Chen;Yunpeng Yao
尖峰神经网络(SNN)由于其事件驱动的稀疏性和具有二进制尖峰的泄漏积分与点火(LIF)神经元，而不是传统的具有模拟输出的激活函数，在实现低功耗AI硬件方面显示出良好的前景。然而，由于SNN的尖峰编码范例，其处理本地输入的能力较低。为了解决这些问题，我们提出了一种新的SNN结构，该结构具有梯度隔离记忆机制、水平LIF和垂直LIF神经元来学习局部表示。此外，为了充分利用局部表示和全局表示的优势，我们在局部-全局联合表示中设计了一种并发SNN结构。对于概念验证，我们在MNIST、Fashion MNIST和CIFAR-10数据集上进行了评估，在保持较低计算复杂度的情况下，在5个时间步长上分别获得了98.68%和90.37%的准确率和87.19%的准确率。
Spiking neural networks (SNNs) show promise in implementing low-power AI hardware due to their event-driven sparsity and leaky integrate-and-fire (LIF) neurons with binary spikes, rather than traditional activation functions with analog output. However, SNNs have a low capacity to process the local input due to their spike encoding paradigm. To address these problems, we propose a novel SNN architecture with a gradient-isolated memorizing mechanism, horizontal LIF, and vertical LIF neurons to learn local representation. Moreover, to take full advantage of local representations and global representations, we design a concurrent SNN architecture within the local-global joint representation. For proof-of-concept, we evaluate our proposal on MNIST, Fashion MNIST, and CIFAR-10 datasets, which achieve 98.68% and 90.37%, and 87.19% accuracy with 5 time steps, respectively, while maintaining low computational complexity.