A Soft-Pruning Method Applied During Training of Spiking Neural Networks for In-memory Computing Applications
A Soft-Pruning Method Applied During Training of Spiking Neural Networks for In-memory Computing Applications
复制标题
DOI：
10.3389/fnins.2019.00405
复制
发表时间：
2019-04-26
影响因子：
4.3
通讯作者：
Kuzum, Duygu
中科院分区：
文献类型：
--
作者：
Shi, Yuhan;Nguyen, Leon;Kuzum, Duygu
受生物大脑计算效率的启发，尖峰神经网络(SNN)模拟生物神经网络、神经代码、动力学和电路。SNN在利用记忆计算实现无监督学习方面显示出巨大的潜力。在这里，我们报告了一种算法优化，通过在新兴的非易失性存储器(ENVM)设备上使用SNN来提高在线学习的能效。我们利用神经元的输出放电特性，提出了一种SNN的剪枝方法。我们的剪枝方法可以应用于网络训练，这与文献中在已经训练的网络上使用剪枝的方法不同。此方法可防止在训练期间不必要地更新网络参数。这种算法优化可以补充eNVM技术的能效，eNVM技术为神经网络操作的并行化提供了独特的内存计算平台。我们的SNN在MNIST数据集上保持了类似90%的分类准确率，最高可达75%的剪枝，显著减少了权重更新的次数。本文开发的SNN和剪枝方案可以为基于eNVM的神经启发系统在低功耗应用中实现高能效的在线学习铺平道路。
Inspired from the computational efficiency of the biological brain, spiking neural networks (SNNs) emulate biological neural networks, neural codes, dynamics, and circuitry. SNNs show great potential for the implementation of unsupervised learning using in-memory computing. Here, we report an algorithmic optimization that improves energy efficiency of online learning with SNNs on emerging non-volatile memory (eNVM) devices. We develop a pruning method for SNNs by exploiting the output firing characteristics of neurons. Our pruning method can be applied during network training, which is different from previous approaches in the literature that employ pruning on already-trained networks. This approach prevents unnecessary updates of network parameters during training. This algorithmic optimization can complement the energy efficiency of eNVM technology, which offers a unique in-memory computing platform for the parallelization of neural network operations. Our SNN maintains similar to 90% classification accuracy on the MNIST dataset with up to similar to 75% pruning, significantly reducing the number of weight updates. The SNN and pruning scheme developed in this work can pave the way toward applications of eNVM based neuro-inspired systems for energy efficient online learning in low power applications.