Enabling Non-Hebbian Learning in Recurrent Spiking Neural Processors With Hardware-Friendly On-Chip Intrinsic Plasticity
Enabling Non-Hebbian Learning in Recurrent Spiking Neural Processors With Hardware-Friendly On-Chip Intrinsic Plasticity
复制标题
DOI：
10.1109/jetcas.2019.2934939
复制
发表时间：
2019-08
影响因子：
4.6
通讯作者：
Yu Liu;Wenrui Zhang;Peng Li
中科院分区：
文献类型：
--
作者：
Yu Liu;Wenrui Zhang;Peng Li
内在可塑性（IP）是一种非赫布学习机制，它自适应每个神经元的内在参数，而不是突触权重，为学习性能的改善提供了机会。然而，在芯片上集成IP以实现每个神经元的自适应可能会导致非常大的设计开销。本文是第一篇基于液态机（LSM）的循环尖峰神经网络模型探索神经加速器的高效片上非Hebbian IP学习的工作。从算法和硬件设计的角度来看，所提出的LSM神经处理器与片上IP集成的成本效益方面的改进。我们优化了一个基线IP规则，这使得国家的最先进的学习性能，使一个可行的硬件片上集成，并进一步提出了一个新的硬件友好的IP规则SpiKL IFIP。采用新的IP规则及其优化实现，极大地改善了片上IP的硬件LSM神经加速器的面积/功耗开销以及训练延迟。在Xilinx ZC 706 FPGA板上，所提出的协同优化显著提高了片上IP的成本效益。使用IP的自适应水库神经元在TI46语音语料库上将分类准确率提高了10.33%，在TIMIT声学语音数据集上提高了8%。此外，所提出的技术将训练能量降低了49.6%，资源利用率降低了64.9%，同时优雅地权衡了分类精度和设计效率。
Intrinsic plasticity (IP) is a non-Hebbian learning mechanism that self-adapts intrinsic parameters of each neuron as opposed to synaptic weights, offering complimentary opportunities for learning performance improvement. However, integrating IP onchip to enable per-neuron self-adaptation can lead to very large design overheads. This paper is the first work exploring efficient on-chip non-Hebbian IP learning for neural accelerators based on the recurrent spiking neural network model of the liquid state machine (LSM). The proposed LSM neural processor integrated with onchip IP is improved in terms of cost-effectiveness from both algorithmic and hardware design points of view. We optimize a baseline IP rule, which gives the state-of-the-art learning performance, to enable a feasible hardware onchip integration and further propose a new hardware-friendly IP rule SpiKL-IFIP. The hardware LSM neural accelerator with onchip IP is dramatically improved in area/power overhead as well as training latency with the proposed new IP rule and its optimized implementation. On the Xilinx ZC706 FPGA board, the proposed co-optimization dramatically improves the cost-effectiveness of on-chip IP. Self-adapting reservoir neurons using IP boosts the classification accuracy by up to 10.33% on the TI46 speech corpus and 8% on the TIMIT acoustic-phonetic dataset. Moreover, the proposed techniques reduce training energy by up to 49.6% and resource utilization by up to 64.9% while gracefully trading off classification accuracy for design efficiency.