SSO-LSM: A Sparse and Self-Organizing architecture for Liquid State Machine based neural processors
SSO-LSM: A Sparse and Self-Organizing architecture for Liquid State Machine based neural processors
复制标题
SSO-LSM：基于液态状态机的神经处理器的稀疏自组织架构
DOI：
10.1145/2950067.2950100
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Peng Li
中科院分区：
文献类型：
--
作者：
Yingyezhe Jin;Yu Liu;Peng Li
液体状态机 (LSM) 是一种强大的循环尖峰神经网络模型，为实现受大脑启发的神经处理器提供了一种有吸引力的计算范式。传统的LSM模型结合了一个随机固定循环储层作为通用预处理内核和一个可训练的读出层，该读出层提取嵌入在储层中的发射活动以促进模式识别。为了实现基于自适应 LSM 的神经处理器，我们提出了一种新颖的稀疏自组织 LSM (SSO-LSM) 架构，该架构具有低开销的硬件友好型尖峰时序相关可塑性 (STDP) 机制，可实现高效的片上储层调整。提出了数据驱动的优化流程，以极低的位分辨率在数字逻辑中有效地实现目标 STDP 规则。所提出的 STDP 规则不仅提高了学习性能，而且还在存储库中引发了理想的自组织行为，从而自然地导致了稀疏的循环网络。此外，SSO-LSM 架构结合了运行时重新配置方案，用于根据监测到的储存器中放电活动的变化，稀疏从储存器投射到读出层的突触连接。使用 TI46 语音语料库中采用的英语口语字母作为基准，我们证明，与基准 LSM 设计相比，SSO-LSM 架构将平均学习性能显着提高了 2.0%，同时将能耗降低了 25%，在 Xilinx Virtex-6 FPGA 上几乎没有额外的硬件开销。
The Liquid State Machine (LSM) is a powerful recurrent spiking neural network model that provides an appealing paradigm of computation for realizing brain-inspired neural processors. The conventional LSM model incorporates a random fixed recurrent reservoir as a general pre-processing kernel and a trainable readout layer which extracts the firing activities embedded in the reservoir to facilitate pattern recognition. To realize adaptive LSM-based neural processors, we propose a novel Sparse and Self-Organizing LSM (SSO-LSM) architecture with a low-overhead hardware-friendly Spike-Timing Dependent Plasticity (STDP) mechanism for efficient on-chip reservoir tuning. A data-driven optimization flow is presented to implement the targeted STDP rule efficiently in digital logic with extremely low bit resolutions. The proposed STDP rule not only boosts learning performance, but also induces desirable self-organizing behaviors in the reservoir that naturally lead to a sparser recurrent network. Furthermore, the SSO-LSM architecture incorporates a runtime reconfiguration scheme for sparsifying the synaptic connections projected from the reservoir to the readout layer based upon the monitored variances of firing activities in the reservoir. Using the spoken English letters adopted from the TI46 speech corpus as a benchmark, we demonstrate that the SSO-LSM architecture boosts the average learning performance rather significantly by 2.0% while reducing energy dissipation by 25% compared to a baseline LSM design with little extra hardware overhead on a Xilinx Virtex-6 FPGA.