Information-Theoretic Intrinsic Plasticity for Online Unsupervised Learning in Spiking Neural Networks
Information-Theoretic Intrinsic Plasticity for Online Unsupervised Learning in Spiking Neural Networks
复制标题
DOI：
10.3389/fnins.2019.00031
复制
发表时间：
2019-02
影响因子：
4.3
通讯作者：
Wenrui Zhang;Peng Li
中科院分区：
文献类型：
--
作者：
Wenrui Zhang;Peng Li
作为一种自适应机制，内在可塑性（IP）在维持体内平衡和塑造神经回路动力学方面起着至关重要的作用。从计算的角度来看，IP有可能在人工神经网络中实现有希望的非Hebbian学习。尽管基于IP的学习已尝试用于尖峰神经元模型，但现有的IP规则本质上是临时的，并且没有特别证明其应用程序的实际成功是为了实现现实生活中的学习任务。这项工作旨在通过提出一个名为Spikl-IP的新IP规则来解决现有作品的理论和实际局限性。 SPIKL-IP是基于严格的信息理论方法开发的，其中IP调整的目标是最大化每个尖峰神经元的输出发射速率分布的熵。通过将输出发射率分布调整为目标最佳指数分布来实现此目标。 SPIKL-IP在提出的点火率转移功能上操作，适应了尖峰神经元的固有参数，同时最大程度地降低了从目标指数分布到实际输出发射速率分布的KL-Divergence。 Spikl-IP可以在复杂的输入和网络设置下以在线方式进行稳健运作。仿真研究表明，Spikl-IP隔离地应用于单个神经元或作为较大的尖峰神经网络的一部分，可以鲁棒地产生所需的指数分布。在实际语音和图像分类任务下对Spikl-IP的评估表明，Spikl-IP明显优于两个现有的IP规则，并且可以显着提高识别精度高达16％以上。
As a self-adaptive mechanism, intrinsic plasticity (IP) plays an essential role in maintaining homeostasis and shaping the dynamics of neural circuits. From a computational point of view, IP has the potential to enable promising non-Hebbian learning in artificial neural networks. While IP based learning has been attempted for spiking neuron models, the existing IP rules are ad hoc in nature, and the practical success of their application has not been demonstrated particularly toward enabling real-life learning tasks. This work aims to address the theoretical and practical limitations of the existing works by proposing a new IP rule named SpiKL-IP. SpiKL-IP is developed based on a rigorous information-theoretic approach where the target of IP tuning is to maximize the entropy of the output firing rate distribution of each spiking neuron. This goal is achieved by tuning the output firing rate distribution toward a targeted optimal exponential distribution. Operating on a proposed firing-rate transfer function, SpiKL-IP adapts the intrinsic parameters of a spiking neuron while minimizing the KL-divergence from the targeted exponential distribution to the actual output firing rate distribution. SpiKL-IP can robustly operate in an online manner under complex inputs and network settings. Simulation studies demonstrate that the application of SpiKL-IP to individual neurons in isolation or as part of a larger spiking neural network robustly produces the desired exponential distribution. The evaluation of SpiKL-IP under real-world speech and image classification tasks shows that SpiKL-IP noticeably outperforms two existing IP rules and can significantly boost recognition accuracy by up to more than 16%.