A Long Short-Term Memory for AI Applications in Spike-based Neuromorphic Hardware
A Long Short-Term Memory for AI Applications in Spike-based Neuromorphic Hardware
复制标题
DOI：
10.1038/s42256-022-00480-w
复制
发表时间：
2022-05-19
影响因子：
23.8
通讯作者：
Maass, Wolfgang
中科院分区：
文献类型：
--
作者：
Rao, Arjun;Plank, Philipp;Maass, Wolfgang
当深度学习在基于尖峰的神经形态芯片上实施时，能源密集度可能会降低。受生物神经元特征（缓慢变化的内部电流的存在）的启发，开发了一种方法，用于在稀疏尖峰机制中模拟长短期记忆单元，以实现神经形态实现。基于尖峰的神经形态硬件有望比 GPU 等标准硬件更节能地实现深度神经网络 (DNN)。但这需要我们了解如何在基于事件的稀疏发射机制中模拟 DNN，否则能量优势就会丧失。特别是，解决序列处理任务的 DNN 通常采用长短期记忆单元，而这些单元很难在很少的尖峰情况下进行模拟。我们证明，许多生物神经元的一个方面，即每次尖峰后缓慢的后超极化电流，提供了一种有效的解决方案。后超极化电流可以轻松地在支持多室神经元模型的神经形态硬件中实现，例如英特尔的 Loihi 芯片。滤波器近似理论解释了为什么后超极化神经元可以模拟长短期记忆单元的功能。这产生了一种高度节能的时间序列分类方法。此外，它为一类重要的大型 DNN 的节能实现提供了基础，这些 DNN 提取单词和句子之间的关系以回答有关文本的问题。
Deep learning could be less energy intensive when implemented on spike-based neuromorphic chips. An approach inspired by a characteristic feature of biological neurons, the presence of slowly changing internal currents, is developed to emulate long short-term memory units in a sparse spiking regime for neuromorphic implementation.Spike-based neuromorphic hardware holds promise for more energy-efficient implementations of deep neural networks (DNNs) than standard hardware such as GPUs. But this requires us to understand how DNNs can be emulated in an event-based sparse firing regime, as otherwise the energy advantage is lost. In particular, DNNs that solve sequence processing tasks typically employ long short-term memory units that are hard to emulate with few spikes. We show that a facet of many biological neurons, slow after-hyperpolarizing currents after each spike, provides an efficient solution. After-hyperpolarizing currents can easily be implemented in neuromorphic hardware that supports multi-compartment neuron models, such as Intel's Loihi chip. Filter approximation theory explains why after-hyperpolarizing neurons can emulate the function of long short-term memory units. This yields a highly energy-efficient approach to time-series classification. Furthermore, it provides the basis for an energy-efficient implementation of an important class of large DNNs that extract relations between words and sentences in order to answer questions about the text.