Energy-Efficient Models for High-Dimensional Spike Train Classification using Sparse Spiking Neural Networks
Energy-Efficient Models for High-Dimensional Spike Train Classification using Sparse Spiking Neural Networks
复制标题
DOI：
10.1145/3447548.3467252
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Hang Yin;John Lee;Thomas Hartvigsen;Sihong Xie
中科院分区：
文献类型：
--
作者：
Hang Yin;John Lee;Thomas Hartvigsen;Sihong Xie
尖峰序列分类是医疗保健和移动传感等许多领域的一个重要问题，其中每个尖峰序列都是二进制值的高维时间序列。尖峰训练分类的传统研究主要集中在资源充足的设置（例如，在 GPU 服务器上）下开发尖峰神经网络（SNN）。 SNN 的神经元通常在每一层中紧密连接。然而，在许多实际应用中，我们经常需要在资源受限的平台（例如移动设备）上部署 SNN 模型来分析高维尖峰序列数据。密集连接的 SNN 对资源的高要求使得它们难以部署在移动设备上。在本文中，我们研究了具有稀疏连接神经元的节能 SNN 问题。我们提出了一种具有稀疏时空编码的 SNN 模型。我们的解决方案基于 SNN 中权重的重新参数化以及优化过程中稀疏正则化的应用。我们将我们的工作与最先进的 SNN 进行比较，并证明我们的稀疏 SNN 在具有可比分类精度的神经形态数据集和标准数据集上实现了显着更好的计算效率。此外，与密集连接的 SNN 相比，我们通过大量实验表明，我们的方法在小规模数据集上具有更好的泛化能力。
Spike train classification is an important problem in many areas such as healthcare and mobile sensing, where each spike train is a high-dimensional time series of binary values. Conventional research on spike train classification mainly focus on developing Spiking Neural Networks (SNNs) under resource-sufficient settings (e.g., on GPU servers). The neurons of the SNNs are usually densely connected in each layer. However, in many real-world applications, we often need to deploy the SNN models on resource-constrained platforms (e.g., mobile devices) to analyze high-dimensional spike train data. The high resource requirement of the densely-connected SNNs can make them hard to deploy on mobile devices. In this paper, we study the problem of energy-efficient SNNs with sparsely-connected neurons. We propose an SNN model with sparse spatio-temporal coding. Our solution is based on the re-parameterization of weights in an SNN and the application of sparsity regularization during optimization. We compare our work with the state-of-the-art SNNs and demonstrate that our sparse SNNs achieve significantly better computational efficiency on both neuromorphic and standard datasets with comparable classification accuracy. Furthermore, compared with densely-connected SNNs, we show that our method has a better capability of generalization on small-size datasets through extensive experiments.