Exploring the Connection Between Binary and Spiking Neural Networks
Exploring the Connection Between Binary and Spiking Neural Networks
复制标题
DOI：
10.3389/fnins.2020.00535
复制
发表时间：
2020-06-24
影响因子：
4.3
通讯作者：
Sengupta, Abhronil
中科院分区：
文献类型：
--
作者：
Lu, Sen;Sengupta, Abhronil
片上边缘智能需要探索算法技术，以减少当前机器学习框架的计算要求。这项工作旨在弥合训练二进制神经网络和尖峰神经网络的最新算法进展，而神经网络却是由相同的动机驱动的，但两者之间的协同作用尚未得到充分探索。我们表明，在极端量化制度中的训练尖峰神经网络在CIFAR-100和Imagenet等大规模数据集上几乎完全精确地精确。这项工作的一个重要含义是，可以通过迎合二进制神经网络的“内存”硬件加速器来启用二进制尖峰神经网络，而不会因二进制而遭受任何准确的降解。我们利用标准培训技术来用于非加速网络，通过转换过程来生成尖峰网络，还进行广泛的经验分析，并探索简单的设计时间和运行时优化技术，以减少峰值网络的推理延迟（用于二进制和完整的推理。 - 精确模型）按先前工作的数量级按数量级。我们的实施源代码和训练有素的模型可在https://github.com/neurocomplab-psu/snn-conversion上找到。
On-chip edge intelligence has necessitated the exploration of algorithmic techniques to reduce the compute requirements of current machine learning frameworks. This work aims to bridge the recent algorithmic progress in training Binary Neural Networks and Spiking Neural Networks-both of which are driven by the same motivation and yet synergies between the two have not been fully explored. We show that training Spiking Neural Networks in the extreme quantization regime results in near full precision accuracies on large-scale datasets like CIFAR-100 and ImageNet. An important implication of this work is that Binary Spiking Neural Networks can be enabled by "In-Memory" hardware accelerators catered for Binary Neural Networks without suffering any accuracy degradation due to binarization. We utilize standard training techniques for non-spiking networks to generate our spiking networks by conversion process and also perform an extensive empirical analysis and explore simple design-time and run-time optimization techniques for reducing inference latency of spiking networks (both for binary and full-precision models) by an order of magnitude over prior work. Our implementation source code and trained models are available at https://github.com/NeuroCompLab-psu/SNN-Conversion.