Efficient Implementation of Spike-by-Spike Neural Networks using Stochastic and Approximative Techniques

该项目的总体目标是利用硬件和算法逼近技术提高峰值人工神经网络的效率。具体来说，该项目侧重于利用所谓的spike-by-spike网络（SbS）的稀疏性和鲁棒性。受真实神经系统的驱动，脉冲神经网络（SNN）为卷积神经网络（cnn）提供了一种替代方案。snn的内在优势，如更好的并行化和鲁棒性，保证了其巨大的改进潜力。这导致了大型研究项目在国际公司（如英特尔和IBM）和欧洲范围内的项目。这些研究努力试图模仿生物模型。相应的实现需要数百个具有大型复杂电路的核心。相比之下，SbS提供了计算需求和生物现实性之间的折衷，既保留了生物网络的基本优势，又允许更紧凑的技术实现。为了充分利用SbS的健壮性和效率，需要专用的硬件体系结构。通过将优化的硬件架构与随机和近似处理方法相结合，我们的目标是将基于脉冲的神经网络的性能和能量消耗至少提高一个数量级。这些发展将导致在通常的设计规范之外的特殊计算单元，因此必须通过ASIC中的实现进行测试。我们将该项目分为三个支柱：A)为SbS设计专用硬件架构，重点关注鲁棒性方面和使用随机技术的可能性。B)以共生方式将SbS与标准CNN方法相结合的混合架构的分析和设计。C)用ASIC实现评估理论方法，并开发可供其他研究人员使用的设计和评估框架。该项目将产生三重影响：1)对于电气工程，它将为神经网络提供新颖的硬件架构，其效率接近人类大脑。2)对于神经科学来说，它将更好地理解尖峰网络的鲁棒性。3)对于机器学习应用，它将以高效的计算方式为已建立的深度神经网络增加尖峰网络的鲁棒性和稀疏性。申请人的专业是理论神经科学和微电子学。他们在多学科合作方面的丰富经验为这个项目的成功提供了坚实的基础。

The overarching goal of this project is to improve the efficiency of spiking artificial neural networks using hardware and algorithmic approximation techniques. Specifically, the project focuses on the exploitation of the sparseness and robustness of so-called spike-by-spike networks (SbS). Motivated by real nervous systems, spiking neural networks (SNN) offer an alternative to convolutional neural networks (CNNs). The intrinsic advantages of SNNs, such as better parallelization and robustness promise a high potential for improvements. This has led to large research programs in international companies (e.g. Intel and IBM) and to Europe-wide projects. These research efforts try to imitate biological models. The corresponding implementations require hundreds of cores with large and complex circuits. In contrast, SbS offers a compromise between computational requirements and biological realism that preserves essential advantages of biological networks while allowing a much more compact technical implementation. To fully exploit the robustness and efficiency of SbS, dedicated hardware architectures are required. By combining optimized hardware architectures with stochastic and approximate processing approaches, we aim to improve the performance of pulse-based neural networks and their energy consumption by at least one order of magnitude. These developments will lead to special computing units that lie outside the usual design specifications and therefore have to be tested by a realization in an ASIC. We have organized the project in three pillars: A) Design of dedicated hardware architectures for SbS, focusing on robustness aspects and the possibility of using stochastic techniques. B) Analysis and design of hybrid architectures for combining SbS with standard CNN approaches in a symbiotic fashion. C) Evaluation of theoretical approaches with an ASIC implementation and development of a design and evaluation framework accessible to other researchers. This project will have a threefold impact: 1) For electrical engineering, it will provide novel hardware architectures for neural networks with an efficiency approaching that of the human brain. 2) For neuroscience, it will provide a better understanding of the robustness of spiking networks. 3) For machine learning applications, it will add the robustness and sparseness of spiking networks to established deep neural networks in a computationally efficient way. The applicants are specialized in theoretical neuroscience and microelectronics. The rich experience of their groups in multidisciplinary collaboration provides a solid basis for the success of this project.