Efficient Implementation of Spike-by-Spike Neural Networks using Stochastic and Approximative Techniques

这个项目的首要目标是使用硬件和算法逼近技术来提高人工神经网络的效率。具体地说，该项目专注于开发所谓的逐峰网络(SBS)的稀疏性和健壮性。在真实神经系统的激励下，脉冲神经网络(SNN)提供了卷积神经网络(CNN)的另一种选择。SNN的内在优势，如更好的并行化和健壮性，预示着极大的改进潜力。这导致了国际公司(如英特尔和IBM)的大型研究计划和全欧洲的项目。这些研究努力试图模仿生物模型。相应的实现需要数百个具有大型复杂电路的核心。相比之下，SBS提供了计算要求和生物现实主义之间的折衷，保留了生物网络的基本优势，同时允许更紧凑的技术实施。为了充分利用SBS的健壮性和效率，需要专用的硬件架构。通过将优化的硬件结构与随机和近似处理方法相结合，我们的目标是将基于脉冲的神经网络的性能和能量消耗提高至少一个数量级。这些发展将导致特殊的计算单元位于通常的设计规范之外，因此必须通过ASIC中的实现进行测试。我们将该项目组织为三个支柱：a)为SBS设计专用硬件架构，重点关注健壮性方面和使用随机技术的可能性。B)分析和设计以共生方式将SBS与标准CNN方法相结合的混合体系结构。C)通过ASIC实施和制定其他研究人员可利用的设计和评价框架，对理论方法进行评价。该项目将产生三方面的影响：1)对于电气工程，它将为神经网络提供新的硬件架构，其效率接近人脑。2)对于神经科学来说，它将更好地理解尖峰网络的健壮性。3)对于机器学习应用，它将以一种计算高效的方式将尖峰网络的健壮性和稀疏性添加到已建立的深度神经网络中。申请者的专业是理论神经科学和微电子学。他们团队在多学科协作方面的丰富经验为这个项目的成功提供了坚实的基础。

The overarching goal of this project is to improve the efficiency of spiking artificial neural networks using hardware and algorithmic approximation techniques. Specifically, the project focuses on the exploitation of the sparseness and robustness of so-called spike-by-spike networks (SbS). Motivated by real nervous systems, spiking neural networks (SNN) offer an alternative to convolutional neural networks (CNNs). The intrinsic advantages of SNNs, such as better parallelization and robustness promise a high potential for improvements. This has led to large research programs in international companies (e.g. Intel and IBM) and to Europe-wide projects. These research efforts try to imitate biological models. The corresponding implementations require hundreds of cores with large and complex circuits. In contrast, SbS offers a compromise between computational requirements and biological realism that preserves essential advantages of biological networks while allowing a much more compact technical implementation. To fully exploit the robustness and efficiency of SbS, dedicated hardware architectures are required. By combining optimized hardware architectures with stochastic and approximate processing approaches, we aim to improve the performance of pulse-based neural networks and their energy consumption by at least one order of magnitude. These developments will lead to special computing units that lie outside the usual design specifications and therefore have to be tested by a realization in an ASIC. We have organized the project in three pillars: A) Design of dedicated hardware architectures for SbS, focusing on robustness aspects and the possibility of using stochastic techniques. B) Analysis and design of hybrid architectures for combining SbS with standard CNN approaches in a symbiotic fashion. C) Evaluation of theoretical approaches with an ASIC implementation and development of a design and evaluation framework accessible to other researchers. This project will have a threefold impact: 1) For electrical engineering, it will provide novel hardware architectures for neural networks with an efficiency approaching that of the human brain. 2) For neuroscience, it will provide a better understanding of the robustness of spiking networks. 3) For machine learning applications, it will add the robustness and sparseness of spiking networks to established deep neural networks in a computationally efficient way. The applicants are specialized in theoretical neuroscience and microelectronics. The rich experience of their groups in multidisciplinary collaboration provides a solid basis for the success of this project.