Developing Bio-inspired Neuro-Engine for Intelligent Pervasive Computing

计算机科学和工程的最新进展引入了新的计算范式，如物联网（IoT）和环境智能（AmI）。这些新兴技术的核心思想依赖于与环境的分布式物理交互，其中低成本的计算节点在基础设施中无处不在地连接，以实时收集和分析本地数据，并且在功耗、通信开销、安全性和隐私方面都很有利。在这种情况下，利用人工智能（AI）和机器学习（ML）算法，这些算法大多具有显著的计算复杂性，将是极具挑战性的，并且需要在算法和硬件层面上大幅提高计算效率。深度学习（DL）作为机器学习的一个子集，是各种人工智能应用的核心。目前的深度学习应用基于人工神经网络（ANN），该网络由具有连续激活函数的神经元和一组连续时间加权输入组成。尽管这些网络非常强大，但它们也非常需要资源，这使得在边缘计算节点上部署它们非常具有挑战性。与人工神经网络不同，峰值神经网络（SNN）使用“异步”和离散时间峰值“事件”来编码、计算和传输信息。与中枢神经系统（Central Nervous System， CNS）的生物学行为一致，这些单独的峰值在时间上是稀疏的，并且具有均匀的振幅，因此能够通过在峰值速率和/或时间上编码数据来携带信息内容。由于尖峰数据编码和处理的性质，snn能够比ann更硬件友好和节能，因此对资源有限的处理器非常有吸引力。尽管具有丰富的数学和生物学背景，但实现和训练深度snn仍然是一个重大挑战。最近，一些有前途的方法已经被开发出来，以实现低成本的二值化人工神经网络和类尖峰输出，以及训练深度snn，与较低的硬件实现成本相比，它们提供了可接受的精度。本研究探索了一种紧密交织的算法-硬件协同设计技术，用于低成本的脉冲神经引擎，并遵循实现驱动的算法创新，以及可用于环境智能应用的定制而灵活的处理架构。该研究的主要目标是基于嵌入式系统特定应用的神经引擎的设计和实现，作为机器学习，生物启发计算，普适计算和硬件设计与优化的跨学科工作。作为一名博士后研究员，与SpiNNaker集团[1]合作，并在生物启发计算领域进行了几年的学术研究，这对申请人进行研究的灵感来源很大。

Recent progress in computer science and engineering has introduced new computing paradigms such as the Internet of Things (IoT) and Ambient Intelligence (AmI). The core idea in these emerging technologies relies on distributed physical interaction with environments, in which low-cost computational nodes are ubiquitously connected within an infrastructure to collect and analyze data locally in real-time and favorable in terms of power consumption, communication overhead, security, and privacy. In this context, utilizing Artificial Intelligence (AI) and Machine Learning (ML) algorithms, which mostly come with significant computational complexity, would be highly challenging and requires immense improvements in computational efficiency at both the algorithmic and hardware level. Deep learning (DL), as a subset of ML, is in the heart of various artificial intelligence applications. Current DL applications are based on Artificial Neural Networks (ANN) consisting of neurons with continuous activation functions and a set of continuous-time weighted inputs. Although these networks are extremely powerful, they are also very resource hungry, which makes it very challenging to deploy them on edge computing nodes. Unlike ANNs, Spiking Neural Networks (SNN), use "asynchronous" and discrete-time spiking "events" to code, compute and transmit information. Consistent with biological behavior of Central Nervous System (CNS), these individual spikes are sparse in time and have a uniform amplitude, thus have the capability to carry information content by encoding data in the spike rate and/or timing. Due to the nature of spiking data coding and processing, SNNs are capable to be more hardware friendly and energy-efficient than ANNs and are thus very appealing for resource-restricted processors. Despite their rich mathematical and biological background, implementation and training deep SNNs remains a major challenge. Recently, a few promising methods have been developed to implement low-cost binarized ANN and spike-like outputs as well as training deep SNNs, which offer acceptable accuracy compared with lower hardware implementation cost. This research explores a tightly interwoven Algorithm-Hardware codesign techniques for low-cost spiking neuro-engines and follows implementation driven algorithmic innovations, together with customized yet flexible processing architectures, that can be utilized in ambient intelligent applications. The main goal of the research is based on the design and implementation of application-specific neuro-engines for embedded systems as a cross-disciplinary work on machine learning, bio-inspired computing, pervasive computing, and hardware design and optimization. Working with SpiNNaker group [1], as a postdoc fellow researcher, and several years of academic research in the field of bio-inspired computing has been a great source of inspiration to the applicant to conduct his research.