SGAI: Brain-Inspired Nanosystems for Smart and Green AI

该奖学金将为基于自由能原理（FEP）的算法和利用基于二维材料的新型纳米级设备的随机性的硬件平台的新人工智能范式奠定基础，使嵌入式系统具有前所未有的效率。基于深度学习算法的人工智能（AI）模型已经在从语言翻译到蛋白质折叠的各种任务中展示了超人的表现。然而，开发这种模型的成本——无论是在能源方面还是在时间方面——一直在飙升。例如，最近的研究估计，训练一个最先进的语言翻译模型的碳足迹可能高达纽约和旧金山之间的三次往返航班。造成这种低效率的一个主要原因是当今计算平台中使用的冯·诺伊曼架构——数据存储和数据处理单元在物理上是分开的。因此，运行这些算法需要以高精度表示的数据不断地来回传输。相比之下，人类的大脑是自然界最先进的计算引擎，它不断地做出复杂的认知决策，这也是基于嘈杂的感官数据和不精确的计算基础设施。大脑通过将信息编码成微小的电信号来实现这一惊人的壮举，这些电信号被称为“尖峰”，通过一个由“逻辑”和“记忆”单元（神经元和突触）组成的无缝连接的网络传输，而消耗的能量不到20瓦。显然，大脑的算法和硬件有一些本质上独一无二的东西！这项研究的动机是一种叫做自由能原理（FEP）的理论，它为大脑的认知效率提供了一个统一的基础。FEP的核心原则是，生物有机体倾向于通过改变从环境中接收到的感官输入，或通过修改允许它们感知世界和做出决定的内部状态，来最小化意外事件的发生。此外，由于FEP的理论基础假设大脑的模型本质上是概率性的，因此对数据或模型的高精度表示并不是严格的要求。因此，本研究将寻求利用纳米级器件的不良缺陷作为实现模型概率参数的资源的新方法。因此，这种方法可以使计算系统具有前所未有的效率，因为基本构建模块可以在极低的电压和电流下运行，从而避免不必要的数据移动。这项研究将首先基于自由能原理的数学思想开发人工神经网络，模仿大脑的尖峰触发通信特征。我们将创建人工智能模型，这些模型可以生成值得信赖的决策，并得到可量化的信心指标的支持。同时，我们还将展示使用纳米级器件的随机特性作为计算资源来实现这些算法的原型硬件平台。硬件原型将使用基于二维材料的新型纳米级设备，以及由工业合作伙伴制造的纳米级存储阵列，目标是将计算效率提高1000倍。因此，该奖学金将为新的智能和绿色人工智能范式奠定基础。

This fellowship will lay the foundations for a new AI paradigm featuring algorithms based on the free energy principle (FEP) and hardware platforms leveraging the stochasticity of novel nanoscale devices based on 2-dimensional materials, enabling embedded systems with unprecedented efficiency.Artificial Intelligence (AI) models based on deep learning algorithms have demonstrated super-human performance for a wide variety of such tasks - ranging from language translation to protein folding. However, the cost of developing such models - both in terms of energy and time - has been sky-rocketing. For example, recent studies estimate that the carbon footprint for training a state-of-the-art language translation model can be as high as 3 round-trip flights between New York and San Francisco.One major contributor to this inefficiency is the von Neumann architecture used in today's computing platforms - the data storage and data processing units are physically separated. Hence, running these algorithms require data that is represented in high precision to be constantly shuttled back and forth. Contrast this with the human brain, nature's most evolved computation engine, which continuously makes complex cognitive decisions, that too based on noisy sensory data and an imprecise computational infrastructure. The brain achieves this amazing feat by encoding information in tiny electrical signals called spikes that are transmitted through a seamlessly interconnected network of 'logic' and 'memory' units - neurons and synapses - all while consuming less than 20 Watts. Clearly, there is something fundamentally unique about the algorithms and hardware of the brain! The research in this fellowship is motivated by a theory called the free energy principle (FEP), which provides a unified foundation that underlies the cognitive efficiency of the brain. The central tenet of FEP is that biological organisms tend to minimize the occurrence of surprising events by acting to change the sensory inputs they receive from the environment or by modifying the internal states that allow them to perceive the world and make decisions. Furthermore, since the theoretical foundation of FEP assumes that the brain's models are inherently probabilistic, representing data or the model in high precision is not a strict requirement. Hence, the research in the fellowship will pursue the novel approach of using the undesirable imperfections of nanoscale devices as a resource for implementing the probabilistic parameters of the model. This approach can hence lead to computational systems with unprecedented efficiency as the basic building blocks can be operated at drastically lower voltages and currents, avoiding unnecessary data movement. This research will first develop artificial neural networks that mimic the spike-triggered communication feature of the brain based on the mathematical ideas of the free energy principle. We will create AI models that can generate decisions that are trustworthy and can be supported with quantifiable confidence metrics. In parallel, we will also demonstrate prototype hardware platforms that implement these algorithms using the stochastic properties of nanoscale devices as a resource for computation. Hardware prototypes will be built using novel nanoscale devices that are based on 2-dimensional materials as well as nanoscale memory arrays built by industrial partners targeting a 1000-fold improvement in computational efficiency compared to what is possible today. Thus, the fellowship will lay the foundations of a new Smart and Green AI paradigm.