SGAI: Brain-Inspired Nanosystems for Smart and Green AI

该奖学金将为新的人工智能范式奠定基础，该范式具有基于自由能原理（FEP）的算法和利用基于二维材料的新型纳米级器件的随机性的硬件平台，基于深度学习算法的人工智能（AI）模型在各种各样的此类任务中表现出了超人的性能-从语言翻译到蛋白质折叠。然而，开发这种模型的成本-无论是在能源还是时间方面-都在飙升。例如，最近的研究估计，训练一个最先进的语言翻译模型的碳足迹可能高达纽约和旧金山弗朗西斯科之间的3个往返航班。造成这种低效率的一个主要原因是当今计算平台中使用的冯诺依曼架构-数据存储和数据处理单元在物理上是分开的。因此，运行这些算法需要以高精度表示的数据不断来回穿梭。与此形成对比的是，人类大脑是自然界最进化的计算引擎，它不断做出复杂的认知决策，也是基于嘈杂的感官数据和不精确的计算基础设施。大脑通过将信息编码成微小的电信号来实现这一惊人的壮举，这些电信号被称为尖峰，通过一个无缝互连的“逻辑”和“记忆”单元网络-神经元和突触-传输，而所有这些都消耗不到20瓦。显然，大脑的算法和硬件有着根本性的独特之处！这项研究的动机是一种名为自由能原理（FEP）的理论，该理论为大脑的认知效率提供了统一的基础。FEP的核心原则是生物有机体倾向于通过改变它们从环境中接收的感官输入或通过修改允许它们感知世界并做出决定的内部状态来最大限度地减少意外事件的发生。此外，由于FEP的理论基础假设大脑的模型本质上是概率性的，因此以高精度表示数据或模型并不是一个严格的要求。因此，该奖学金的研究将采用新的方法，使用纳米级器件的不期望的缺陷作为实现模型的概率参数的资源。因此，这种方法可以使计算系统具有前所未有的效率，因为基本构建模块可以在极低的电压和电流下运行，避免不必要的数据移动。 本研究将首先基于自由能原理的数学思想开发模拟大脑尖峰触发通信特征的人工神经网络。我们将创建人工智能模型，这些模型可以生成值得信赖的决策，并且可以通过可量化的置信度指标来支持。同时，我们还将展示原型硬件平台，使用纳米级设备的随机特性作为计算资源来实现这些算法。硬件原型将使用基于二维材料的新型纳米级器件以及由工业合作伙伴构建的纳米级存储器阵列构建，目标是将计算效率提高1000倍。因此，该奖学金将为新的智能和绿色人工智能范式奠定基础。

This fellowship will lay the foundations for a new AI paradigm featuring algorithms based on the free energy principle (FEP) and hardware platforms leveraging the stochasticity of novel nanoscale devices based on 2-dimensional materials, enabling embedded systems with unprecedented efficiency.Artificial Intelligence (AI) models based on deep learning algorithms have demonstrated super-human performance for a wide variety of such tasks - ranging from language translation to protein folding. However, the cost of developing such models - both in terms of energy and time - has been sky-rocketing. For example, recent studies estimate that the carbon footprint for training a state-of-the-art language translation model can be as high as 3 round-trip flights between New York and San Francisco.One major contributor to this inefficiency is the von Neumann architecture used in today's computing platforms - the data storage and data processing units are physically separated. Hence, running these algorithms require data that is represented in high precision to be constantly shuttled back and forth. Contrast this with the human brain, nature's most evolved computation engine, which continuously makes complex cognitive decisions, that too based on noisy sensory data and an imprecise computational infrastructure. The brain achieves this amazing feat by encoding information in tiny electrical signals called spikes that are transmitted through a seamlessly interconnected network of 'logic' and 'memory' units - neurons and synapses - all while consuming less than 20 Watts. Clearly, there is something fundamentally unique about the algorithms and hardware of the brain! The research in this fellowship is motivated by a theory called the free energy principle (FEP), which provides a unified foundation that underlies the cognitive efficiency of the brain. The central tenet of FEP is that biological organisms tend to minimize the occurrence of surprising events by acting to change the sensory inputs they receive from the environment or by modifying the internal states that allow them to perceive the world and make decisions. Furthermore, since the theoretical foundation of FEP assumes that the brain's models are inherently probabilistic, representing data or the model in high precision is not a strict requirement. Hence, the research in the fellowship will pursue the novel approach of using the undesirable imperfections of nanoscale devices as a resource for implementing the probabilistic parameters of the model. This approach can hence lead to computational systems with unprecedented efficiency as the basic building blocks can be operated at drastically lower voltages and currents, avoiding unnecessary data movement. This research will first develop artificial neural networks that mimic the spike-triggered communication feature of the brain based on the mathematical ideas of the free energy principle. We will create AI models that can generate decisions that are trustworthy and can be supported with quantifiable confidence metrics. In parallel, we will also demonstrate prototype hardware platforms that implement these algorithms using the stochastic properties of nanoscale devices as a resource for computation. Hardware prototypes will be built using novel nanoscale devices that are based on 2-dimensional materials as well as nanoscale memory arrays built by industrial partners targeting a 1000-fold improvement in computational efficiency compared to what is possible today. Thus, the fellowship will lay the foundations of a new Smart and Green AI paradigm.