Three-Dimensional Multilayer Nanomagnetic Arrays for Neuromorphic Low-Energy Magnonic Processing

计算和人工智能(AI)的能源成本正在失控，预计到2030年将达到全球能源消耗的20.9%。训练神经网络来机器人解决魔方玩具需要2.8GWh，而人脑只消耗~20W。最近大型机器学习模型的成功，如OpenAI的GPT-3和Chat-GPT，伴随着巨大的碳足迹-Chat-GPT在训练期间消耗了1500万美元的电力，并产生了约552吨二氧化碳。其正在进行的能源账单估计为每月约300万美元，以及相应的温室气体排放水平。这种不可持续的能源消耗既是实现净零未来的真正障碍，也是人工智能计算能力的上限。这个问题的很大一部分是，我们目前正试图用与大脑完全不同的计算机进行类似大脑的计算。今天的计算机在独立的内存和处理器单元之间传输数据所使用的能量远远超过实际处理的能量，而大脑中的神经元提供集成的内存和处理--这是它们能量成本大幅降低的关键驱动因素。因此，迫切需要以类似大脑(神经形态)的方式工作的硬件系统，在同一个单位内原生地存储和处理信息。在许多方面，纳米磁铁的行为很像大脑中的神经元。它们可以对周围磁铁的行为做出反应，将极点从北向南翻转，类似于神经元发送电流的方式。纳米磁铁可以记住它们过去看到的东西，并相应地改变它们的行为，从它们的经验中学习，并在语音识别和模式预测等任务中逐渐改进。纳米磁铁既提供记忆，因为它们能够记住数千年的数据(硬盘最初是由纳米磁铁制成的，因此)，也提供处理能力，因为它们能够对以GHz速度输入的数据做出非线性反应--以一种特殊的方式振荡，这种方式被称为“魔术”。事实上，驱动现代软件神经网络的数学起源于物理学家在20世纪70年代开发的描述强相互作用磁网络的理论框架。早期的机器学习社区采用了这些框架(最初称为Hopfield网络)，并将其改编和改进为今天的神经网络。自从机器学习的早期成功以来，工程师们一直梦想着去除抽象的软件层，直接在物理磁网络中实现机器学习。然而，直到最近，提供有效的数据输入和输出方案的工程挑战阻碍了此类系统的实现。我们的团队现在已经解决了这些问题，完成了世界上第一个在纳米磁阵列中进行神经形态计算的例子，使用纳米磁阵列的磁振子动力学来处理信息并解决一系列人工智能任务，包括对复杂生物信号的未来预测。我们现在有了一种方法，可以在不额外能源成本的情况下大幅提高我们的人工智能计算能力，通过将我们的纳米磁阵列从2D结构转移到3D结构，我们的早期结果和模拟表明，我们的计算能力可能会从根本上增加。在这个项目中，我们将在英国由职业早期研究员Jack Gartside领导的小组和由世界专家Benjamin Jungfleisch教授领导的美国小组之间合作，以测试我们的想法&使低能源、低碳的人工智能更接近现实。

The energy cost of computing and artificial intelligence (AI) is spiraling out of control, forecast to reach 20.9% of global energy consumption by 2030. Training a neural net to robotically solve a Rubik's Cube toy consumed 2.8 GWh , while human brains consume just ~20 W. The recent successes of large machine learning models such as OpenAI's GPT-3 and Chat-GPT are accompanied by huge carbon footprints - Chat-GPT consumed $15 million in electricity during training & generated ~552 tons of CO2 . Its ongoing energy bill is estimated at ~$3 million/month, with accompanying levels of greenhouse emissions. This unsustainable energy consumption represents both a real barrier to reaching net-zero futures and a ceiling on the power of AI computing.A big part of this problem is that we're currently trying to do brain-like computing with computers that are nothing like a brain. Today's computers use far more energy shuttling data between separate memory and processor units than actually processing, whereas neurons in the brain provide integrated memory and processing - a key driver for their radically lower energy cost. Consequently, there is a pressing need for hardware systems that function in a brain-like (neuromorphic) manner, storing and processing information natively in the same unit.In many ways, nanomagnets behave a lot like neurons in the brain. They can react to the behaviour of surrounding magnets, flipping their poles from north to south similar to how neurons send jolts of electricity. Nanomagnets can remember what they've seen in the past and change their behaviour in response to this, learning from their experiences and gradually improving at tasks like voice recognition and pattern prediction. Nanomagnets provide both memory from their ability to remember data for 1000s of years (hard drives were originally made from nanomagnets for this reason), and processing from their ability to react nonlinearly to input data at GHz speeds - oscillating in a special way known as 'magnonics'. Indeed, the maths powering modern software neural networks originate from theoretical frameworks developed by physicists in the 1970's to describe strongly-interacting magnetic networks . The early machine learning community adopted these frameworks (originally termed Hopfield networks ) and adapted & refined them into the neural networks of today. Since the early successes of machine learning, engineers have dreamt of removing the software layer of abstraction and implementing machine learning directly in physical magnetic networks. However until recently, the engineering challenges of providing efficient data input and output schemes had prevented realisation of such systems. Our team have now solved these issues to accomplish the world-first example of neuromorphic computing in nanomagnetic arrays, using the magnon dynamics of a nanomagnetic array to process information and solve a range of AI tasks including future prediction of complex biological signals.We now have a way to massively improve the power of our AI computation at no extra energy cost, by moving our nanomagnetic arrays from 2D into 3D structures, our early results and simulations show that our computing power is likely to radically increase. In this project, we will work between a group in the UK led by early-career researcher Jack Gartside and a group in the USA lead by world-expert Prof. Benjamin Jungfleisch to test our ideas & bring low-energy, low-carbon AI one step closer to reality.