Memory Impedance for Efficient Complex-valued Neural Networks

人工智能存在硬件问题，因为目前的计算系统消耗的能量太多。这是不可持续的，并正在迅速成为一个关键的社会挑战。对计算能力飙升的需求远远超过了通过摩尔的扩展或创新架构解决方案所做的改进-现在计算需求每两个月翻一番。其直接结果是，培训最先进的复杂人工智能模型的成本从2012年的几美元增加到2020年的约1000万美元[自然604,255-260(2022)]。]。在能源资源有限的情况下(例如物联网设备)，挑战更加明显。迫切需要从根本上解决这个问题，并开发高效的人工智能系统。忆阻器(存储器+电阻)是未来非cmos计算解决方案的有力候选者，能够显著提高能效。具体地说，与效率低得多的传统冯·诺伊曼系统不同，记忆器件阵列实现了并行乘法和累加(MAC)操作，同时减少了对昂贵数据移动的需求。然而，忆阻器、晶体管和其他计算原语只对实值信号(数字或模拟形式)起作用。这是一个重要的限制，因为同时代表幅度和相位的复数要紧凑得多，并被用作生物医学、物理、机器人、通信、图像和音频处理、雷达等领域的标准。能够直接处理复值信号的计算基元将产生更好的能量效率，并使用更少的基本构建块提供更高的计算能力。我们建议通过推广内存的概念和开发能够直接处理复值信号的根本新型纳米级电子元件来解决这个问题。这种纳米级的设备还不存在；然而，如果开发出来，它们将在记录信号的边缘实现极高能效的直接处理。我们建议开发电可编程模拟记忆阻抗(存储器+阻抗)--一类新型的具有可编程阻抗的计算纳米器件。一旦实现了工作记忆阻抗，我们将通过构建记忆阻抗交叉开关和复值神经网络(CVNN)的第一个硬件加速器来探索和演示它们的功能。与常用的人工神经网络(ANN)不同，CVNN使用复数值表示权重和激活函数，其中幅度和相位都是必不可少的。CVNN虽然比传统的实值人工神经网络少得多，但对于复值数据的结构表现出了更好的性能[IEEE Trans。神经网络。学习。系统23,541-551，(2012年)]，并能解决传统神经网络中的已知问题(如过度匹配)[arxiv：2101.12249(2021年)]。该项目有可能在材料科学、微电子学、人工智能之间的界面上产生新的研究方向，并成为及时节能、高功能人工智能、神经形态和信号处理应用的游戏规则改变者。记忆阻抗的开发和CVNN的有效实施将在深度学习和人工智能领域提供新的令人兴奋的途径。

AI has a hardware problem because current computing systems consume far too much energy. This is not sustainable and is rapidly becoming a critical societal challenge. The soaring demand for computing power vastly outpaces improvements made through Moore's scaling or innovative architectural solutions - the computing demands now double every 2 months. As a direct consequence, the cost of training state-of-the-art sophisticated AI models increased from a few $ in 2012 to ~$10m in 2020 [Nature 604, 255-260 (2022). ]. The challenge is even more pronounced where energy resources are limited (e.g. IoT devices). There is a pressing need to address this issue at the fundamental level and develop efficient AI systems. Memristors (memory + resistor) are a strong candidate for future non-CMOS computing solutions, capable of yielding significant energy-efficiency improvements. Specifically, arrays of memristive devices enable parallel multiply-and-accumulate (MAC) operations while mitigating the need for costly data movement, unlike much less efficient conventional von Neumann systems. However, memristors, transistors, and other computational primitives operate only on real-valued signals (either in a digital or analogue form). This is a significant limitation because complex numbers representing both amplitude and phase are much more compact and are used as a standard in biomedical sciences, physics, robotics, communications, image & audio processing, radar, etc. Computational primitives capable of directly manipulating complex-valued signals would yield much better energy efficiency and provide higher computational power using fewer elemental building blocks.We propose to solve this problem by generalising the concept of memristance and developing fundamentally novel nanoscale electronic elements capable of directly processing complex-valued signals. Such nanoscale devices do not yet exist; however, if developed, they would enable extremely energy-efficient direct processing at the edge where signals are recorded. We propose to develop electrically programmable analogue memimpedors (memory + impedance) - new class of computational nanodevices with programmable impedance. Once working memimpedors are realised, we will explore and demonstrate their functionality by constructing memimpedor crossbars and the first hardware accelerator of complex-valued neural networks (CVNNs). CVNNs, in contrast to commonly used artificial neural networks (ANNs), use complex values for weights and activation functions, where both magnitude and phase are essential. CVNNs, although much less studied than conventional real-valued artificial neural networks, have demonstrated better performance for structures of complex-valued data [IEEE Trans. Neural Netw. Learn. Syst. 23, 541-551, (2012)] and can solve known problems in conventional ANNs (e.g. overfitting) [arXiv:2101.12249 (2021)]. This project has the potential to generate new research directions at the interface between materials science, microelectronics, AI, and be a game-changer for timely energy-efficient, highly functional AI, neuromorphic and signal processing applications. Development of memimpedors and efficient implementations of CVNN will offer new exciting avenues in deep learning and AI.