权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Memory Impedance for Efficient Complex-valued Neural Networks

高效复值神经网络的内存阻抗

基本信息

批准号：
EP/X018431/1
负责人：
Adnan Mehonic
金额：
$ 25.73万
依托单位：
University College London
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2023
资助国家：
英国
起止时间：
2023 至无数据
项目状态：
未结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FX018431%2F1
关键词：
Memory Impedance Efficient Complex valued

项目摘要

AI has a hardware problem because current computing systems consume far too much energy. This is not sustainable and is rapidly becoming a critical societal challenge. The soaring demand for computing power vastly outpaces improvements made through Moore's scaling or innovative architectural solutions - the computing demands now double every 2 months. As a direct consequence, the cost of training state-of-the-art sophisticated AI models increased from a few $ in 2012 to ~$10m in 2020 [Nature 604, 255-260 (2022). ]. The challenge is even more pronounced where energy resources are limited (e.g. IoT devices). There is a pressing need to address this issue at the fundamental level and develop efficient AI systems. Memristors (memory + resistor) are a strong candidate for future non-CMOS computing solutions, capable of yielding significant energy-efficiency improvements. Specifically, arrays of memristive devices enable parallel multiply-and-accumulate (MAC) operations while mitigating the need for costly data movement, unlike much less efficient conventional von Neumann systems. However, memristors, transistors, and other computational primitives operate only on real-valued signals (either in a digital or analogue form). This is a significant limitation because complex numbers representing both amplitude and phase are much more compact and are used as a standard in biomedical sciences, physics, robotics, communications, image & audio processing, radar, etc. Computational primitives capable of directly manipulating complex-valued signals would yield much better energy efficiency and provide higher computational power using fewer elemental building blocks.We propose to solve this problem by generalising the concept of memristance and developing fundamentally novel nanoscale electronic elements capable of directly processing complex-valued signals. Such nanoscale devices do not yet exist; however, if developed, they would enable extremely energy-efficient direct processing at the edge where signals are recorded. We propose to develop electrically programmable analogue memimpedors (memory + impedance) - new class of computational nanodevices with programmable impedance. Once working memimpedors are realised, we will explore and demonstrate their functionality by constructing memimpedor crossbars and the first hardware accelerator of complex-valued neural networks (CVNNs). CVNNs, in contrast to commonly used artificial neural networks (ANNs), use complex values for weights and activation functions, where both magnitude and phase are essential. CVNNs, although much less studied than conventional real-valued artificial neural networks, have demonstrated better performance for structures of complex-valued data [IEEE Trans. Neural Netw. Learn. Syst. 23, 541-551, (2012)] and can solve known problems in conventional ANNs (e.g. overfitting) [arXiv:2101.12249 (2021)]. This project has the potential to generate new research directions at the interface between materials science, microelectronics, AI, and be a game-changer for timely energy-efficient, highly functional AI, neuromorphic and signal processing applications. Development of memimpedors and efficient implementations of CVNN will offer new exciting avenues in deep learning and AI.

人工智能存在硬件问题，因为目前的计算系统消耗了太多的能量。这是不可持续的，正迅速成为一个重大的社会挑战。对计算能力的不断增长的需求远远超过了摩尔的扩展或创新架构解决方案所带来的改进——计算需求现在每两个月翻一番。其直接后果是，训练最先进的复杂人工智能模型的成本从2012年的几美元增加到2020年的约1000万美元[Nature 604, 255-260(2022)]。]。在能源资源有限的情况下（例如物联网设备），挑战更加明显。迫切需要从根本上解决这个问题，开发高效的人工智能系统。忆阻器（存储器+电阻器）是未来非cmos计算解决方案的有力候选，能够显著提高能效。具体来说，记忆器件阵列可以实现并行的乘法累加（MAC）操作，同时减少了对昂贵的数据移动的需求，这与效率低得多的传统冯·诺伊曼系统不同。然而，忆阻器、晶体管和其他计算基元只能处理实值信号（数字或模拟形式）。这是一个重要的限制，因为表示幅度和相位的复数要紧凑得多，并且在生物医学、物理、机器人、通信、图像和音频处理、雷达等领域被用作标准。能够直接操作复杂值信号的计算原语将产生更好的能源效率，并使用更少的基本构建块提供更高的计算能力。我们建议通过推广忆阻的概念和开发能够直接处理复杂值信号的新型纳米级电子元件来解决这个问题。这种纳米级的装置目前还不存在；然而，如果发展起来，它们将在记录信号的边缘实现极其节能的直接处理。我们建议开发电可编程模拟记忆阻抗（存储器+阻抗）-一类具有可编程阻抗的新型计算纳米器件。一旦实现工作memememdors，我们将通过构建memememdors交叉杆和复值神经网络（CVNNs）的第一个硬件加速器来探索和演示其功能。与常用的人工神经网络（ann）相比，cvnn使用复值作为权重和激活函数，其中幅度和相位都是必不可少的。尽管与传统的实值人工神经网络相比，cvnn的研究要少得多，但它在处理复杂值数据结构方面表现出了更好的性能。神经。学习。系统学报，23,541-551,(2012)]，可以解决传统人工神经网络中的已知问题（例如过拟合）[arXiv:2101.12249(2021)]。该项目有可能在材料科学、微电子学、人工智能之间的界面产生新的研究方向，并成为及时节能、高功能人工智能、神经形态和信号处理应用的游戏规则改变者。记忆阻抗的发展和CVNN的有效实现将为深度学习和人工智能提供新的令人兴奋的途径。