EFRI BRAID: Scalable-Learning Neuromorphics

EFRI BRAID：可扩展学习神经形态

• 批准号: 2318152
• 负责人: Dmitri Strukov
• 金额: $ 200万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318152&HistoricalAwards=false
• 关键词: EFRI; BRAID; Scalable; Learning; Neuromorphics

Recent advances in the field of artificial intelligence suggest that the route toward higher-intelligence brain-inspired systems is via significantly increasing network size. However, such a rather evolutionary approach faces dire challenges. Training the largest machine learning models for natural language processing requires months of data-center-scale computations, in other words, enormous energy, time, and cost. Reserves for improvements, e.g., due to the refinement of algorithms and hardware, seem limited, and most importantly, further advances could no longer be fueled by semiconductor technology scaling. Additionally, state-of-the-art models rely on offline training with vast amount of training data, i.e., are not capable of continual real-time learning. Such challenges naturally bring more attention to the biological neural networks, which are living proof of superior, agile, and adaptive intelligence running on very energy-efficient “brain” hardware. Exciting opportunities are presented by recent developments in the theory of spiking neural networks, the most biologically plausible models, and neuromorphic circuits implemented with dense emerging memory devices. The proposed project aims to capitalize on these advances and address the most pressing challenges to develop scalable algorithms and hardware for human-brain-scale neuromorphic systems with practically useful (robust, fast, inexpensive) learning capabilities. The project will enable neuromorphic systems of immediate importance for many practical applications, including autonomous robots and vehicles, and biomedicine, including portable and personal medical devices. Furthermore, the broad algorithm-to-system nature of the proposed research provides attractive opportunities for high-school, undergraduate, and graduate students to explore novel research and get exposed to the emerging field of neuromorphic computing. The proposed project will pursue outreach activities by leveraging programs at participating universities, with a particular focus on attracting minority students.The key features of our research are hardware-friendly local learning algorithms, a framework for continual online “one-shot” learning, and variation-tolerant in-memory computing hardware circuits. Specifically, on the algorithmic front, we focus on recurrent spiking neural networks with biologically-plausible spike frequency adaptation neurons. We will build on local learning algorithms recently proposed by our team members that facilitate learning over longer time scales via synaptic plasticity. These algorithms will be further extended to support continual learning and co-optimized with the hardware using neural architecture search techniques. On the hardware front, the focus is on hybrid neuromorphic circuits that take advantage of analog in-memory computing. Critical hardware challenges, such as the scaling of network complexity and implementation of robust in-situ learning, will be addressed by utilizing ultra-high-density crosspoint devices implementing fixed-value weights of the learning algorithms and novel variation-tolerant memristive synapses featuring both short-term and long-term plasticity. Algorithms and hardware circuits will be holistically integrated into the learning-to-learn spiking neural network framework. Such a framework enables two kinds of learning – a slow incremental one mimicking developmental learning and fast “one-shot” learning utilizing network dynamics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能领域的最新进展表明，通往更高智能的大脑启发系统的道路是通过显着增加网络规模。然而，这种相当渐进的方法面临着严峻的挑战。训练用于自然语言处理的最大机器学习模型需要数月的数据中心规模计算，换句话说，需要大量的精力、时间和成本。改进准备金，例如，由于算法和硬件的改进，似乎是有限的，最重要的是，进一步的进步不再能通过半导体技术缩放来推动。此外，最先进的模型依赖于具有大量训练数据的离线训练，即，无法持续实时学习。 这些挑战自然会引起人们对生物神经网络的更多关注，生物神经网络是运行在非常节能的“大脑”硬件上的上级、敏捷和自适应智能的活生生的证明。令人兴奋的机会是由尖峰神经网络理论的最新发展，生物学上最合理的模型，以及密集的新兴记忆设备实现的神经形态电路。拟议的项目旨在利用这些进展并解决最紧迫的挑战，为具有实用（强大，快速，廉价）学习能力的人脑规模神经形态系统开发可扩展的算法和硬件。该项目将使神经形态系统对许多实际应用具有直接重要性，包括自主机器人和车辆，以及生物医学，包括便携式和个人医疗设备。此外，所提出的研究的广泛的算法到系统的性质为高中，本科和研究生提供了有吸引力的机会，探索新的研究，并接触到神经形态计算的新兴领域。该项目将通过利用参与大学的项目来开展外展活动，特别关注吸引少数民族学生。我们研究的主要特点是硬件友好的本地学习算法，持续在线“一次性”学习的框架，以及可变容差的内存计算硬件电路。具体来说，在算法方面，我们专注于具有生物学上合理的尖峰频率适应神经元的递归尖峰神经网络。我们将建立在我们的团队成员最近提出的局部学习算法的基础上，这些算法通过突触可塑性促进了更长时间尺度的学习。这些算法将被进一步扩展，以支持持续学习，并使用神经架构搜索技术与硬件协同优化。在硬件方面，重点是利用模拟内存计算的混合神经形态电路。关键的硬件挑战，如网络复杂性的扩展和鲁棒的原位学习的实现，将通过利用超高密度交叉点设备实现固定值的学习算法和新的变化容忍忆阻突触的权重来解决，具有短期和长期的可塑性。算法和硬件电路将整体集成到学习-学习尖峰神经网络框架中。这样的框架可以实现两种学习方式--一种是模仿发展性学习的缓慢渐进式学习，另一种是利用网络动态的快速“一次性”学习。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。