Addressing neuron-to-network energy-efficiency gap by investigating neuromorphic processors as a unified dynamical system

该项目研究了一个完全耦合的模拟神经形态架构，其中整个学习网络被设计为一个统一的动态系统，使用短期和长期网络动态编码信息。在基本层面上，生物神经元产生的单个动作电位没有针对能量进行优化，并且比图形处理单元（GPU）或张量处理单元（TPU）中的等效浮点操作消耗更多的功率。然而，与通常使用1 Peta 8位/16位浮点运算或更多浮点运算的特定应用深度学习平台相比，人类大脑中的耦合神经元群体使用~100千兆的粗神经运算（或峰值）可以学习和实现各种功能。该提案的智力优势在于通过研究基于生长转换神经网络（GTNN）的动态系统框架来设计节能、实时的神经形态处理器，从而解决了神经元与网络之间的能量效率差距。首先，该项目正在研究GTNN如何利用种群动态来提高系统能源效率，同时实时优化学习或任务目标。其次，该项目正在研究短期和长期网络动态如何能够将提议的GTNN扩展到数十亿个神经元，而不需要明确的尖峰路由，并利用网络的极限环固定点作为模拟存储器。第三，该项目正在研究一种连续时间的模拟GTNN处理器，与其他基准神经形态和深度学习处理器相比，该处理器可用于证明所提出方法的能效。该项目还支持GTNN模拟器的开源开发，该模拟器将传播到神经网络、神经形态工程和神经科学社区。这个开源工具也将成为IEEE会议上组织教程和特别会议的基础。通过这个项目开发的示范平台被用来与华盛顿大学其他国家科学基金会赞助的外展项目联系起来，其中包括向属于代表性不足群体的学生提供外展服务。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project investigates a fully-coupled, analog neuromorphic architecture where the entire learning network is designed as a unified dynamical system encoding information using short-term and long-term network dynamics. At the fundamental level, a single action potential generated by a biological neuron is not optimized for energy and consumes significantly more power than an equivalent floating-point operation in a Graphical Processing Unit (GPU) or a Tensor Processing Unit (TPU). Yet a population of coupled neurons in the human brain, using ~100 Giga coarse neural operations (or spikes) can learn and implement diverse functions compared to an application-specific deep-learning platform that typically use ~1 Peta 8-bit/16-bit floating-point operations or more. The intellectual merit of this proposal is addressing this neuron-to-network energy-efficiency gap by investigating a growth-transform neural network (GTNN) based dynamical systems framework for designing energy-efficient, real-time neuromorphic processors. First, the project is investigating how a GTNN can exploit population dynamics to improve system energy-efficiency, while optimizing a learning or task objective in real-time. Second, the project is investigating how short-term and long-term network dynamics can enable scaling the proposed GTNN to billions of neurons without the need for explicit spike-routing and by exploiting network's limit-cycle fixed-points as analog memory. Third, the project is investigating a continuous-time, analog GTNN processor that can be used to demonstrate the energy-efficiency of proposed approach compared to other benchmark neuromorphic and deep-learning processors. The project is also supporting open-source development of a GTNN simulator which will be disseminated to the neural network, neuromorphic engineering and neuroscience communities. The open-source tool will also form the basis for organizing tutorials and special sessions at IEEE conferences. The demonstration platforms developed through this project is being be used to connect with other NSF sponsored outreach programs at Washington University, which includes outreach to students belonging to underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.