CAREER: Rethinking Spiking Neural Networks from a Dynamical System Perspective

点击翻译按钮获取中文摘要

Neuromorphic computing algorithms are emerging to be a disruptive paradigm driving machine learning research. Despite the significant energy savings enabled by such brain-inspired systems due to event-driven network operation, neuromorphic spiking neural networks (SNNs) remain largely limited to static vision tasks and convolutional architectures. Hence, there is an unmet need to revisit scalable SNN training algorithms from the ground-up by forging stronger correlations with bio-plausibility to leverage the enormous potential of time-based information processing and local learning capability of SNNs for sequential tasks. The project approaches spiking architectures as event-driven dynamical systems, wherein learning occurs through the convergence towards equilibrium states. The idea that neurons collectively adjust themselves to configurations (according to the sensory input being fed into a neural network system) such that they can better predict the input data has been a popular hypothesis. The collective neuron states can be interpreted as explanations of the input data. This compelling central idea provides motivation for this research and education program by pursuing two recently emerging methodologies for training neural architectures viz - Equilibrium Propagation (EP) and Implicit Differentiation on Equilibrium (IDE) that bear strong synergies with each other. The research has far-reaching impacts on Artificial Intelligence (AI) and the semiconductor industry, and on society at large, where disruptive computing paradigms like neuromorphic computing, emerging device technologies and cross-layer optimizations can potentially achieve significant improvements in data-intensive machine learning workloads in contrast to state-of-the-art approaches. The project will consider an integrated research, education and outreach plan that considers interdisciplinary curriculum development, graduate and undergraduate research mentoring, K-12 involvement, online educational module development and enhancing minority research participation to train the next generation of researchers and engineers jointly in the fields of Machine Learning and Nanoelectronics.The presented end-to-end research agenda has the potential of enabling a quantum leap in the efficiency of AI platforms by pursuing a multi-disciplinary perspective -- combining insights from machine learning and dynamical systems to hardware. The project spans complementary and inter-twined explorations across the following thrust areas: (1) Enabling local learning in SNNs for complex tasks by integrating EP with modern Hopfield networks to implement attention mechanisms, (2) Using IDE for developing a scalable and computationally efficient training method for Spiking Language Models, (3) Cross-layer software-hardware-application optimizations for efficient implementation of the algorithmic innovations on neuromorphic platforms for large-scale sequential learning tasks. The cross-layer nature of the project ranging from machine learning, dynamical system modelling, cutting edge AI applications and hardware design will serve as an ideal platform to pursue an interdisciplinary workforce development program. If successful, the research has the potential of developing scalable, robust, power and energy efficient neuromorphic computing paradigms that are applicable to a broad range of sequential processing tasks - a significant shift from the huge computational requirements of conventional deep learning solutions like Large Language Models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.