NCS-FO: State Representations in Multi-purpose and Multi-region Neural Network Models of Cognition

Our understanding of the human brain has rapidly progressed with recent technological advances in both experimental neuroscience and artificial intelligence. Despite this, neither approach in isolation is able to explain how distinct cognitive functions such as learning, remembering, reasoning, and intuition emerge from processes inside the brain. In particular, we lack an understanding of how a relatively small and finite number of brain areas are used to accomplish this large and varied repertoire of cognitive functions. Bridging the fields of neuroscience and artificial intelligence, we seek to discover how the brain tracks the cognitive function that is currently engaged and switches between functions during ongoing behavior. We will apply new computational models, called "multi-purpose recurrent neural networks," to neural activity captured from the brains of different animal models to identify common mechanisms that allow animals to track and switch among cognitive functions. By bridging across experimental species, our findings will reveal fundamental features of brain processing. Further, our integrated approach, which uses a multi-disciplinary team of investigators and industry-academia partnerships, will promote cross-fertilization of knowledge and methods between artificial intelligence and neuroscience. We will also achieve broader societal benefits through collaboration with a graphic artist to develop graphic novel abstracts for widely comprehensible, visually appealing representations of the science for publication. A relatively small number of neural circuits in the brain are used to accomplish a large and varied repertoire of cognitive functions. Achieving this multi-purpose functionality requires neural circuits to both track the engaged function(s) and switch between them. How such tracking and switching is accomplished remains unclear. Computational models based on neural and behavioral data offer an opportunity to identify these key components of the brain's multipurpose functionality. However, existing models that simulate one task at a time lack the flexibility that underlies the brain's capacity to support many tasks. On the other hand, models that simulate multiple cognitive functions lack biologically realistic tracking and switching mechanisms. Here, we propose a new approach to this problem. We will develop a new class of data-inspired multi-purpose recurrent neural network (RNN) models that incorporate biologically plausible mechanisms to track the task being performed and the transitions between tasks. We will also analyze three distinct experimental datasets using machine learning to identify principles underlying multi-purpose functionality, particularly those that are conserved across species. Specifically, we will characterize multi-purpose functionality at the level of dynamic states. We define dynamic states as time-varying patterns of population activity that allow neural circuits to perform multiple tasks, engage them sequentially, and switch between them as task conditions or contexts change. We hypothesize that multi-purpose RNNs can incorporate dynamic states and simulate the brain's ability to track and switch between tasks, in a manner consistent with experimental data. First, we will develop and characterize data-inspired multi-purpose RNNs with internal state representations that track the engaged cognitive function/task performed. Second, we will incorporate functional and structural modularity into RNNs and analyze them in parallel with multi-region neural recordings. The resulting computational framework will enable us to identify key features of state representations and mechanisms underlying multi-purpose functionality in experimental data. What we discover will lay the foundation for understanding and testing core principles of how neural networks throughout the brain support diverse cognitive functions, enabling key advances in the study of cognition. Further, these robust, scalable multi-purpose RNN models containing internally represented states will better leverage existing large-scale neural data and galvanize new experiments designed to test model predictions. For instance, we expect to identify spatio-temporal markers from ongoing neural dynamics that predict upcoming behavioral transitions. In summary, we will build on recent advances in computer science, specifically, deep learning and other AI/ML-based techniques for neural networks, and bring them to bear on a key problem in neuroscience. Our integrative strategy maximally leverages the rapid pace of advances in computer science toward serving neuroscience and neuroengineering to catalyze new investigations beyond the confines of a lab.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Our understanding of the human brain has rapidly progressed with recent technological advances in both experimental neuroscience and artificial intelligence. Despite this, neither approach in isolation is able to explain how distinct cognitive functions such as learning, remembering, reasoning, and intuition emerge from processes inside the brain. In particular, we lack an understanding of how a relatively small and finite number of brain areas are used to accomplish this large and varied repertoire of cognitive functions. Bridging the fields of neuroscience and artificial intelligence, we seek to discover how the brain tracks the cognitive function that is currently engaged and switches between functions during ongoing behavior. We will apply new computational models, called "multi-purpose recurrent neural networks," to neural activity captured from the brains of different animal models to identify common mechanisms that allow animals to track and switch among cognitive functions. By bridging across experimental species, our findings will reveal fundamental features of brain processing. Further, our integrated approach, which uses a multi-disciplinary team of investigators and industry-academia partnerships, will promote cross-fertilization of knowledge and methods between artificial intelligence and neuroscience. We will also achieve broader societal benefits through collaboration with a graphic artist to develop graphic novel abstracts for widely comprehensible, visually appealing representations of the science for publication. A relatively small number of neural circuits in the brain are used to accomplish a large and varied repertoire of cognitive functions. Achieving this multi-purpose functionality requires neural circuits to both track the engaged function(s) and switch between them. How such tracking and switching is accomplished remains unclear. Computational models based on neural and behavioral data offer an opportunity to identify these key components of the brain's multipurpose functionality. However, existing models that simulate one task at a time lack the flexibility that underlies the brain's capacity to support many tasks. On the other hand, models that simulate multiple cognitive functions lack biologically realistic tracking and switching mechanisms. Here, we propose a new approach to this problem. We will develop a new class of data-inspired multi-purpose recurrent neural network (RNN) models that incorporate biologically plausible mechanisms to track the task being performed and the transitions between tasks. We will also analyze three distinct experimental datasets using machine learning to identify principles underlying multi-purpose functionality, particularly those that are conserved across species. Specifically, we will characterize multi-purpose functionality at the level of dynamic states. We define dynamic states as time-varying patterns of population activity that allow neural circuits to perform multiple tasks, engage them sequentially, and switch between them as task conditions or contexts change. We hypothesize that multi-purpose RNNs can incorporate dynamic states and simulate the brain's ability to track and switch between tasks, in a manner consistent with experimental data. First, we will develop and characterize data-inspired multi-purpose RNNs with internal state representations that track the engaged cognitive function/task performed. Second, we will incorporate functional and structural modularity into RNNs and analyze them in parallel with multi-region neural recordings. The resulting computational framework will enable us to identify key features of state representations and mechanisms underlying multi-purpose functionality in experimental data. What we discover will lay the foundation for understanding and testing core principles of how neural networks throughout the brain support diverse cognitive functions, enabling key advances in the study of cognition. Further, these robust, scalable multi-purpose RNN models containing internally represented states will better leverage existing large-scale neural data and galvanize new experiments designed to test model predictions. For instance, we expect to identify spatio-temporal markers from ongoing neural dynamics that predict upcoming behavioral transitions. In summary, we will build on recent advances in computer science, specifically, deep learning and other AI/ML-based techniques for neural networks, and bring them to bear on a key problem in neuroscience. Our integrative strategy maximally leverages the rapid pace of advances in computer science toward serving neuroscience and neuroengineering to catalyze new investigations beyond the confines of a lab.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.