NCS-FO: State Representations in Multi-purpose and Multi-region Neural Network Models of Cognition

随着实验神经科学和人工智能领域的最新技术进步，我们对人脑的理解迅速进步。尽管如此，这两种孤立的方法都不能解释学习、记忆、推理和直觉等不同的认知功能是如何从大脑内部的过程中出现的。特别是，我们缺乏对相对较小和有限数量的大脑区域是如何用来完成这一庞大而多样的认知功能的理解。作为神经科学和人工智能领域的桥梁，我们试图发现大脑是如何跟踪目前正在进行的认知功能，以及在正在进行的行为中如何在不同功能之间切换的。我们将把新的计算模型，称为“多用途递归神经网络”，应用于从不同动物模型的大脑中捕捉到的神经活动，以确定允许动物跟踪认知功能并在认知功能之间切换的共同机制。通过跨越实验物种，我们的发现将揭示大脑处理的基本特征。此外，我们的综合方法使用了一个由多学科研究人员组成的团队和产业界与学术界的伙伴关系，将促进人工智能和神经科学之间的知识和方法的交叉培养。我们还将通过与一位图形艺术家合作开发图形小说摘要，为出版的科学提供广泛理解的、视觉上有吸引力的表现，从而获得更广泛的社会效益。大脑中数量相对较少的神经回路被用来完成大量不同的认知功能。要实现这一多功能功能，需要神经电路既跟踪参与的功能(S)，又在它们之间切换。目前尚不清楚这种跟踪和切换是如何完成的。基于神经和行为数据的计算模型提供了识别大脑多用途功能的这些关键组成部分的机会。然而，现有的一次模拟一项任务的模型缺乏支撑大脑支持多项任务的能力的灵活性。另一方面，模拟多种认知功能的模型缺乏生物上真实的跟踪和切换机制。在这里，我们提出了一个新的方法来解决这个问题。我们将开发一类新的受数据启发的多用途递归神经网络(RNN)模型，该模型结合了生物学上可信的机制来跟踪正在执行的任务和任务之间的转换。我们还将使用机器学习分析三个不同的实验数据集，以确定多用途功能背后的原理，特别是那些跨物种保守的原理。具体地说，我们将在动态级别描述多用途功能。我们将动态定义为群体活动的时变模式，这种模式允许神经回路执行多项任务，按顺序进行，并随着任务条件或环境的变化而在它们之间切换。我们假设多用途RNN可以结合动态，并以与实验数据一致的方式模拟大脑跟踪任务和在任务之间切换的能力。首先，我们将开发和表征受数据启发的多用途RNN，它具有跟踪参与的认知功能/执行的任务的内部状态表示。其次，我们将把功能和结构的模块化融入到RNN中，并结合多区域神经记录对它们进行并行分析。由此产生的计算框架将使我们能够在实验数据中识别状态表示的关键特征和支持多用途功能的机制。我们的发现将为理解和测试大脑中的神经网络如何支持不同的认知功能的核心原则奠定基础，从而使认知研究取得关键进展。此外，这些包含内部表示状态的健壮、可扩展的多用途RNN模型将更好地利用现有的大规模神经数据，并激发旨在测试模型预测的新实验。例如，我们希望从正在进行的神经动力学中识别预测即将到来的行为转变的时空标记。总之，我们将以计算机科学的最新进展为基础，特别是深度学习和其他基于AI/ML的神经网络技术，并将它们应用于神经科学中的一个关键问题。我们的综合战略最大限度地利用计算机科学的快速发展步伐，为神经科学和神经工程服务，以催化超出实验室范围的新研究。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Our understanding of the human brain has rapidly progressed with recent technological advances in both experimental neuroscience and artificial intelligence. Despite this, neither approach in isolation is able to explain how distinct cognitive functions such as learning, remembering, reasoning, and intuition emerge from processes inside the brain. In particular, we lack an understanding of how a relatively small and finite number of brain areas are used to accomplish this large and varied repertoire of cognitive functions. Bridging the fields of neuroscience and artificial intelligence, we seek to discover how the brain tracks the cognitive function that is currently engaged and switches between functions during ongoing behavior. We will apply new computational models, called "multi-purpose recurrent neural networks," to neural activity captured from the brains of different animal models to identify common mechanisms that allow animals to track and switch among cognitive functions. By bridging across experimental species, our findings will reveal fundamental features of brain processing. Further, our integrated approach, which uses a multi-disciplinary team of investigators and industry-academia partnerships, will promote cross-fertilization of knowledge and methods between artificial intelligence and neuroscience. We will also achieve broader societal benefits through collaboration with a graphic artist to develop graphic novel abstracts for widely comprehensible, visually appealing representations of the science for publication. A relatively small number of neural circuits in the brain are used to accomplish a large and varied repertoire of cognitive functions. Achieving this multi-purpose functionality requires neural circuits to both track the engaged function(s) and switch between them. How such tracking and switching is accomplished remains unclear. Computational models based on neural and behavioral data offer an opportunity to identify these key components of the brain's multipurpose functionality. However, existing models that simulate one task at a time lack the flexibility that underlies the brain's capacity to support many tasks. On the other hand, models that simulate multiple cognitive functions lack biologically realistic tracking and switching mechanisms. Here, we propose a new approach to this problem. We will develop a new class of data-inspired multi-purpose recurrent neural network (RNN) models that incorporate biologically plausible mechanisms to track the task being performed and the transitions between tasks. We will also analyze three distinct experimental datasets using machine learning to identify principles underlying multi-purpose functionality, particularly those that are conserved across species. Specifically, we will characterize multi-purpose functionality at the level of dynamic states. We define dynamic states as time-varying patterns of population activity that allow neural circuits to perform multiple tasks, engage them sequentially, and switch between them as task conditions or contexts change. We hypothesize that multi-purpose RNNs can incorporate dynamic states and simulate the brain's ability to track and switch between tasks, in a manner consistent with experimental data. First, we will develop and characterize data-inspired multi-purpose RNNs with internal state representations that track the engaged cognitive function/task performed. Second, we will incorporate functional and structural modularity into RNNs and analyze them in parallel with multi-region neural recordings. The resulting computational framework will enable us to identify key features of state representations and mechanisms underlying multi-purpose functionality in experimental data. What we discover will lay the foundation for understanding and testing core principles of how neural networks throughout the brain support diverse cognitive functions, enabling key advances in the study of cognition. Further, these robust, scalable multi-purpose RNN models containing internally represented states will better leverage existing large-scale neural data and galvanize new experiments designed to test model predictions. For instance, we expect to identify spatio-temporal markers from ongoing neural dynamics that predict upcoming behavioral transitions. In summary, we will build on recent advances in computer science, specifically, deep learning and other AI/ML-based techniques for neural networks, and bring them to bear on a key problem in neuroscience. Our integrative strategy maximally leverages the rapid pace of advances in computer science toward serving neuroscience and neuroengineering to catalyze new investigations beyond the confines of a lab.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.