Acquiring cognitive maps: how brains learn hidden structure

项目摘要：动物在复杂的环境中执行目标导向的行为，不需要 丰富的经验，通过利用世界的内部模型。这些内部模型，或认知图， 是基于模型的决策的核心。重要的是，理解基于模型的神经电路 决策为改善神经精神障碍的治疗带来了希望，在这种疾病中，决策发生了 歪歪扭扭。强化学习(RL)中一个突出的问题是认知地图是如何从经验中学习的， 以及支持它们的神经底物。眼眶前额叶皮质(OFC)被认为是 认知地图，这个项目建议表征啮齿动物OFC在 一项基于价值的决策任务。该项目提出了三个核心问题：1)基于模型的RL如何 代理使用纯粹的无模型RL方法学习认知地图？在这里，Meta-RL的最新进展处于- 最先进的递归神经网络(RNN)模型将用无模型的RL进行训练，其中出现了一个 认知图谱可以得到充分的刻画。2)OFC如何代表认知图谱？在这里接受过部分训练 老鼠将被植入神经像素探测器，神经记录将捕捉到新出现的表征 在大鼠学习任务结构的过程中，认知地图在群体水平的OFC活动中的作用。3)学习能力差会不会 认知地图可以通过有组织的行为训练来支持？一种有吸引力的治疗方法，以改善 决策策略仅通过行为训练，这里将使用啮齿动物的RNN模型 来表征学习不良的模式并发展指令性训练，然后将在大鼠身上应用 独一无二地解决特定的学习缺陷。 这个拟议的项目同时使用实验和计算技术，作为综合职业生涯的一部分。 发展计划，成为一名独立的调查员。电物理的特殊实验训练-- 来自行为动物的生物学记录补充了对神经活动建模的计算培训 深度神经网络。职业发展计划还包括与 实验者。它通过研究会议整合了广泛的科学交流经验 并在当地培训机构内进行公开演讲。重要的是，职业发展计划纳入了有针对性的 准备独立调查员应用程序，包括粉笔对话机会、要开发的研讨会 独特的研究问题，以及对教师寻找过程的了解。君士坦丁堡、萨文和格利姆- 纽约大学神经科学中心的Cher实验室是一个独特的培训环境 提供这种跨学科的培训：他们对决策、动物行为的尖端研究，加强- MET学习和系统神经科学使其成为开发与健康相关的 研究神经科学决策策略的职业。

Project Summary: Animals perform goal-directed behaviors in complex environments, without the need for extensive experience, by harnessing an internal model of the world. These internal models, or cognitive maps, are central to model-based decision making. Importantly, understanding the neural circuitry of model-based decisions holds promise for improving treatments of neuropsychiatric disorders in which decision-making goes awry. An outstanding question in reinforcement learning (RL) is how cognitive maps are learned from experience, and what neural substrates support them. The orbitofrontal cortex (OFC) has been implicated in representing cognitive maps, and this project proposes to characterize the emergence of a cognitive map in rodent OFC during a value-based decision making task. This project asks three core questions: 1) How does a model-based RL agent learn a cognitive map using purely model-free RL methods? Here recent advances in meta-RL in state-of- the-art recurrent neural network (RNN) models will be trained with model-free RL, in which the emergence of a cognitive map can be fully characterized. 2) How does the OFC represent cognitive maps? Here partially trained rats will be implanted with Neuropixels probes, and neural recordings will capture emerging representations of cognitive maps in population-level OFC activity as rats learn the task structure. 3) Can poor learning of cognitive maps be bolstered with structured behavioral training? An attractive therapeutic approach for improving decision making strategies is through behavioral training alone, and here an RNN model of rodents will be used to characterize modes of poor learning and develop prescriptive training, and will then be employed in rats to uniquely addresses specific learning deficits. This proposed project employs both experimental and computational techniques as part a comprehensive career development plan toward becoming an independent investigator. Specific experimental training for electrophysi- ological recordings from behaving animals complements computational training on modeling neural activity with deep neural networks. The career development plan also includes structured opportunities for collaborating with experimentalists. It incorporates a breadth of science communication experiences through research conferences and public talks within the local training institution. Importantly, the career development plan incorporates targeted preparation for independent investigator applications, including chalk-talk opportunities, workshops to develop unique research questions, and exposure to the faculty search process. The Constantinople, Savin, and Glim- cher labs at New York University’s Center for Neural Science are a training environment uniquely positioned to deliver this interdisciplinary training: their cutting-edge research into decision-making, animal behavior, reinforce- ment learning, and systems neuroscience make it an ideal institution and set of labs to develop a health-related career studying decision-making strategies in neuroscience.