Implicit serial learning

项目摘要 强化学习（RL）是一个强大的框架，用于理解结果驱动行为及其神经网络。 基础强化学习的一个关键问题是如何整合一种推理能力，以允许适当的响应 在训练中没有遇到的新情况。推理依赖于对潜在关系的检测， 预测奖励，但没有明确的暗示。在上一个融资期，我们开发了行为 和计算方法来严格解决这个问题，并使单细胞神经生理学 这些观察开始揭示大脑如何表现潜在结构。在这里，我们重新定义了 问题作为一个使用认知地图来表示潜在秩序和支持推理。我们建议测试 这一想法与新开发的行为方法，这使得前所未有的洞察力的作用， 皮质-纹状体神经回路和基于模型的RL中的上行神经调节系统。这些 研究将集中在神经生理学、神经影像学和病变研究表明具有 在内隐序列学习中的重要作用，特别是背外侧（dlPFC）和腹内侧前额叶皮层 （vmPFC）和背侧纹状体。三个目标1。测试NHP在以下环境中进行基于模型的推理的能力 存在补偿性奖励激励，2。识别dlPFC，vmPFC， 和尾状核;测试乙酰胆碱和多巴胺浓度的变化是否与连续学习相关， 特别是在转移过程中。推理能力在许多精神疾病中受损，包括 精神分裂症和躁郁症。这种损害可能是思维障碍的基础，如妄想和 妄想症

Project Summary Reinforcement learning (RL) is a powerful framework for understanding outcome-driven behavior and its neural basis. A key issue for RL is how to incorporate an ability to make inferences that allow appropriate responding to novel conditions not encountered during training. Inference relies on the detection of latent relationships that predict rewards but are not signaled by explicit cues. In the previous funding period, we developed behavioral and computational approaches to rigorously address this problem, and made single-cell neurophysiological observations that began to reveal how the brain represents latent structures. Here, we re-conceptualize this problem as one of using a cognitive map to represent latent order and support inference. We propose to test this idea with a newly developed behavioral approach, which allows unprecedented insight into the role of cortico-striatal neural circuits and ascending neuromodulatory systems in model-based RL. These investigations will focus on brain regions that neurophysiology, neuroimaging, and lesion studies suggest have important roles in implicit serial learning, specifically dorsolateral (dlPFC) and ventromedial prefrontal cortex (vmPFC), and dorsal striatum. Three aims will 1. Test NHPs ability to make model-based inferences in the presence of countervailing reward incentives, 2. Identify neural circuitry of serial learning in the dlPFC, vmPFC, and caudate nucleus, and 3. Test if changes in Ach and DA concentrations are correlated with serial learning, particularly during transfer. Inferential reasoning is impaired in many psychiatric illnesses including schizophrenia and bipolar disorder. This impairment may underlie thought disorders such as delusions and paranoia.