Neuronal mechanisms of model-based learning

基于模型的学习的神经机制

• 批准号: 10722261
• 负责人: Joni D Wallis
• 金额: $ 57.79万
• 依托单位: UNIVERSITY OF CALIFORNIA BERKELEY
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-08 至 2028-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10722261
• 关键词: Affect; Algorithms; Anxiety; Area; Behavioral; Brain; Cells; Code; Cognitive; Communication; Complex; Compulsive Behavior; Computer Models; Decision Making; Devices; Disease; Electric Stimulation; Endowment; Environment; Functional disorder; Future; Goals; Grant; Graph; Hippocampus; Impairment; Learning; Location; Major Depressive Disorder; Maps; Mental Depression; Modeling; Monkeys; Neurons; Pathway interactions; Patients; Pharmaceutical Preparations; Play; Post-Traumatic Stress Disorders; Predictive Value; Prevalence; Probability; Process; Property; Psychiatry; Research; Rewards; Role; Schizophrenia; Severities; Specific qualifier value; Symptoms; Testing; Therapeutic; Therapeutic Intervention; Theta Rhythm; Update; awake; environmental change; falls; flexibility; frontal lobe; knowledge graph; learning algorithm; maladaptive behavior; microstimulation; neural; neural circuit; neuromechanism; neurophysiology; neuropsychiatric disorder; novel; response; spatiotemporal

The field of computational psychiatry seeks to understand the symptoms and causes of neuropsychiatric diseases as dysfunctional learning processes. The learning algorithms used by the brain fall along a continuum between two extremes. At one end of the continuum is model-free learning, an automatic process that relies on trial-and-error, storing the values of past actions, and inflexibly repeating those actions that led to higher values. On the other end is model-based learning, which generates predictions via a computationally expensive, deliberative process that models the environment, which endows flexibility to respond to environmental changes. Dysfunction of these algorithms can produce maladaptive behaviors. For example, compulsive behavior is argued to arise from disruption of model-based learning, which biases patients towards more inflexible model-free learning mechanisms. Although a great deal of progress has been made in understanding the neural mechanisms underlying model-free learning, we have limited understanding of how the brain uses models to generate reward predictions. The grant aims to test the hypothesis that interactions between hippocampus (HPC) and orbitofrontal cortex (OFC) implement model-based learning. Specifically, we predict that HPC is responsible for constructing a cognitive map that instantiates a neural representation of behavioral tasks, and OFC is responsible for using the cognitive map to generate reward predictions that can be used to generate flexible decision-making. The current grant will test key predictions of this hypothesis. Our first aim uses a novel task that temporally separates the presentation of information about states and values. We will use high-channel count recordings from HPC and OFC and closed-loop microstimulation to examine how the putative HPC state representation affects the coding of value in OFC. In addition, we will examine whether this interaction occurs through the synchronization of the theta rhythm between the two areas. In the second aim, we will examine how a more complex map involving multiple distinct states might be used to enable rapid readjustments to reward changes. Dysfunction of pathways between HPC and frontal cortex are implicated in several neuropsychiatric disorders, including schizophrenia, major depression, and post-traumatic stress disorder. Medication-based treatments have failed to show significant reduction in the prevalence or severity of these disorders. An alternative approach is to use electrical stimulation, but to date this has also yielded mixed results. Our goal is to develop more sophisticated devices that will interact with neural circuits in a more principled way to treat neuropsychiatric disorders, such as using neural activity to detect symptoms and microstimulation to intervene. An impediment to this approach is that the neural coding in many of these circuits remains poorly understood. The aim of the current grant is to understand the neuronal properties of HPC and OFC to help lay the groundwork for future potential therapeutic approaches based on closed-loop microstimulation.

计算精神病学领域寻求理解神经精神病的症状和原因 疾病是功能失调的学习过程。大脑使用的学习算法沿着一个连续体 两个极端之间。在连续体的一端是无模型学习，这是一个依赖于 反复试验，存储过去行为的值，并重复那些导致更高收益的行为。 价值观另一端是基于模型的学习，它通过计算生成预测。 昂贵的，深思熟虑的过程，模拟环境，这赋予灵活性，以回应 环境变化。这些算法的功能障碍会产生适应不良的行为。比如说， 强迫行为被认为是由于基于模型的学习中断引起的，这使患者倾向于 更不灵活的无模型学习机制。虽然在这方面取得了很大的进展， 了解无模型学习背后的神经机制，我们对如何理解有限 大脑使用模型来产生奖励预测。 该基金旨在验证海马体（HPC）和眶额皮质之间的相互作用 (OFC)实施基于模型的学习。具体来说，我们预测HPC负责构建一个 认知地图，实例化行为任务的神经表示，OFC负责使用 认知地图生成奖励预测，可用于生成灵活的决策。的 目前的赠款将测试这一假设的关键预测。我们的第一个目标使用了一个新的任务， 将有关状态和值的信息的表示分开。我们将使用高通道计数记录 从HPC和OFC和闭环微刺激来检查假定的HPC状态表示如何 影响OFC中的值编码。此外，我们将研究这种相互作用是否通过 两个区域之间θ节律的同步。在第二个目标中，我们将研究如何更好地 可以使用涉及多个不同状态的复杂映射来实现对奖励变化的快速重新调整。 HPC和额叶皮质之间的通路功能障碍与几种神经精神疾病有关， 包括精神分裂症、重度抑郁症和创伤后应激障碍。药物治疗 未能显示这些疾病的患病率或严重程度的显著降低。一个替代 一种方法是使用电刺激，但迄今为止，这也产生了混合的结果。我们的目标是开发 更复杂的设备将以更有原则的方式与神经回路相互作用， 神经精神疾病，如使用神经活动来检测症状和微刺激进行干预。 这种方法的一个障碍是，这些回路中的许多神经编码仍然知之甚少。 目前拨款的目的是了解HPC和OFC的神经元特性，以帮助奠定基础。 为基于闭环微刺激的未来潜在治疗方法奠定基础。