Neuronal mechanisms of model-based learning

计算精神病学领域试图了解神经精神病学的症状和原因。 疾病是学习过程的障碍。大脑使用的学习算法遵循一个连续体 在两个极端之间。连续体的一端是无模型学习，这是一个依赖于 反复尝试，存储过去操作的值，并不灵活地重复那些导致更高 价值观。另一端是基于模型的学习，它通过计算生成预测 对环境进行建模的成本高昂、深思熟虑的过程，这赋予了应对环境的灵活性 环境变化。这些算法的功能障碍可能会产生适应不良行为。例如, 强迫行为被认为是由于基于模型的学习的破坏而产生的，这种学习使患者倾向于 更加僵化的无模型学习机制。尽管在这方面取得了很大的进展 了解无模型学习背后的神经机制，我们对如何 大脑使用模型来生成奖励预测。 这项拨款旨在测试海马体(HPC)和眶前皮质之间的相互作用这一假设 (OFC)实施基于模型的学习。具体地说，我们预测HPC负责构建一个 实例化行为任务的神经表示的认知图，OFC负责使用 产生奖励预测的认知图，可以用来做出灵活的决策。这个 目前的拨款将检验这一假说的关键预测。我们的第一个目标是使用一项新的任务，在时间上 将有关状态和值的信息的表示形式分开。我们将使用高声道计数的录音 从HPC和OFC和闭环微刺激来检查假定的HPC状态表征 影响OFC中值的编码。此外，我们还将检查这种交互是否通过 两个区域之间的theta节奏的同步。在第二个目标中，我们将研究如何更多地 涉及多个不同状态的复杂地图可能被用来实现快速重新调整以奖励变化。 HPC和额叶皮质之间的通路功能障碍与几种神经精神障碍有关， 包括精神分裂症、抑郁症和创伤后应激障碍。以药物为基础的治疗 没有显示出这些疾病的患病率或严重程度显著下降。另一种选择 方法是使用电刺激，但到目前为止，这种方法也产生了好坏参半的结果。我们的目标是发展 更复杂的设备，将以更有原则的方式与神经电路相互作用来治疗 神经精神障碍，如使用神经活动来检测症状和微刺激进行干预。 这种方法的一个障碍是，许多这样的电路中的神经编码仍然知之甚少。 目前拨款的目的是了解HPC和OFC的神经元特性，以帮助奠定 为未来基于闭环微刺激的潜在治疗方法奠定基础。

The field of computational psychiatry seeks to understand the symptoms and causes of neuropsychiatric diseases as dysfunctional learning processes. The learning algorithms used by the brain fall along a continuum between two extremes. At one end of the continuum is model-free learning, an automatic process that relies on trial-and-error, storing the values of past actions, and inflexibly repeating those actions that led to higher values. On the other end is model-based learning, which generates predictions via a computationally expensive, deliberative process that models the environment, which endows flexibility to respond to environmental changes. Dysfunction of these algorithms can produce maladaptive behaviors. For example, compulsive behavior is argued to arise from disruption of model-based learning, which biases patients towards more inflexible model-free learning mechanisms. Although a great deal of progress has been made in understanding the neural mechanisms underlying model-free learning, we have limited understanding of how the brain uses models to generate reward predictions. The grant aims to test the hypothesis that interactions between hippocampus (HPC) and orbitofrontal cortex (OFC) implement model-based learning. Specifically, we predict that HPC is responsible for constructing a cognitive map that instantiates a neural representation of behavioral tasks, and OFC is responsible for using the cognitive map to generate reward predictions that can be used to generate flexible decision-making. The current grant will test key predictions of this hypothesis. Our first aim uses a novel task that temporally separates the presentation of information about states and values. We will use high-channel count recordings from HPC and OFC and closed-loop microstimulation to examine how the putative HPC state representation affects the coding of value in OFC. In addition, we will examine whether this interaction occurs through the synchronization of the theta rhythm between the two areas. In the second aim, we will examine how a more complex map involving multiple distinct states might be used to enable rapid readjustments to reward changes. Dysfunction of pathways between HPC and frontal cortex are implicated in several neuropsychiatric disorders, including schizophrenia, major depression, and post-traumatic stress disorder. Medication-based treatments have failed to show significant reduction in the prevalence or severity of these disorders. An alternative approach is to use electrical stimulation, but to date this has also yielded mixed results. Our goal is to develop more sophisticated devices that will interact with neural circuits in a more principled way to treat neuropsychiatric disorders, such as using neural activity to detect symptoms and microstimulation to intervene. An impediment to this approach is that the neural coding in many of these circuits remains poorly understood. The aim of the current grant is to understand the neuronal properties of HPC and OFC to help lay the groundwork for future potential therapeutic approaches based on closed-loop microstimulation.