Neuronal mechanisms of model-based learning

点击翻译按钮获取中文摘要

The field of computational psychiatry seeks to understand the symptoms and causes of neuropsychiatric diseases as dysfunctional learning processes. The learning algorithms used by the brain fall along a continuum between two extremes. At one end of the continuum is model-free learning, an automatic process that relies on trial-and-error, storing the values of past actions, and inflexibly repeating those actions that led to higher values. On the other end is model-based learning, which generates predictions via a computationally expensive, deliberative process that models the environment, which endows flexibility to respond to environmental changes. Dysfunction of these algorithms can produce maladaptive behaviors. For example, compulsive behavior is argued to arise from disruption of model-based learning, which biases patients towards more inflexible model-free learning mechanisms. Although a great deal of progress has been made in understanding the neural mechanisms underlying model-free learning, we have limited understanding of how the brain uses models to generate reward predictions. The grant aims to test the hypothesis that interactions between hippocampus (HPC) and orbitofrontal cortex (OFC) implement model-based learning. Specifically, we predict that HPC is responsible for constructing a cognitive map that instantiates a neural representation of behavioral tasks, and OFC is responsible for using the cognitive map to generate reward predictions that can be used to generate flexible decision-making. The current grant will test key predictions of this hypothesis. Our first aim uses a novel task that temporally separates the presentation of information about states and values. We will use high-channel count recordings from HPC and OFC and closed-loop microstimulation to examine how the putative HPC state representation affects the coding of value in OFC. In addition, we will examine whether this interaction occurs through the synchronization of the theta rhythm between the two areas. In the second aim, we will examine how a more complex map involving multiple distinct states might be used to enable rapid readjustments to reward changes. Dysfunction of pathways between HPC and frontal cortex are implicated in several neuropsychiatric disorders, including schizophrenia, major depression, and post-traumatic stress disorder. Medication-based treatments have failed to show significant reduction in the prevalence or severity of these disorders. An alternative approach is to use electrical stimulation, but to date this has also yielded mixed results. Our goal is to develop more sophisticated devices that will interact with neural circuits in a more principled way to treat neuropsychiatric disorders, such as using neural activity to detect symptoms and microstimulation to intervene. An impediment to this approach is that the neural coding in many of these circuits remains poorly understood. The aim of the current grant is to understand the neuronal properties of HPC and OFC to help lay the groundwork for future potential therapeutic approaches based on closed-loop microstimulation.