Model-based credit assignment

点击翻译按钮获取中文摘要

Abstract How the brain forms, tunes, and uses predictive models that specify the causal links between stimuli in the environment, our choices, and their outcomes is a fundamental question in Psychology and Neuroscience. A great deal of progress has been made identifying the neural computations theorized to form and update predictive models. This research has played a central role in the rise of computational psychiatry, but its relevance to clinical disorders has been limited in part by the use of relatively simple learning/choice paradigms that capture only a narrow subset of the structural complexity of real-world learning. In order to make sound predictions in a complex world, the brain needs to attribute good and bad outcomes to their most likely causes, a problem known as “credit assignment”. There is little understanding of how outcomes are attributed to their most likely causes in structured real-world environments. Most real-world learning occurs in complex and structured environments, such as hierarchical systems (e.g. seasonal events, social hierarchies, contextual rules, etc.). Recent evidence suggests that humans can use an understanding of the environment’s causal structure to attribute outcomes to their most likely causes (which I call “model-based credit assignment)”, rather than simply attributing them to the most recently experienced stimuli and choices that were made (which I call “model-free” credit assignment), as standard models have proposed. The purpose of the present proposal is to develop the first neural model of model-based credit assignment. We hypothesize that the brain reinstates the cause when a reinforcement outcome is experienced to associate with the outcome. In other words, so that “fire-together/wire-together” plasticity mechanisms can link a choice with an outcome, the choice representation and the outcome representation must both be active at the same time even though the causal choice or event may have actually occurred some time beforehand. To test this and other predictions, we will integrate mathematical descriptions of learning and decision making with “representational” analysis methods that allow inferences to be made about the information represented in brain areas, applied to fMRI and scalp EEG data. fMRI will reveal how neural learning signals update neural representations of likely causes during learning, while EEG will reveal the timing of the hypothesized reinstatement. These experiments will set the stage to apply the insights gained to investigate how the brain attributes outcomes to more abstract “latent” causes in hierarchically structured environments prevalent in the real world. The proposed project will thus move this general program of research strategy toward learning tasks that better reflect the complexity and structure in many real-world learning/choice situations important for both typical and atypical individuals.