Model-based credit assignment

基于模型的学分分配

• 批准号: 10083230
• 负责人: Erie D Boorman
• 金额: $ 38.5万
• 依托单位: UNIVERSITY OF CALIFORNIA AT DAVIS
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-01-10 至 2022-12-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10083230
• 关键词: Address; Allergic Reaction; Anatomy; Animals; Anxiety; Area; Basic Science; Bayesian Modeling; Bayesian learning; Behavior; Behavioral; Biological Markers; Brain; Brain region; Child; Clams; Clinical; Code; Complex; Coupled; Data; Decision Making; Dimensions; Disease; Electroencephalography; Environment; Event; Fire - disasters; Functional Magnetic Resonance Imaging; Goals; Human; Hypersensitivity; Individual; Investigation; Knowledge; Lateral; Learning; Link; Mathematics; Measures; Mental disorders; Methodology; Methods; Modeling; Neurobiology; Neurons; Neurosciences; Outcome; Pattern; Play; Process; Psychiatry; Psychological reinforcement; Psychology; Reaction; Research; Restaurants; Role; Scallop; Scalp structure; Scientist; Service delivery model; Shellfish; Signal Transduction; Social Dominance; Social Hierarchy; Specific qualifier value; Stimulus; Structure; System; Taxonomy; Testing; Time; Update; Weight; base; experience; experimental study; flexibility; functional magnetic resonance imaging/electroencephalography; insight; neural model; neuromechanism; novel; predictive modeling; programs; relating to nervous system; shellfish hypersensitivity; sound; two-dimensional

Abstract How the brain forms, tunes, and uses predictive models that specify the causal links between stimuli in the environment, our choices, and their outcomes is a fundamental question in Psychology and Neuroscience. A great deal of progress has been made identifying the neural computations theorized to form and update predictive models. This research has played a central role in the rise of computational psychiatry, but its relevance to clinical disorders has been limited in part by the use of relatively simple learning/choice paradigms that capture only a narrow subset of the structural complexity of real-world learning. In order to make sound predictions in a complex world, the brain needs to attribute good and bad outcomes to their most likely causes, a problem known as “credit assignment”. There is little understanding of how outcomes are attributed to their most likely causes in structured real-world environments. Most real-world learning occurs in complex and structured environments, such as hierarchical systems (e.g. seasonal events, social hierarchies, contextual rules, etc.). Recent evidence suggests that humans can use an understanding of the environment’s causal structure to attribute outcomes to their most likely causes (which I call “model-based credit assignment)”, rather than simply attributing them to the most recently experienced stimuli and choices that were made (which I call “model-free” credit assignment), as standard models have proposed. The purpose of the present proposal is to develop the first neural model of model-based credit assignment. We hypothesize that the brain reinstates the cause when a reinforcement outcome is experienced to associate with the outcome. In other words, so that “fire-together/wire-together” plasticity mechanisms can link a choice with an outcome, the choice representation and the outcome representation must both be active at the same time even though the causal choice or event may have actually occurred some time beforehand. To test this and other predictions, we will integrate mathematical descriptions of learning and decision making with “representational” analysis methods that allow inferences to be made about the information represented in brain areas, applied to fMRI and scalp EEG data. fMRI will reveal how neural learning signals update neural representations of likely causes during learning, while EEG will reveal the timing of the hypothesized reinstatement. These experiments will set the stage to apply the insights gained to investigate how the brain attributes outcomes to more abstract “latent” causes in hierarchically structured environments prevalent in the real world. The proposed project will thus move this general program of research strategy toward learning tasks that better reflect the complexity and structure in many real-world learning/choice situations important for both typical and atypical individuals.

摘要 大脑是如何形成、调整和使用预测模型的，这些模型指定了大脑中刺激之间的因果联系 环境、我们的选择以及它们的结果是心理学和神经科学的一个基本问题。一个 识别神经计算的理论形成和更新已经取得了很大的进展 预测性模型。这项研究在计算精神病学的兴起中发挥了核心作用，但它的 与临床疾病的相关性在一定程度上受到相对简单的学习/选择范例的使用的限制 这只捕捉到了现实世界学习结构复杂性的一小部分。 为了在复杂的世界中做出可靠的预测，大脑需要将好的和坏的属性归因于 结果是最有可能的原因，这就是所谓的“信用分配”问题。人们对……了解甚少 在结构化的真实世界环境中，结果如何归因于其最可能的原因。最真实的世界 学习发生在复杂和结构化的环境中，例如分层系统(例如，季节性事件， 社会等级、上下文规则等)。最近的证据表明，人类可以利用对 环境的因果结构，将结果归因于其最可能的原因(我称之为基于模型 信用分配)“，而不是简单地将它们归因于最近经历的刺激和选择 (我称之为“无模型”信用分配)，正如标准模型所建议的那样。 本建议的目的是开发基于模型的信用分配的第一个神经模型。 我们假设，当经历强化结果时，大脑会恢复原因 结果如何。换句话说，这样“一起火/一根线”的可塑性机制可以将一种选择联系起来 对于结果，选择表示和结果表示必须同时处于活动状态 时间，即使因果选择或事件实际上可能发生在一段时间之前。 为了测试这一预测和其他预测，我们将整合学习和决策的数学描述 使用“代表性”分析方法，允许对信息进行推论 代表大脑区域，应用于功能磁共振成像和头皮脑电数据。功能磁共振成像将揭示神经学习信号如何 在学习过程中更新可能原因的神经表征，而脑电波将揭示 假想复职。这些实验将为应用所获得的见解来研究 大脑将结果归因于在普遍存在的分层结构环境中更抽象的“潜伏”原因 在真实的世界见面。因此，拟议的项目将把这一研究策略的一般方案推向学习 更好地反映许多现实世界学习/选择情景中的复杂性和结构的任务 既有典型的，也有非典型的。