权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Determining the neural substrates of model-based and model-free reinforcement-learning during Pavlovian conditioning

确定巴甫洛夫条件反射期间基于模型和无模型强化学习的神经基础

基本信息

批准号：
10117323
负责人：
JOHN P O'DOHERTY
金额：
$ 4.31万
依托单位：
CALIFORNIA INSTITUTE OF TECHNOLOGY
依托单位国家：
美国
项目类别：
财政年份：
2016
资助国家：
美国
起止时间：
2016-07-01 至 2022-03-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/10117323
关键词：
Address Amygdaloid structure Animals Area Behavior Brain Brain region Cell Nucleus Choice Behavior Classification Code Complement Computer Analysis Conditioned Reflex Conditioned Stimulus Cues Data Desire for food Dorsal Environment Evaluation Functional Magnetic Resonance Imaging Goals Homologous Gene Human Incentives Individual Knowledge Learning Link Literature Maintenance Methodology Midbrain structure Modeling Multivariate Analysis Nature Neurobiology Neurons Neurosciences Operant Conditioning Outcome Patients Pattern Pharmaceutical Preparations Play Process Psychological reinforcement Psychology Recording of previous events Research Resolution Rewards Role Signal Transduction Stimulus Structure Techniques Testing Ventral Striatum Ventral Tegmental Area addiction base classical conditioning flexibility imaging study insight interest neurophysiology neurosurgery novel operation psychologic public health relevance reinforcer relating to nervous system response

项目摘要

DESCRIPTION (provided by applicant): The study of the psychological, computational and neurobiological basis of Pavlovian conditioning is one of the longest standing research questions in psychology and neuroscience. In spite of the ubiquity and the importance of this form of learning, the computational mechanisms underlying the learning and expression of Pavlovian associations' remains poorly understood. Here, we investigate whether or not there exists two distinct forms of Pavlovian conditioning, a model-based form in which the expression of conditioned responses to a conditioned stimulus is sensitive to the incentive value of the associated unconditioned stimulus (US), and another "model-free" form in which conditioned responses elicited by a conditioned stimulus are insensitive to the current US value. The distinction between model-based and model-free reinforcement-learning mechanisms has received strong empirical support in the domain of instrumental conditioning, but little is known about whether or not a similar dichotomy exists in Pavlovian conditioning. Understanding the nature of the encoding of Pavlovian associations in the brain is important because of the critical role that learned Pavlovian associations might play in the maintenance of addiction, in which cues linked to drug outcomes might promote or invigorate responding for drugs, even if those drugs are no longer deemed valuable/desirable to the individual. In the present application we address this goal by performing both functional magnetic resonance imaging (fMRI) and single-unit recordings in humans while they undergo sequential Pavlovian conditioning with appetitive outcomes. We will use a number of different cutting-edge experimental and analytical techniques, including computational based analyses, multivariate pattern classification and high-resolution fMRI. We will test for the existence of these different representations in a number of distinct structures in the brain including the amygdala, orbitofrontal cortex, ventral striatum and dopaminergic midbrain. Because we will be using high- resolution fMRI, we will have the capacity to resolve the contribution of distinct sub-regions within these brain structures to model-based and model-free Pavlovian learning, including the basolateral versus centromedial amygdala, the human homologue of the core versus shell of the accumbens, different sectors of orbitofrontal cortex, and dorsal versus ventral parts of the substantial nigra and ventral tegmental area. To complement the fMRI studies, we will record from neurons primarily in the amygdala and orbitofrontal cortex in human neurosurgery patients while they while they perform one of the main tasks used in the fMRI studies, thereby enabling us to gain insight into the relationship between the observed fMRI signals and underlying neuronal activity in at least two of our key regions of interest. By combining across these different techniques and methodologies, we will be able to address the question of whether or not model-based and model-free forms of Pavlovian conditioning are implemented in parallel in the brain, and begin to gain insight into the specific contributions of different brain regions towards these two very distinct forms of learning.

描述(由申请人提供)：巴甫洛夫条件反射的心理、计算和神经生物学基础的研究是心理学和神经科学中历史最悠久的研究问题之一。尽管这种学习形式无处不在，也很重要，但人们对巴甫洛夫联想的学习和表达背后的计算机制仍然知之甚少。在这里，我们研究是否存在两种不同形式的巴甫洛夫条件反射，一种是基于模型的形式，其中对条件刺激的条件反应的表达对相关的无条件刺激(US)的激励值敏感，另一种是无模型的形式，其中由条件刺激引起的条件反应对当前的美国值不敏感。基于模型的强化学习机制和无模型强化学习机制的区别在工具条件作用领域得到了强有力的经验支持，但对于巴甫洛夫条件作用中是否存在类似的二分法知之甚少。了解大脑中巴甫洛夫联想编码的性质很重要，因为习得的巴甫洛夫联想可能在维持成瘾方面发挥关键作用，在这种情况下，与药物结果有关的线索可能会促进或激发对药物的反应，即使这些药物不再被认为对个人有价值/可取。在本申请中，我们通过在人类中进行功能性磁共振成像(FMRI)和单单位记录来解决这一目标，同时在他们接受具有食欲的顺序巴甫洛夫条件反射时进行记录。我们将使用一些不同的尖端实验和分析技术，包括基于计算的分析、多变量模式分类和高分辨率功能磁共振成像。我们将测试这些不同的表达在大脑中一些不同的结构中的存在，包括杏仁核、眶前皮质、腹侧纹状体和多巴胺能中脑。因为我们将使用高分辨率的功能磁共振成像，我们将有能力解析这些大脑结构中不同的子区域对基于模型的和非模型的巴甫洛夫学习的贡献，包括基底外侧杏仁核和中央内侧杏仁核、人类伏隔核与核壳的同源物、眶前皮质的不同部分，以及实质黑质和腹侧被盖区的背侧和腹侧部分。为了补充fMRI研究，我们将记录人类神经外科患者在执行fMRI研究中使用的主要任务之一时，他们主要位于杏仁核和眶前皮质的神经元，从而使我们能够深入了解观察到的fMRI信号与至少两个关键感兴趣区域的潜在神经元活动之间的关系。通过结合这些不同的技术和方法，我们将能够解决基于模型的和无模型的巴甫洛夫条件反射是否在大脑中并行实施的问题，并开始深入了解不同大脑区域对这两种非常不同的学习形式的具体贡献。