Determining the neural substrates of model-based and model-free reinforcement-learning during Pavlovian conditioning

 描述（由申请人提供）：巴甫洛夫条件反射的心理学，计算和神经生物学基础的研究是心理学和神经科学中最古老的研究问题之一。尽管这种学习形式的普遍性和重要性，但巴甫洛夫联想学习和表达的计算机制仍然知之甚少。在这里，我们调查是否存在两种不同形式的巴甫洛夫条件反射，一个基于模型的形式，其中条件刺激的条件反应的表达是敏感的相关的非条件刺激（US）的激励值，和另一个“无模型”的形式，其中条件刺激引起的条件反应是不敏感的当前US值。基于模型和无模型的学习机制之间的区别在工具性条件反射领域得到了强有力的经验支持，但很少有人知道巴甫洛夫条件反射中是否存在类似的二分法。了解大脑中巴甫洛夫关联编码的性质很重要，因为学习的巴甫洛夫关联可能在维持成瘾中发挥关键作用，其中与药物结果相关的线索可能会促进或激励对药物的反应，即使这些药物不再被认为对个体有价值/可取。在本申请中，我们通过在人类中进行功能性磁共振成像（fMRI）和单单位记录来解决这个目标，同时他们经历具有食欲结果的顺序巴甫洛夫条件反射。我们将使用一些不同的尖端实验和分析技术，包括基于计算的分析，多变量模式分类和高分辨率功能磁共振成像。我们将测试这些不同的表征是否存在于大脑中的一些不同结构中，包括杏仁核、眶额皮质、腹侧纹状体和 多巴胺能中脑因为我们将使用高分辨率的功能磁共振成像，我们将有能力解决这些大脑结构中不同的子区域对基于模型和无模型的巴甫洛夫学习的贡献，包括基底外侧杏仁核与中央内侧杏仁核，人类杏仁核核的核心与外壳的同源物，眶额皮质的不同部分，以及黑质和腹侧被盖区的背侧与腹侧部分。为了补充功能磁共振成像研究，我们将记录主要在人类神经外科患者的杏仁核和眶额皮质的神经元，而他们执行功能磁共振成像研究中使用的主要任务之一，从而使我们能够深入了解所观察到的功能磁共振成像信号和我们感兴趣的至少两个关键区域的潜在神经元活动之间的关系。通过结合这些不同的技术和方法，我们将能够解决基于模型和无模型形式的巴甫洛夫条件反射是否在大脑中并行实施的问题，并开始深入了解不同大脑区域对这两种非常不同的学习形式的具体贡献。

 DESCRIPTION (provided by applicant): The study of the psychological, computational and neurobiological basis of Pavlovian conditioning is one of the longest standing research questions in psychology and neuroscience. In spite of the ubiquity and the importance of this form of learning, the computational mechanisms underlying the learning and expression of Pavlovian associations' remains poorly understood. Here, we investigate whether or not there exists two distinct forms of Pavlovian conditioning, a model-based form in which the expression of conditioned responses to a conditioned stimulus is sensitive to the incentive value of the associated unconditioned stimulus (US), and another "model-free" form in which conditioned responses elicited by a conditioned stimulus are insensitive to the current US value. The distinction between model-based and model-free reinforcement-learning mechanisms has received strong empirical support in the domain of instrumental conditioning, but little is known about whether or not a similar dichotomy exists in Pavlovian conditioning. Understanding the nature of the encoding of Pavlovian associations in the brain is important because of the critical role that learned Pavlovian associations might play in the maintenance of addiction, in which cues linked to drug outcomes might promote or invigorate responding for drugs, even if those drugs are no longer deemed valuable/desirable to the individual. In the present application we address this goal by performing both functional magnetic resonance imaging (fMRI) and single-unit recordings in humans while they undergo sequential Pavlovian conditioning with appetitive outcomes. We will use a number of different cutting-edge experimental and analytical techniques, including computational based analyses, multivariate pattern classification and high-resolution fMRI. We will test for the existence of these different representations in a number of distinct structures in the brain including the amygdala, orbitofrontal cortex, ventral striatum and dopaminergic midbrain. Because we will be using high- resolution fMRI, we will have the capacity to resolve the contribution of distinct sub-regions within these brain structures to model-based and model-free Pavlovian learning, including the basolateral versus centromedial amygdala, the human homologue of the core versus shell of the accumbens, different sectors of orbitofrontal cortex, and dorsal versus ventral parts of the substantial nigra and ventral tegmental area. To complement the fMRI studies, we will record from neurons primarily in the amygdala and orbitofrontal cortex in human neurosurgery patients while they while they perform one of the main tasks used in the fMRI studies, thereby enabling us to gain insight into the relationship between the observed fMRI signals and underlying neuronal activity in at least two of our key regions of interest. By combining across these different techniques and methodologies, we will be able to address the question of whether or not model-based and model-free forms of Pavlovian conditioning are implemented in parallel in the brain, and begin to gain insight into the specific contributions of different brain regions towards these two very distinct forms of learning.