Determining the neural substrates of model-based and model-free reinforcement-learning during Pavlovian conditioning

确定巴甫洛夫条件反射期间基于模型和无模型强化学习的神经基础

• 批准号: 9106549
• 负责人: JOHN P O'DOHERTY
• 金额: $ 52.54万
• 依托单位: CALIFORNIA INSTITUTE OF TECHNOLOGY
• 依托单位国家: 美国
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-07-01 至 2021-03-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9106549
• 关键词: Address; Amygdaloid structure; Animals; Area; Behavior; Brain; Brain region; Cell Nucleus; Choice Behavior; Classification; Code; Complement; Computer Analysis; Conditioned Reflex; Conditioned Stimulus; Cues; Data; Dorsal; Environment; Evaluation; Functional Magnetic Resonance Imaging; Goals; Homologous Gene; Human; Incentives; Individual; Knowledge; Learning; Link; Literature; Maintenance; Methodology; Midbrain structure; Modeling; Multivariate Analysis; Nature; Neurobiology; Neurons; Neurosciences; Operant Conditioning; Outcome; Patients; Pattern; Pharmaceutical Preparations; Play; Process; Psychological reinforcement; Psychology; Recording of previous events; Research; Resolution; Rewards; Role; Signal Transduction; Stimulus; Structure; Techniques; Testing; Ventral Striatum; Ventral Tegmental Area; addiction; base; classical conditioning; insight; interest; neurophysiology; neurosurgery; novel; operation; psychologic; public health relevance; reinforcer; relating to nervous system; response

 DESCRIPTION (provided by applicant): The study of the psychological, computational and neurobiological basis of Pavlovian conditioning is one of the longest standing research questions in psychology and neuroscience. In spite of the ubiquity and the importance of this form of learning, the computational mechanisms underlying the learning and expression of Pavlovian associations' remains poorly understood. Here, we investigate whether or not there exists two distinct forms of Pavlovian conditioning, a model-based form in which the expression of conditioned responses to a conditioned stimulus is sensitive to the incentive value of the associated unconditioned stimulus (US), and another "model-free" form in which conditioned responses elicited by a conditioned stimulus are insensitive to the current US value. The distinction between model-based and model-free reinforcement-learning mechanisms has received strong empirical support in the domain of instrumental conditioning, but little is known about whether or not a similar dichotomy exists in Pavlovian conditioning. Understanding the nature of the encoding of Pavlovian associations in the brain is important because of the critical role that learned Pavlovian associations might play in the maintenance of addiction, in which cues linked to drug outcomes might promote or invigorate responding for drugs, even if those drugs are no longer deemed valuable/desirable to the individual. In the present application we address this goal by performing both functional magnetic resonance imaging (fMRI) and single-unit recordings in humans while they undergo sequential Pavlovian conditioning with appetitive outcomes. We will use a number of different cutting-edge experimental and analytical techniques, including computational based analyses, multivariate pattern classification and high-resolution fMRI. We will test for the existence of these different representations in a number of distinct structures in the brain including the amygdala, orbitofrontal cortex, ventral striatum and dopaminergic midbrain. Because we will be using high- resolution fMRI, we will have the capacity to resolve the contribution of distinct sub-regions within these brain structures to model-based and model-free Pavlovian learning, including the basolateral versus centromedial amygdala, the human homologue of the core versus shell of the accumbens, different sectors of orbitofrontal cortex, and dorsal versus ventral parts of the substantial nigra and ventral tegmental area. To complement the fMRI studies, we will record from neurons primarily in the amygdala and orbitofrontal cortex in human neurosurgery patients while they while they perform one of the main tasks used in the fMRI studies, thereby enabling us to gain insight into the relationship between the observed fMRI signals and underlying neuronal activity in at least two of our key regions of interest. By combining across these different techniques and methodologies, we will be able to address the question of whether or not model-based and model-free forms of Pavlovian conditioning are implemented in parallel in the brain, and begin to gain insight into the specific contributions of different brain regions towards these two very distinct forms of learning.