Generalization of Stimulus Value in Reinforcement Learning

强化学习中刺激值的推广

• 批准号: RGPIN-2014-05069
• 负责人: Britt, Jonathan
• 金额: $ 2.26万
• 依托单位: McGill University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2016
• 资助国家: 加拿大
• 起止时间: 2016-01-01 至 2017-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=611421
• 关键词: Generalization; Stimulus; Value; Reinforcement; Learning

The goal of my research program is to determine how reward learning and motivational processes are encoded in neural activity in the striatum. To accomplish this goal, I am studying different facets of information processing in this brain region, starting with how specific glutamate inputs impact the extent to which animals generalize from past experience. Stimulus generalization is a process where predictive value is ascribed to novel stimuli that are perceptually similar to previously learned stimuli. This results in animals’ actively approaching or avoiding stimuli that they have never previously encountered. The neurobiological mechanisms underlying this phenomenon are of great interest, as there is a wide variability in the extent that humans generalize, and certain neuropsychiatric conditions, such as schizophrenia, generalized anxiety disorder, and post-traumatic stress disorder, are associated with marked abnormalities in this process. The research proposal described here utilizes modern neuroscience techniques to manipulate neural activity at precise moments during reward seeking with the goal of gaining mechanistic insight into the neurobiology of stimulus generalization. The research focuses on information processing in a part of the ventral striatum called the nucleus accumbens. This area integrates midbrain dopaminergic signals with excitatory input from cortical structures. The principal excitatory afferents to the nucleus accumbens come from the ventral hippocampus, basolateral amygdala, and prefrontal cortex, all regions which are thought to regulate distinct aspects of memory. Recent data from human fMRI research have implicated these glutamate inputs to the nucleus accumbens in regulating the extent to which people generalize during simple reward-seeking tasks. These findings are consistent with data from gain of function experiments in rodents, which demonstrate that enhanced activity in each of these pathways is sufficient to reinforce and motivate instrumental behavior. The technical approach I am proposing involves inhibiting discrete glutamate inputs to the nucleus accumbens at specific moments during a discriminative cue learning task. Optogenetic techniques will be used to accomplish this, which utilize genetically-encoded, light-sensitive proteins to inhibit neural activity. The spatial and temporal precision afforded by these techniques allow for pathway-specific inhibition of transmitter release at precise moments during learning and choice behaviour. I have extensive experience with this approach, with integrating optogenetic manipulations with clever behavioural assays. My publication record underscores how I am a well-qualified, highly trained, and committed researcher with a passion for studying the neural circuits underlying motivated behaviour. The overarching goal of the research proposed here is to understand how activity in different glutamate inputs to the nucleus accumbens impacts reward-related learning and choice behaviour. This work will be of wide interest as it touches upon the foundations of motivation, decision making, and reinforcement learning. In addition, the multifaceted nature of this research will provide trainees with extensive technical skills as well as experience collecting, analyzing, and interpreting data. Ultimately, my research program will further our understanding of specific neuropsychiatric conditions. This will hopefully lead to novel treatments and alleviate some of the social stigma attached to these disorders. A better understanding of mental disease from a neural system prospective is absolutely necessary to promote mental health-related rational drug discovery.

我的研究计划的目标是确定奖赏学习和动机过程是如何在纹状体的神经活动中编码的。为了实现这一目标，我正在研究这个大脑区域信息处理的不同方面，从特定的谷氨酸输入如何影响动物从过去经验中概括的程度开始。 刺激泛化是一种将预测值归因于新的刺激的过程，这些新的刺激在感知上与先前学习的刺激相似。这导致动物主动接近或避免它们以前从未遇到过的刺激。这种现象背后的神经生物学机制引起了人们的极大兴趣，因为人类泛化的程度存在很大的差异，而某些神经精神疾病，如精神分裂症、广泛性焦虑症和创伤后应激障碍，与这一过程中的显著异常有关。本文描述的研究方案利用现代神经科学技术在寻找奖励的精确时刻操纵神经活动，目的是从机制上洞察刺激泛化的神经生物学。 这项研究的重点是腹侧纹状体中称为伏隔核的部分的信息处理。这一区域整合了中脑多巴胺能信号和来自皮质结构的兴奋性输入。伏隔核的主要兴奋性传入神经来自腹侧海马体、杏仁基底外侧核和前额叶皮质，所有这些区域都被认为调节着记忆的不同方面。来自人类功能磁共振成像研究的最新数据表明，这些谷氨酸输入伏隔核调控着人们在简单的奖赏任务中概括的程度。这些发现与啮齿动物功能实验的数据一致，这些实验表明，这些通路中每一条通路的活动增强足以加强和激励工具性行为。 我提议的技术方法包括在辨别性线索学习任务的特定时刻抑制对伏隔核的离散谷氨酸输入。光遗传技术将被用来实现这一点，它利用遗传编码的光敏蛋白质来抑制神经活动。这些技术提供的空间和时间精确度允许在学习和选择行为的精确时刻对递质释放进行特定途径的抑制。我对这种方法有着丰富的经验，将光基因操作与聪明的行为分析相结合。我的出版记录突显了我是一个合格的、训练有素的、致力于研究动机行为背后的神经回路的热情的研究人员。 这里提出的这项研究的首要目标是了解伏隔核中不同谷氨酸输入的活动如何影响与奖励相关的学习和选择行为。这项工作将引起广泛的兴趣，因为它涉及到动机，决策制定和强化学习的基础。此外，这项研究的多面性将为受训者提供广泛的技术技能以及收集、分析和解释数据的经验。 最终，我的研究计划将进一步加深我们对特定神经精神疾病的理解。这有望带来新的治疗方法，并减轻一些与这些疾病有关的社会耻辱。从神经系统的角度更好地理解精神疾病，对于促进与精神健康相关的合理药物发现是绝对必要的。