Generalization of Stimulus Value in Reinforcement Learning

强化学习中刺激值的推广

• 批准号: RGPIN-2014-05069
• 负责人: Britt, Jonathan
• 金额: $ 2.26万
• 依托单位: McGill University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2017
• 资助国家: 加拿大
• 起止时间: 2017-01-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=631475
• 关键词: Generalization; Stimulus; Value; Reinforcement; Learning

The goal of my research program is to determine how reward learning and motivational processes are encoded in neural activity in the striatum. To accomplish this goal, I am studying different facets of information processing in this brain region, starting with how specific glutamate inputs impact the extent to which animals generalize from past experience. Stimulus generalization is a process where predictive value is ascribed to novel stimuli that are perceptually similar to previously learned stimuli. This results in animals’ actively approaching or avoiding stimuli that they have never previously encountered. The neurobiological mechanisms underlying this phenomenon are of great interest, as there is a wide variability in the extent that humans generalize, and certain neuropsychiatric conditions, such as schizophrenia, generalized anxiety disorder, and post-traumatic stress disorder, are associated with marked abnormalities in this process. The research proposal described here utilizes modern neuroscience techniques to manipulate neural activity at precise moments during reward seeking with the goal of gaining mechanistic insight into the neurobiology of stimulus generalization.The research focuses on information processing in a part of the ventral striatum called the nucleus accumbens. This area integrates midbrain dopaminergic signals with excitatory input from cortical structures. The principal excitatory afferents to the nucleus accumbens come from the ventral hippocampus, basolateral amygdala, and prefrontal cortex, all regions which are thought to regulate distinct aspects of memory. Recent data from human fMRI research have implicated these glutamate inputs to the nucleus accumbens in regulating the extent to which people generalize during simple reward-seeking tasks. These findings are consistent with data from gain of function experiments in rodents, which demonstrate that enhanced activity in each of these pathways is sufficient to reinforce and motivate instrumental behavior.The technical approach I am proposing involves inhibiting discrete glutamate inputs to the nucleus accumbens at specific moments during a discriminative cue learning task. Optogenetic techniques will be used to accomplish this, which utilize genetically-encoded, light-sensitive proteins to inhibit neural activity. The spatial and temporal precision afforded by these techniques allow for pathway-specific inhibition of transmitter release at precise moments during learning and choice behaviour. I have extensive experience with this approach, with integrating optogenetic manipulations with clever behavioural assays. My publication record underscores how I am a well-qualified, highly trained, and committed researcher with a passion for studying the neural circuits underlying motivated behaviour. The overarching goal of the research proposed here is to understand how activity in different glutamate inputs to the nucleus accumbens impacts reward-related learning and choice behaviour. This work will be of wide interest as it touches upon the foundations of motivation, decision making, and reinforcement learning. In addition, the multifaceted nature of this research will provide trainees with extensive technical skills as well as experience collecting, analyzing, and interpreting data. Ultimately, my research program will further our understanding of specific neuropsychiatric conditions. This will hopefully lead to novel treatments and alleviate some of the social stigma attached to these disorders. A better understanding of mental disease from a neural system prospective is absolutely necessary to promote mental health-related rational drug discovery.

我的研究项目的目标是确定奖励学习和动机过程是如何在纹状体的神经活动中编码的。为了实现这一目标，我正在研究这个大脑区域信息处理的不同方面，从特定的谷氨酸输入如何影响动物从过去经验中归纳的程度开始。刺激泛化是一个过程，在这个过程中，新的刺激在知觉上与以前学过的刺激相似，从而具有预测价值。这导致动物主动接近或避开它们以前从未遇到过的刺激。这一现象背后的神经生物学机制引起了极大的兴趣，因为人类泛化的程度存在很大的可变性，某些神经精神疾病，如精神分裂症、广泛性焦虑症和创伤后应激障碍，都与这一过程中的明显异常有关。这里描述的研究计划利用现代神经科学技术来操纵在奖励寻求过程中精确时刻的神经活动，目的是获得刺激泛化的神经生物学机制。这项研究的重点是腹侧纹状体中被称为伏隔核的部分的信息处理。这个区域整合了中脑多巴胺能信号和来自皮质结构的兴奋性输入。伏隔核的主要兴奋性传入来自海马体腹侧、杏仁核基底外侧和前额皮质，这些区域被认为调节记忆的不同方面。最近来自人类功能磁共振成像研究的数据表明，这些谷氨酸输入到伏隔核中，在调节人们在简单的寻求奖励任务中泛化的程度。这些发现与啮齿类动物的功能增益实验的数据一致，这些实验表明，这些通路的增强活动足以加强和激发工具行为。我提出的技术方法包括在辨别线索学习任务的特定时刻抑制离散谷氨酸对伏隔核的输入。光遗传学技术将用于实现这一目标，该技术利用遗传编码的光敏蛋白来抑制神经活动。这些技术提供的空间和时间精度允许在学习和选择行为的精确时刻对递质释放的通路特异性抑制。我对这种方法有丰富的经验，将光遗传学操作与聪明的行为分析相结合。我的发表记录强调了我是一个非常合格的、训练有素的、忠诚的研究人员，对研究动机行为背后的神经回路充满热情。这里提出的研究的总体目标是了解不同谷氨酸输入伏隔核的活动如何影响与奖励相关的学习和选择行为。这项工作将引起广泛的兴趣，因为它涉及动机、决策和强化学习的基础。此外，这项研究的多面性将为受训者提供广泛的技术技能以及收集、分析和解释数据的经验。最终，我的研究项目将进一步加深我们对特定神经精神疾病的理解。这将有望带来新的治疗方法，并减轻与这些疾病有关的一些社会耻辱。从神经系统的角度更好地了解精神疾病对于促进精神健康相关药物的发现是绝对必要的。