Neural Mechanisms Underlying Reinforcement Learning

描述（由申请人提供）：情绪过程中神经回路的功能障碍导致各种神经精神疾病的症状，包括抑郁症、焦虑症、精神分裂症和药物滥用。对动物和人类的大量研究表明，杏仁核参与了这些情绪过程。然而，在灵长类动物中，杏仁核神经生理学受到的关注有限，尤其是在情绪学习方面。我们建议使用经典条件反射技术，使猴子学习新的视觉（目标1）或听觉（目标2）条件刺激（CS）的价值。我们通过将每个CS与非条件刺激（US）任意配对来为CS分配一个值：液体奖励（正），充气（负）或无强化。猴子通过两种反应来展示他们的学习能力：获得奖励的CSs的预期舔，以及消极的CSs的预期眨眼，一种防御行为。我们将记录猴子在学习CSs值时单个杏仁核神经元的活动。强化学习理论假设学习依赖于更新CS值的神经表示，这一过程是由表明期望和实际强化之间差异的错误信号驱动的。我们假设杏仁核神经元编码CSs的值，并且这种表示在学习过程中迅速更新。此外，我们假设杏仁核神经元对意外强化和预期强化的反应不同，这表明杏仁核的活动反映了错误信号。最后，我们假设反映错误信号的神经元也更有可能编码CS值。在目标3中，我们试图确定在经典条件反射过程中反映CS值的神经信号是否最好地解释为CS值编码或预期强化编码，而在更复杂的范式中，预期强化不一定与CS值一致。在经典条件反射研究中，不可能消除这些解释的歧义，因为CS值总是预测试验结果。我们将使用一种场合设定范式，通过在一组探针试验中短暂地在CS发作前提供线索来操纵预期结果，同时在常规条件反射试验中保持个体CS值。然后我们将确定杏仁核神经元是否代表CS值或预期结果。这些研究与公共卫生直接相关，因为这些神经回路在许多精神疾病中出现功能障碍，包括成瘾、抑郁和精神分裂症。

DESCRIPTION (provided by applicant): Dysfunction in neural circuits underlying emotional processes causes symptoms in a variety of neuropsychiatric disorders, including depression, anxiety disorders, schizophrenia, and drug abuse. Much work in animals and humans has implicated the amygdala in these emotional processes. In primates, however, amygdala neurophysiology has received limited attention, especially with regard to emotional learning. We propose using classical conditioning techniques so that monkeys learn the value of novel visual (Aim 1) or auditory (Aim 2) conditioned stimuli (CS). We assign CSs a value by arbitrarily pairing each CS with unconditioned stimuli (US): liquid reward (positive), air-puff (negative), or no reinforcement. Monkeys demonstrate their learning with two responses: anticipatory licking for rewarded CSs, and anticipatory blinking, a defensive behavior, for negative CSs. We will record the activity of individual amygdala neurons in monkeys as they learn the value of the CSs. Theories of reinforcement learning posit that learning depends upon updating a neural representation of CS value, a process that is driven by error signals indicating the difference between expected and actual reinforcement. We hypothesize that amygdala neurons code the value of CSs, and that this representation is rapidly updated during learning. Furthermore, we hypothesize that amygdala neurons will respond differentially to unexpected compared to expected reinforcement, indicating that amygdala activity reflects error signals. Finally, we hypothesize that neurons whose activity reflects error signals are more likely to also encode CS value. In Aim 3, we seek to determine whether neural signals reflecting CS value during classical conditioning are best interpreted as coding for CS value or for expected reinforcement, which in more complex paradigms is not necessarily aligned with CS value. In classical conditioning studies, it is not possible to disambiguate these interpretations because CS value always predicts trial outcome. We will use an occasion setting paradigm in which expected outcome is manipulated by presenting a cue transiently before CS onset on a subset of probe trials, while maintaining individual CS value with regular conditioning trials. We will then determine if amygdala neurons represent CS value or expected outcome. These studies have direct relevance to public health, since these neural circuits dysfunction in many psychiatric disorders, including addiction, depression, and schizophrenia.