Neural Mechanisms Underlying Reinforcement Learning

描述(由申请人提供)：情绪过程中的神经回路功能障碍会导致各种神经精神障碍的症状，包括抑郁症、焦虑症、精神分裂症和药物滥用。在动物和人类身上的大量研究表明，杏仁核参与了这些情绪过程。然而，在灵长类动物中，杏仁核的神经生理学受到的关注有限，特别是在情感学习方面。我们建议使用经典的条件反射技术，以便猴子学习新的视觉(目标1)或听觉(目标2)条件刺激(CS)的价值。我们通过将每个CS与非条件刺激(US)任意配对来为CS赋值：液体奖励(积极)、喷气(消极)或不强化。猴子通过两种反应来展示它们的学习：对有奖励的CS的预期舔，以及对消极的CS的预期眨眼，这是一种防御性行为。我们将记录猴子体内单个杏仁核神经元的活动，因为它们了解到了css的价值。强化学习理论认为，学习依赖于更新CS值的神经表示，这一过程是由指示预期强化与实际强化之间的差异的错误信号驱动的。我们假设杏仁核神经元编码了css的值，并且这种表示在学习过程中会迅速更新。此外，我们假设杏仁核神经元对意外强化的反应与预期强化相比会有所不同，这表明杏仁核活动反映了错误信号。最后，我们假设，其活动反映错误信号的神经元更有可能也编码CS值。在目标3中，我们试图确定在经典条件反射中反映CS值的神经信号是最好地解释为CS值的编码还是预期强化的编码，在更复杂的范例中，预期强化不一定与CS值对齐。在经典的条件反射研究中，不可能消除这些解释的歧义，因为CS值总是预测试验结果。我们将使用一种情景设置范式，在这一范式中，预期结果是通过在一组探针试验中在CS开始之前短暂地呈现一个提示来操纵的，同时通过常规的条件性试验来维持个体的CS值。然后我们将确定杏仁核神经元是否代表CS值或预期结果。这些研究与公众健康直接相关，因为这些神经回路功能障碍在许多精神疾病中都存在，包括成瘾、抑郁症和精神分裂症。

DESCRIPTION (provided by applicant): Dysfunction in neural circuits underlying emotional processes causes symptoms in a variety of neuropsychiatric disorders, including depression, anxiety disorders, schizophrenia, and drug abuse. Much work in animals and humans has implicated the amygdala in these emotional processes. In primates, however, amygdala neurophysiology has received limited attention, especially with regard to emotional learning. We propose using classical conditioning techniques so that monkeys learn the value of novel visual (Aim 1) or auditory (Aim 2) conditioned stimuli (CS). We assign CSs a value by arbitrarily pairing each CS with unconditioned stimuli (US): liquid reward (positive), air-puff (negative), or no reinforcement. Monkeys demonstrate their learning with two responses: anticipatory licking for rewarded CSs, and anticipatory blinking, a defensive behavior, for negative CSs. We will record the activity of individual amygdala neurons in monkeys as they learn the value of the CSs. Theories of reinforcement learning posit that learning depends upon updating a neural representation of CS value, a process that is driven by error signals indicating the difference between expected and actual reinforcement. We hypothesize that amygdala neurons code the value of CSs, and that this representation is rapidly updated during learning. Furthermore, we hypothesize that amygdala neurons will respond differentially to unexpected compared to expected reinforcement, indicating that amygdala activity reflects error signals. Finally, we hypothesize that neurons whose activity reflects error signals are more likely to also encode CS value. In Aim 3, we seek to determine whether neural signals reflecting CS value during classical conditioning are best interpreted as coding for CS value or for expected reinforcement, which in more complex paradigms is not necessarily aligned with CS value. In classical conditioning studies, it is not possible to disambiguate these interpretations because CS value always predicts trial outcome. We will use an occasion setting paradigm in which expected outcome is manipulated by presenting a cue transiently before CS onset on a subset of probe trials, while maintaining individual CS value with regular conditioning trials. We will then determine if amygdala neurons represent CS value or expected outcome. These studies have direct relevance to public health, since these neural circuits dysfunction in many psychiatric disorders, including addiction, depression, and schizophrenia.