Neural Mechanisms Underlying Reinforcement Learning

描述（由申请人提供）：情绪过程的神经回路功能障碍导致各种神经精神疾病的症状，包括抑郁症、焦虑症、精神分裂症和药物滥用。在动物和人类身上进行的大量研究表明，杏仁核参与了这些情绪过程。然而，在灵长类动物中，杏仁核神经生理学受到的关注有限，特别是在情绪学习方面。我们建议使用经典的条件反射技术，使猴子学习新的视觉（目标1）或听觉（目标2）条件刺激（CS）的价值。我们通过将每个CS与无条件刺激（US）任意配对来为CS分配一个值：液体奖励（积极），空气抽吸（消极）或无强化。猴子用两种反应来展示他们的学习：对奖励性CS的预期性舔，以及对负面CS的预期性眨眼，这是一种防御行为。我们将记录猴子学习CS值时单个杏仁核神经元的活动。强化学习理论认为，学习依赖于更新CS值的神经表示，这是一个由错误信号驱动的过程，错误信号指示预期和实际强化之间的差异。我们假设杏仁核神经元编码CS的值，并且这种表示在学习过程中迅速更新。此外，我们假设，杏仁核神经元将不同的反应意外相比，预期的强化，表明杏仁核活动反映错误信号。最后，我们假设其活动反映错误信号的神经元更有可能也编码CS值。在目标3中，我们试图确定在经典条件反射过程中反映CS值的神经信号是否最好被解释为CS值或预期强化的编码，这在更复杂的范例中不一定与CS值一致。在经典条件反射研究中，不可能消除这些解释的歧义，因为CS值总是预测试验结果。我们将使用一个场合设置的范例，其中预期的结果是操纵提出一个提示短暂CS发病前的一个子集的探测试验，同时保持个人CS值与定期空调试验。然后，我们将确定杏仁核神经元是否代表CS值或预期结果。这些研究与公共卫生直接相关，因为这些神经回路在许多精神疾病中功能障碍，包括成瘾，抑郁症和精神分裂症。

DESCRIPTION (provided by applicant): Dysfunction in neural circuits underlying emotional processes causes symptoms in a variety of neuropsychiatric disorders, including depression, anxiety disorders, schizophrenia, and drug abuse. Much work in animals and humans has implicated the amygdala in these emotional processes. In primates, however, amygdala neurophysiology has received limited attention, especially with regard to emotional learning. We propose using classical conditioning techniques so that monkeys learn the value of novel visual (Aim 1) or auditory (Aim 2) conditioned stimuli (CS). We assign CSs a value by arbitrarily pairing each CS with unconditioned stimuli (US): liquid reward (positive), air-puff (negative), or no reinforcement. Monkeys demonstrate their learning with two responses: anticipatory licking for rewarded CSs, and anticipatory blinking, a defensive behavior, for negative CSs. We will record the activity of individual amygdala neurons in monkeys as they learn the value of the CSs. Theories of reinforcement learning posit that learning depends upon updating a neural representation of CS value, a process that is driven by error signals indicating the difference between expected and actual reinforcement. We hypothesize that amygdala neurons code the value of CSs, and that this representation is rapidly updated during learning. Furthermore, we hypothesize that amygdala neurons will respond differentially to unexpected compared to expected reinforcement, indicating that amygdala activity reflects error signals. Finally, we hypothesize that neurons whose activity reflects error signals are more likely to also encode CS value. In Aim 3, we seek to determine whether neural signals reflecting CS value during classical conditioning are best interpreted as coding for CS value or for expected reinforcement, which in more complex paradigms is not necessarily aligned with CS value. In classical conditioning studies, it is not possible to disambiguate these interpretations because CS value always predicts trial outcome. We will use an occasion setting paradigm in which expected outcome is manipulated by presenting a cue transiently before CS onset on a subset of probe trials, while maintaining individual CS value with regular conditioning trials. We will then determine if amygdala neurons represent CS value or expected outcome. These studies have direct relevance to public health, since these neural circuits dysfunction in many psychiatric disorders, including addiction, depression, and schizophrenia.