Encoding of both positive and negative reward prediction errors by neurons of the primate lateral prefrontal cortex and caudate nucleus.
Encoding of both positive and negative reward prediction errors by neurons of the primate lateral prefrontal cortex and caudate nucleus.
复制标题
DOI：
10.1523/jneurosci.3793-11.2011
复制
发表时间：
2011-12-07
期刊：
影响因子：
--
通讯作者：
Eskandar EN
中科院分区：
文献类型：
--
作者：
Asaad WF;Eskandar EN
意想不到的成功或意想不到的失败都可能是学习的动力。前者鼓励我们重复一个动作或活动，而后者则引导我们找到另一种策略。因此，理解这些意外事件的神经表征对于阐明学习相关回路至关重要。我们检测了猴子在进行试错学习任务时外侧前额叶皮层（PFC）和尾状核神经元的活动。意想不到的结果在这两个结构中都广泛存在，被意想不到的消极结果驱动的神经元和被意想不到的积极结果激活的神经元一样频繁。此外，正奖励和负奖励预测误差主要表现为放电率的增加，这与观察到的多巴胺神经元反映这些值的方式不同。有趣的是，阳性rpe出现的潜伏期往往比阴性rpe短，这可能反映了它们产生的机制。最后，在PFC而不是尾状核中，与结果相关的活动的一次又一次的变化与动物随后的行为决定有关。更广泛地说，这些神经元的RPE信号的鲁棒性表明，强化学习的行动者-批评者模型（PFC，特别是尾状核被认为主要是“行动者”而不是“批评者”）应该被重新考虑，以包括这些结构的突出评估作用。
Learning can be motivated by unanticipated success or unexpected failure. The former encourages us to repeat an action or activity, whereas the latter leads us to find an alternative strategy. Understanding the neural representation of these unexpected events is therefore critical to elucidate learning-related circuits. We examined the activity of neurons in the lateral prefrontal cortex (PFC) and caudate nucleus of monkeys as they performed a trial-and-error learning task. Unexpected outcomes were widely represented in both structures, and neurons driven by unexpectedly negative outcomes were as frequent as those activated by unexpectedly positive outcomes. Moreover, both positive and negative reward prediction errors were represented primarily by increases in firing rate, unlike the manner in which dopamine neurons have been observed to reflect these values. Interestingly, positive RPEs tended to appear with shorter latency than negative RPEs, perhaps reflecting the mechanism of their generation. Lastly, in the PFC but not the caudate, trial-by-trial variations in outcome-related activity were linked to the animals' subsequent behavioral decisions. More broadly, the robustness of RPE signaling by these neurons suggests that actor-critic models of reinforcement learning in which the PFC and particularly the caudate are considered primarily to be “actors” rather than “critics,” should be reconsidered to include a prominent evaluative role for these structures.