Dopamine-Mediated Reinforcement Learning Signals in the Striatum and Ventromedial Prefrontal Cortex Underlie Value-Based Choices
Dopamine-Mediated Reinforcement Learning Signals in the Striatum and Ventromedial Prefrontal Cortex Underlie Value-Based Choices
复制标题
DOI：
10.1523/jneurosci.3904-10.2011
复制
发表时间：
2011-02-02
影响因子：
5.3
通讯作者：
Ullsperger, Markus
中科院分区：
文献类型：
--
作者：
Jocham, Gerhard;Klein, Tilmann A.;Ullsperger, Markus
大量的证据表明多巴胺在强化学习中的作用。关于多巴胺如何塑造积极和消极结果的相对影响，以指导基于价值的选择，我们知之甚少。在健康志愿者中，我们将多巴胺D（2）受体拮抗剂氨磺必利与功能性磁共振成像联合给药。氨磺必利不影响初始强化学习。然而，在后来的转移阶段，涉及新的选择情况，需要根据他们以前学到的价值在两个符号之间做出决定，氨磺必利提高了参与者选择两个高回报选项中更好的选项的能力，而它对两个非常差的选项之间的选择没有影响。在学习阶段，纹状体的活动编码了奖励预测错误。在转移阶段，在没有任何结果的情况下，腹内侧前额叶皮层（vmPFC）持续跟踪每次试验中可用选项的学习价值。纹状体预测错误编码和vmPFC中学习值的跟踪都可以预测受试者在转移阶段的选择表现，并且在氨磺必利下都得到了增强。这些发现表明，多巴胺依赖性机制增强了纹状体中的强化学习信号，并使前额叶皮层中用于指导基于认知的决策的关联值的表示更加清晰。
A large body of evidence exists on the role of dopamine in reinforcement learning. Less is known about how dopamine shapes the relative impact of positive and negative outcomes to guide value-based choices. We combined administration of the dopamine D(2) receptor antagonist amisulpride with functional magnetic resonance imaging in healthy human volunteers. Amisulpride did not affect initial reinforcement learning. However, in a later transfer phase that involved novel choice situations requiring decisions between two symbols based on their previously learned values, amisulpride improved participants' ability to select the better of two highly rewarding options, while it had no effect on choices between two very poor options. During the learning phase, activity in the striatum encoded a reward prediction error. In the transfer phase, in the absence of any outcome, ventromedial prefrontal cortex (vmPFC) continually tracked the learned value of the available options on each trial. Both striatal prediction error coding and tracking of learned value in the vmPFC were predictive of subjects' choice performance in the transfer phase, and both were enhanced under amisulpride. These findings show that dopamine-dependent mechanisms enhance reinforcement learning signals in the striatum and sharpen representations of associative values in prefrontal cortex that are used to guide reinforcement-based decisions.