Stimulus-dependent adjustment of reward prediction error in the midbrain.
Stimulus-dependent adjustment of reward prediction error in the midbrain.
复制标题
DOI：
10.1371/journal.pone.0028337
复制
发表时间：
2011
期刊：
影响因子：
3.7
通讯作者：
Okuda J
中科院分区：
文献类型：
--
作者：
Takemura H;Samejima K;Vogels R;Sakagami M;Okuda J
以前的报告已经描述了中脑多巴胺区域的神经活动对意外的奖励传递和遗漏很敏感。这些活动与强化学习模型中的奖赏预测误差、奖赏预测值与获得的奖赏结果之间的差异相关。这些发现表明，大脑中的奖赏预测误差信号通过刺激-奖赏经验更新奖赏预测。然而，奖赏预测刺激的感觉加工如何影响奖赏预测误差的计算，目前尚不清楚。为了阐明这一问题，我们使用功能磁共振成像(FMRI)检测了奖赏预测刺激的刺激可识别性与大脑中奖赏预测误差信号之间的关系。在主要实验之前，受试者学习感知显著(高对比度)Gabor贴片的方向与果汁奖励之间的联系。然后向受试者呈现对比度较低的Gabor贴片刺激，以预测奖励。我们计算了两种强化学习模型中fMRI信号与奖赏预测误差之间的相关性：一种模型考虑了刺激可区分性对奖赏预测的调制，另一种模型排除了这种调制。结果表明，在包含刺激可识别性的模型中，中脑的fMRI信号与奖励预测误差的相关性比在排除刺激可识别性的模型中更高。没有地区与排除刺激辨别力的模型显示出更高的相关性。此外，结果表明，两个模型之间的相关性从实验的第一阶段开始就有了显著的差异，这表明在学习知觉模糊刺激和奖励之间的新的偶然性之前，中脑的奖励计算是基于刺激的辨别力来调节的。这些结果表明，人类的奖赏系统可以通过调节以前获得的典型刺激的奖励值，灵活地将刺激可分辨程度纳入奖赏计算。
Previous reports have described that neural activities in midbrain dopamine areas are sensitive to unexpected reward delivery and omission. These activities are correlated with reward prediction error in reinforcement learning models, the difference between predicted reward values and the obtained reward outcome. These findings suggest that the reward prediction error signal in the brain updates reward prediction through stimulus–reward experiences. It remains unknown, however, how sensory processing of reward-predicting stimuli contributes to the computation of reward prediction error. To elucidate this issue, we examined the relation between stimulus discriminability of the reward-predicting stimuli and the reward prediction error signal in the brain using functional magnetic resonance imaging (fMRI). Before main experiments, subjects learned an association between the orientation of a perceptually salient (high-contrast) Gabor patch and a juice reward. The subjects were then presented with lower-contrast Gabor patch stimuli to predict a reward. We calculated the correlation between fMRI signals and reward prediction error in two reinforcement learning models: a model including the modulation of reward prediction by stimulus discriminability and a model excluding this modulation. Results showed that fMRI signals in the midbrain are more highly correlated with reward prediction error in the model that includes stimulus discriminability than in the model that excludes stimulus discriminability. No regions showed higher correlation with the model that excludes stimulus discriminability. Moreover, results show that the difference in correlation between the two models was significant from the first session of the experiment, suggesting that the reward computation in the midbrain was modulated based on stimulus discriminability before learning a new contingency between perceptually ambiguous stimuli and a reward. These results suggest that the human reward system can incorporate the level of the stimulus discriminability flexibly into reward computations by modulating previously acquired reward values for a typical stimulus.
登录
查看更多内容
DOI：
10.1073/pnas.0607716103
发表时间：
2006-12-05
影响因子：
11.1
作者：
Lau, Hakwan C.;Passingham, Richard E.
通讯作者：
Passingham, Richard E.
影响因子：
64.8
作者：
NEWSOME, WT;BRITTEN, KH;MOVSHON, JA
通讯作者：
MOVSHON, JA
影响因子：
--
作者：
Pelli, DG
通讯作者：
Pelli, DG
影响因子：
56.9
作者：
Samejima, K;Ueda, Y;Kimura, M
通讯作者：
Kimura, M
影响因子：
16.2
作者：
O'Doherty, JP;Buchanan, TW;Dolan, RJ
通讯作者：
Dolan, RJ