Rethinking dopamine as generalized prediction error
Rethinking dopamine as generalized prediction error
复制标题
DOI：
10.1098/rspb.2018.1645
复制
发表时间：
2018-11-21
影响因子：
4.7
通讯作者：
Gershman, Samuel J.
中科院分区：
文献类型：
--
作者：
Gardner, Matthew P. H.;Schoenbaum, Geoffrey;Gershman, Samuel J.
根据强化学习(RL)理论的假设，多巴胺神经元中的中脑通常被认为报告了奖赏预测错误(RPE)。虽然这一理论非常成功，但有几条证据表明，多巴胺活动也会编码与奖励无关的感觉预测错误。在这里，我们发展了一种新的多巴胺功能理论，它包含了对预测误差的更广泛的概念化。通过发出感官和奖励预测错误的信号，多巴胺支持介于基于模型的算法和无模型算法之间的一种形式的RL。这一解释与目前关于多巴胺瞬变和RPE之间的对应关系的经典观点保持一致，同时也解释了新的数据，表明这些信号在感觉预适应和身份解锁等现象中发挥了作用，这些现象表面上利用的知识超出了奖赏预测。
Midbra in dopamine neurons are commonly thought to report a reward prediction error (RPE), as hypothesized by reinforcement learning (RL) theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here, we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signalling errors in both sensory and reward predictions, dopamine supports a form of RL that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and RPEs, while also accounting for new data suggesting a role for those signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.