Adding prediction risk to the theory of reward learning
Adding prediction risk to the theory of reward learning
复制标题
DOI：
10.1196/annals.1390.005
复制
发表时间：
2007-01-01
期刊：
影响因子：
--
通讯作者：
Bossaerts, Peter
中科院分区：
文献类型：
--
作者：
Preuschoff, Kerstin;Bossaerts, Peter
本文从最小二乘学习理论的角度分析了简单的Rescorla-Wagner学习规则。特别是，它建议如何使用风险度量，如预测风险，来调整强化学习中的学习常数。它认为，通过对预测误差进行缩放，可以最有效地纳入预测风险。这样，只有当最优预测和过去(按比例调整的)预测误差之间的协方差改变时，才需要调整学习率。讨论的证据表明，(人类和非人类)灵长类动物大脑中的多巴胺能系统编码预测风险，并且预测误差确实与预测风险(自适应编码)成比例。
This article analyzes the simple Rescorla-Wagner learning rule from the vantage point of least squares learning theory. In particular, it suggests how measures of risk, such as prediction risk, can be used to adjust the learning constant in reinforcement learning. It argues that prediction risk is most effectively incorporated by scaling the prediction errors. This way, the learning rate needs adjusting only when the covariance between optimal predictions and past (scaled) prediction errors changes. Evidence is discussed that suggests that the dopaminergic system in the (human and nonhuman) primate brain encodes prediction risk, and that prediction errors are indeed scaled with prediction risk (adaptive encoding).