Involvement of dopamine system and orbito frontal cortex in update and memory of value signals

我们研究了中脑多巴胺神经元是否编码长期奖励预测的信号及其误差。三只日本猴子进行多步选择3个奖励。在任务执行过程中的预期舔表明，猴子的期望并不反映单个试验的奖励概率，但精确地反映了预期总奖励的奖励值（概率x体积），未来奖励折扣，强化学习的价值函数。值得注意的是，多巴胺神经元活动编码了长期奖励预测，而不是单个试验的奖励值。结果表明，长期奖赏预测的多巴胺信号可能在纹状体等靶结构中更新指导未来目标的价值信号中发挥关键作用。

We examined whether the midbrain dopamine neurons encode signals of long-term reward prediction and its error. Three Japanese monkeys performed multi-step choices for 3 rewards. Anticipatory licking during the task performance showed that monkey's expectation did not reflect reward probability of individual trials but precisely reflected reward value (probability x volume) of expected total rewards with future rewards discounted, value function of reinforcement learning. Remarkably, dopamine neuron activity encoded the long-term reward prediction rather than reward value of individual trials. The results suggested that the dopamine signals of long-term reward prediction may play critical roles in update of value signals for directing future goals in the target structures such as the striatum.