A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning.
A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning.
复制标题
DOI：
10.1038/s41593-022-01109-2
复制
发表时间：
2022-08
影响因子：
25
通讯作者：
Watabe-Uchida, Mitsuko
中科院分区：
文献类型：
--
作者：
Amo, Ryunosuke;Matias, Sara;Yamanaka, Akihiro;Tanaka, Kenji F.;Uchida, Naoshige;Watabe-Uchida, Mitsuko
大量证据表明，中脑多巴胺神经元的相位反应与机器学习中使用的一种教学信号（时间差[TD]错误）有着显著的相似性。然而，以前的研究未能观察到该算法的一个关键预测：当代理将在时间上分离的提示和奖励联系起来时，多巴胺信号的时间应该在多次试验中从奖励时间逐渐向后移动到提示时间。在这里，我们证明，这种逐渐转变发生在多巴胺能细胞活性和多巴胺释放水平在小鼠腹侧纹状体。我们的研究结果建立了多巴胺能活动和TD学习算法之间长期寻求的联系，为大脑如何将时间上分离的线索和奖励联系起来提供了基本见解。
A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference [TD] error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here, we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.
登录
查看更多内容
神经元型特异性信号，用于腹侧对段区域的奖励和惩罚。
影响因子：
64.8
作者：
Cohen, Jeremiah Y.;Haesler, Sebastian;Vong, Linh;Lowell, Bradford B.;Uchida, Naoshige
通讯作者：
Uchida, Naoshige
影响因子：
64.8
作者：
Flagel SB;Clark JJ;Robinson TE;Mayo L;Czuj A;Willuhn I;Akers CA;Clinton SM;Phillips PE;Akil H
通讯作者：
Akil H
影响因子：
4.6
作者：
Inutsuka A;Yamashita A;Chowdhury S;Nakai J;Ohkura M;Taguchi T;Yamanaka A
通讯作者：
Yamanaka A
影响因子：
25
作者：
Madisen L;Zwingman TA;Sunkin SM;Oh SW;Zariwala HA;Gu H;Ng LL;Palmiter RD;Hawrylycz MJ;Jones AR;Lein ES;Zeng H
通讯作者：
Zeng H
影响因子：
3.4
作者：
Botvinick MM;Niv Y;Barto AG
通讯作者：
Barto AG