An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning.
An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning.
复制标题
DOI：
10.3389/fncom.2014.00047
复制
发表时间：
2014
影响因子：
3.2
通讯作者：
Moustafa AA
中科院分区：
文献类型：
--
作者：
Balasubramani PP;Chakravarthy VS;Ravindran B;Moustafa AA
虽然经验和神经研究表明5-羟色胺(5-羟色胺)在大脑中起着许多功能作用，但以前的计算模型大多集中在它在行为抑制中的作用。在这项研究中，我们提出了一个改进的强化学习(RL)框架下的基于风险的决策模型。该模型描述了多巴胺(DA)和5-羟色胺(5HT)在基底节(BG)中的作用。在该模型中，DA信号由时间差分误差(δ)表示，而5HT信号由控制风险预测误差的参数(α)表示。这种既能适应5-羟色胺又能适应DA的提法调和了5-羟色胺的一些不同作用，特别是与BG系统有关的作用。我们将该模型应用于不同的实验范式，用于研究5HT的作用：(1)风险敏感决策，其中5HT控制风险评估；(2)时间奖励预测，其中5HT控制奖励预测的时间尺度；(3)奖惩敏感性，其中惩罚预测误差取决于5HT水平。因此，提出的综合RL模型协调了BG中现有的几种5HT和DA理论。
Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.
登录
查看更多内容
影响因子：
3.8
作者：
Di Mascio, M;Di Giovanni, G;Esposito, E
通讯作者：
Esposito, E
影响因子：
5.4
作者：
BELL, DE
通讯作者：
BELL, DE
影响因子：
14.5
作者：
Bodi, Nikoletta;Keri, Szabolcs;Gluck, Mark A.
通讯作者：
Gluck, Mark A.
影响因子：
5.3
作者：
Floresco, SB;Ghods-Sharifi, S;Magyar, O
通讯作者：
Magyar, O
影响因子：
8.6
作者：
Goetz, CG;Chmura, TA;Lanska, DJ
通讯作者：
Lanska, DJ