fMRI and EEG Predictors of Dynamic Decision Parameters during Human Reinforcement Learning
fMRI and EEG Predictors of Dynamic Decision Parameters during Human Reinforcement Learning
复制标题
人类强化学习期间动态决策参数的 fMRI 和 EEG 预测器
DOI：
10.1523/jneurosci.2036-14.2015
复制
发表时间：
2015-01-14
影响因子：
5.3
通讯作者：
Badre, David
中科院分区：
文献类型：
--
作者：
Frank, Michael J.;Gagne, Chris;Badre, David
在强化学习过程中，选择过程的神经动力学是什么？两个主要独立的文献研究了强化学习（RL）的动态作为经验的函数，但假设一个静态的选择过程，或者相反，决策过程中的选择过程的动态，但基于静态的决策值。本研究表明，在强化学习过程中，人类的选择过程可以通过决策的漂移扩散模型（DDM）来很好地描述，在该模型中，对学习到的逐次奖励值进行顺序采样，并在值信号超过决策阈值时做出选择。此外，同时进行的功能磁共振成像和脑电图记录显示，这一决策阈值在不同的试验中并不是固定的，而是作为丘脑下核（STN）活动的函数而变化，并通过对决策冲突和背内侧额叶皮层活动的试验测量进一步调节（在脑电图中，前sma bold和中额叶θ）。这些发现为一个模型提供了收敛的多模态证据，在这个模型中，当选择的奖励值有细微差异时，基于奖励的任务的决策阈值作为从前sma到STN的通信函数进行调整，从而允许更多的时间选择统计上更有回报的选项。
What are the neural dynamics of choice processes during reinforcement learning? Two largely separate literatures have examined dynamics of reinforcement learning (RL) as a function of experience but assuming a static choice process, or conversely, the dynamics of choice processes in decision making but based on static decision values. Here we show that human choice processes during RL are well described by a drift diffusion model (DDM) of decision making in which the learned trial-by-trial reward values are sequentially sampled, with a choice made when the value signal crosses a decision threshold. Moreover, simultaneous fMRI and EEG recordings revealed that this decision threshold is not fixed across trials but varies as a function of activity in the subthalamic nucleus (STN) and is further modulated by trial-by-trial measures of decision conflict and activity in the dorsomedial frontal cortex (pre-SMA BOLDand mediofrontal theta in EEG). These findings provide converging multimodal evidence for a model in which decision threshold in reward-based tasks is adjusted as a function of communication from pre-SMA to STN when choices differ subtly in reward values, allowing more time to choose the statistically more rewarding option.