The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection.
The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection.
复制标题
DOI：
10.1016/j.neuron.2020.10.013
复制
发表时间：
2021-01-06
期刊：
影响因子：
16.2
通讯作者：
Costa RM
中科院分区：
文献类型：
--
作者：
Akam T;Rodrigues-Vaz I;Marcelo I;Zhang X;Pereira M;Oliveira RF;Dayan P;Costa RM
行为控制不是单一的。它包括基于模型和无模型的并行系统，它们分别产生灵活的和习惯性的行为。基于模型的决策使用对行为的具体后果的预测，但这些预测是如何在大脑中实现的，人们知之甚少。我们在小鼠的顺序决策任务中使用钙成像和光遗传学来显示前扣带皮层（ACC）预测行为将导致的状态，而不仅仅是它们是好是坏，并监测结果是否与这些预测相符。ACC代表任务的完整状态空间，其中奖励信号强烈依赖于获得奖励的状态，但对之前的选择影响最小。因此，ACC仅对基于模型的策略的更新是必要的，而对基本的奖励驱动的行动强化则不需要。这些结果表明，ACC在基于模型的控制中是一个关键节点，在预测给定动作的未来状态中起着特定的作用。一种新的两步任务消除了小鼠基于模型和无模型的RL的模棱两可，ACC代表任务状态空间，奖励是由状态语境化的。ACC预测所选动作的未来状态，并编码状态预测惊喜。发现ACC预测了给定行为的未来状态，并指出这些预测何时被违反。短暂抑制ACC会阻止小鼠使用观察到的状态转换来指导后续选择，从而损害基于模型的强化学习。
Behavioral control is not unitary. It comprises parallel systems, model based and model free, that respectively generate flexible and habitual behaviors. Model-based decisions use predictions of the specific consequences of actions, but how these are implemented in the brain is poorly understood. We used calcium imaging and optogenetics in a sequential decision task for mice to show that the anterior cingulate cortex (ACC) predicts the state that actions will lead to, not simply whether they are good or bad, and monitors whether outcomes match these predictions. ACC represents the complete state space of the task, with reward signals that depend strongly on the state where reward is obtained but minimally on the preceding choice. Accordingly, ACC is necessary only for updating model-based strategies, not for basic reward-driven action reinforcement. These results reveal that ACC is a critical node in model-based control, with a specific role in predicting future states given chosen actions. A novel two-step task disambiguates model-based and model-free RL in mice ACC represents the task state space, and reward is contextualized by state ACC predicts future states given chosen actions and encodes state prediction surprise Inhibiting ACC prevents state transitions, but not rewards, from influencing choice Akam et al. investigate mouse anterior cingulate cortex (ACC) in a sequential decision-making task, finding that ACC predicts future states given chosen actions and indicates when these predictions are violated. Transiently inhibiting ACC prevents mice from using observed state transitions to guide subsequent choices, impairing model-based reinforcement learning.
登录
查看更多内容
影响因子：
25
作者：
Doll, Bradley B.;Duncan, Katherine D.;Daw, Nathaniel D.
通讯作者：
Daw, Nathaniel D.
影响因子：
16.2
作者：
Daw ND;Gershman SJ;Seymour B;Dayan P;Dolan RJ
通讯作者：
Dolan RJ
DOI：
10.1080/14640748108400816
发表时间：
1981-01-01
期刊：
QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY SECTION B-COMPARATIVE AND PHYSIOLOGICAL PSYCHOLOGY
影响因子：
--
作者：
ADAMS, CD;DICKINSON, A
通讯作者：
DICKINSON, A
影响因子：
5.7
作者：
Huang, Yi;Yaple, Zachary A.;Yu, Rongjun
通讯作者：
Yu, Rongjun
DOI：
10.1523/jneurosci.3864-11.2012
发表时间：
2012-03-14
期刊：
The Journal of neuroscience : the official journal of the Society for Neuroscience
影响因子：
--
作者：
Cai X;Padoa-Schioppa C
通讯作者：
Padoa-Schioppa C