Multiple systems in macaques for tracking prediction errors and other types of surprise.
Multiple systems in macaques for tracking prediction errors and other types of surprise.
复制标题
DOI：
10.1371/journal.pbio.3000899
复制
发表时间：
2020-10
期刊：
影响因子：
9.8
通讯作者：
Rushworth MFS
中科院分区：
文献类型：
--
作者：
Grohn J;Schüffelgen U;Neubert FX;Bongioanni A;Verhagen L;Sallet J;Kolling N;Rushworth MFS
动物从过去学习来做出预测。在预测误差之后调整这些预测，即，在令人惊讶的事件之后。通常，大多数奖励预测误差模型学习平均预期奖励量。然而，在这里，我们证明了存在不同的机制来检测其他类型的令人惊讶的事件。六只猕猴学会了对视觉刺激做出反应，以获得不同数量的果汁奖励。大多数试验以1或3滴果汁结束，因此动物学会了平均期望2滴果汁，尽管精确的2滴的情况很少。为了鼓励学习，我们还包括1和3滴之间的比例变化的会话。此外，在所有会话中，刺激有时会出现在意想不到的位置。因此，可能发生3种类型的意外事件：奖励金额意外（即，标量奖励预测误差）、罕见奖励惊喜和视觉空间惊喜。重要的是，我们可以将标量奖励预测错误（奖励偏离预期的平均奖励金额）和罕见奖励事件（奖励符合平均奖励预期，但很少发生）分离开来。我们使用功能性磁共振成像将每种类型的惊喜与不同的神经活动模式联系起来。多巴胺能中脑附近的活动只反映了对奖励数量的惊讶。外侧前额叶皮层在检测令人惊讶的事件方面具有更普遍的作用。后外侧眶额皮质专门检测罕见的奖励事件，无论他们是否遵循平均奖励金额的期望，但只有在可学习的奖励环境。动物从过去学习做出预测，每当令人惊讶的结果违反这些预测时，预测就会被调整。这项研究表明，猕猴使用多个系统来检测不同类型的罕见奖励事件;而多巴胺能中脑附近的活动反映了标量奖励预测错误（即奖励量），眶额皮质的活动反映了罕见的奖励信号。
Animals learn from the past to make predictions. These predictions are adjusted after prediction errors, i.e., after surprising events. Generally, most reward prediction errors models learn the average expected amount of reward. However, here we demonstrate the existence of distinct mechanisms for detecting other types of surprising events. Six macaques learned to respond to visual stimuli to receive varying amounts of juice rewards. Most trials ended with the delivery of either 1 or 3 juice drops so that animals learned to expect 2 juice drops on average even though instances of precisely 2 drops were rare. To encourage learning, we also included sessions during which the ratio between 1 and 3 drops changed. Additionally, in all sessions, the stimulus sometimes appeared in an unexpected location. Thus, 3 types of surprising events could occur: reward amount surprise (i.e., a scalar reward prediction error), rare reward surprise, and visuospatial surprise. Importantly, we can dissociate scalar reward prediction errors—rewards that deviated from the average reward amount expected—and rare reward events—rewards that accorded with the average reward expectation but that rarely occurred. We linked each type of surprise to a distinct pattern of neural activity using functional magnetic resonance imaging. Activity in the vicinity of the dopaminergic midbrain only reflected surprise about the amount of reward. Lateral prefrontal cortex had a more general role in detecting surprising events. Posterior lateral orbitofrontal cortex specifically detected rare reward events regardless of whether they followed average reward amount expectations, but only in learnable reward environments. Animals learn from the past to make predictions, and predictions are adjusted whenever surprising outcomes violate those predictions. This study shows that macaques use multiple systems to detect different types of rare reward events; while activity in the vicinity of the dopaminergic midbrain reflected scalar reward prediction errors (i.e. reward amount), activity in the orbitofrontal cortex reflected a rare reward signal.
登录
查看更多内容
影响因子：
16.2
作者：
Boorman ED;Rajendran VG;O'Reilly JX;Behrens TE
通讯作者：
Behrens TE
影响因子：
3.7
作者：
Caspari, Natalie;Arsenault, John T.;Vanduffel, Wim
通讯作者：
Vanduffel, Wim
影响因子：
16.2
作者：
Daw ND;Gershman SJ;Seymour B;Dayan P;Dolan RJ
通讯作者：
Dolan RJ
影响因子：
5.3
作者：
Hart, Andrew S.;Rutledge, Robb B.;Phillips, Paul E. M.
通讯作者：
Phillips, Paul E. M.
影响因子：
64.8
作者：
Burke, Kathryn A.;Franz, Theresa M.;Schoenbaum, Geoffrey
通讯作者：
Schoenbaum, Geoffrey