Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain
Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain
复制标题
DOI：
10.1016/j.neuron.2022.05.025
复制
发表时间：
2022-08-17
期刊：
影响因子：
16.2
通讯作者：
O'Doherty, John P.
中科院分区：
文献类型：
--
作者：
Cockburn, Jeffrey;Man, Vincent;O'Doherty, John P.
新颖性和不确定性都是指导探索的有力特征；然而，它们经常在实验上被混为一谈，并且对它们如何相互作用以调节探索和利用之间的平衡的理解已被证明是难以捉摸的。使用旨在解耦新颖性和不确定性影响的任务，我们确定了指导探索的可分离机制。我们表明，不确定性导向的探索对新信息提供的预期收益很敏感，而新颖性导向的探索无论其潜在优势如何都会被维持。使用与功能磁共振成像相结合的计算框架，我们表明不确定性导向的选择植根于索引探索的预期效用的自适应偏差。相比之下，新颖性通过乐观地提高奖励期望同时抑制不确定性信号来持续促进探索。我们的结果确定了可分离的神经基质，负责平衡探索/利用权衡，以促进对其他棘手问题的可控分解。
Both novelty and uncertainty are potent features guiding exploration; however, they are often experimentally conflated, and an understanding of how they interact to regulate the balance between exploration and exploi-tation has proved elusive. Using a task designed to decouple the influence of novelty and uncertainty, we identify separable mechanisms through which exploration is directed. We show that uncertainty-directed exploration is sensitive to the prospective benefit offered by new information, whereas novelty-directed exploration is maintained regardless of its potential advantage. Using a computational framework in conjunc-tion with fMRI, we show that uncertainty-directed choice is rooted in an adaptive bias indexing the prospec-tive utility of exploration. In contrast, novelty persistently promotes exploration by optimistically inflating reward expectations while simultaneously dampening uncertainty signals. Our results identify separable neu-ral substrates charged with balancing the explore/exploit trade-off to foster a manageable decomposition of an otherwise intractable problem.