Visual novelty, curiosity, and intrinsic reward in machine learning and the brain
Visual novelty, curiosity, and intrinsic reward in machine learning and the brain
复制标题
DOI：
10.1016/j.conb.2019.08.004
复制
发表时间：
2019-10-01
影响因子：
5.7
通讯作者：
Rust, Nicole
中科院分区：
文献类型：
--
作者：
Jaegle, Andrew;Mehrpour, Vahid;Rust, Nicole
对新奇事物的强烈偏好出现在婴儿期，并在整个动物王国中普遍存在。当结合基于强化的机器学习算法时，视觉新颖性可以作为一种内在的奖励信号，极大地提高了探索的效率并加速了学习，特别是在难以获得外部奖励的情况下。在这里，我们回顾了新颖性驱动的机器学习算法的最新发展与我们对视觉新颖性如何在灵长类动物大脑中计算和发出信号的理解之间的相似之处。我们提出，在视觉系统中，新颖性表征不是以检测新物体为主要目标，而是以灵活概括新颖性信息为更广泛的目标，以推动基于新颖性的学习。
A strong preference for novelty emerges in infancy and is prevalent across the animal kingdom. When incorporated into reinforcement-based machine learning algorithms, visual novelty can act as an intrinsic reward signal that vastly increases the efficiency of exploration and expedites learning, particularly in situations where external rewards are difficult to obtain. Here we review parallels between recent developments in novelty-driven machine learning algorithms and our understanding of how visual novelty is computed and signaled in the primate brain. We propose that in the visual system, novelty representations are not configured with the principal goal of detecting novel objects, but rather with the broader goal of flexibly generalizing novelty information across different states in the service of driving novelty-based learning.