Abstract representations emerge naturally in neural networks trained to perform multiple tasks.
Abstract representations emerge naturally in neural networks trained to perform multiple tasks.
复制标题
DOI：
10.1038/s41467-023-36583-0
复制
发表时间：
2023-02-23
影响因子：
16.6
通讯作者：
Fusi S
中科院分区：
文献类型：
--
作者：
Johnston WJ;Fusi S
人类和其他动物在自然行为中表现出一种非凡的能力，即在不同的环境和对象中概括知识。我们假设这种泛化能力来自于一种特定的具象几何，我们称之为抽象，在机器学习中被称为解缠。这些抽象表征已在最近的神经生理学研究中观察到。然而，目前尚不清楚它们是如何出现的。在这里，使用前馈神经网络，我们证明了多个任务的学习导致抽象表征的出现，同时使用监督学习和强化学习。我们证明了这些抽象表征能够在新任务上实现少样本学习和可靠的泛化。我们的结论是，感觉和认知变量的抽象表征可能来自动物在自然世界中表现出的多种行为，因此，可能普遍存在于大脑的高级区域。我们还对哪些变量将被抽象地表示做出了一些具体的预测。动物是如何学会从一种环境到另一种环境进行概括的，目前还没有答案。在这里，作者表明，抽象表征被认为是这种泛化形式的基础，自然地出现在经过训练以执行多种任务的神经网络中。
Humans and other animals demonstrate a remarkable ability to generalize knowledge across distinct contexts and objects during natural behavior. We posit that this ability to generalize arises from a specific representational geometry, that we call abstract and that is referred to as disentangled in machine learning. These abstract representations have been observed in recent neurophysiological studies. However, it is unknown how they emerge. Here, using feedforward neural networks, we demonstrate that the learning of multiple tasks causes abstract representations to emerge, using both supervised and reinforcement learning. We show that these abstract representations enable few-sample learning and reliable generalization on novel tasks. We conclude that abstract representations of sensory and cognitive variables may emerge from the multiple behaviors that animals exhibit in the natural world, and, as a consequence, could be pervasive in high-level brain regions. We also make several specific predictions about which variables will be represented abstractly. How animals learn to generalize from one context to another remains unresolved. Here, the authors show that the abstract representations that are thought to underlie this form of generalization emerge naturally in neural networks trained to perform multiple tasks.
登录
查看更多内容
影响因子：
64.8
作者：
Harris CR;Millman KJ;van der Walt SJ;Gommers R;Virtanen P;Cournapeau D;Wieser E;Taylor J;Berg S;Smith NJ;Kern R;Picus M;Hoyer S;van Kerkwijk MH;Brett M;Haldane A;Del Río JF;Wiebe M;Peterson P;Gérard-Marchant P;Sheppard K;Reddy T;Weckesser W;Abbasi H;Gohlke C;Oliphant TE
通讯作者：
Oliphant TE
影响因子：
64.5
作者：
Bernardi S;Benna MK;Rigotti M;Munuera J;Fusi S;Salzman CD
通讯作者：
Salzman CD
影响因子：
2.7
作者：
Dai, Xili;Tong, Shengbang;Li, Mingyang;Wu, Ziyang;Psenka, Michael;Chan, Kwan Ho Ryan;Zhai, Pengyuan;Yu, Yaodong;Yuan, Xiaojun;Shum, Heung-Yeung;Ma, Yi
通讯作者：
Ma, Yi
影响因子：
5.7
作者：
通讯作者：
--
影响因子：
7.5
作者：
Caruana, R
通讯作者：
Caruana, R