Neural modularity helps organisms evolve to learn new skills without forgetting old skills.
Neural modularity helps organisms evolve to learn new skills without forgetting old skills.
复制标题
DOI：
10.1371/journal.pcbi.1004128
复制
发表时间：
2015-04
影响因子：
4.3
通讯作者：
Clune J
中科院分区：
文献类型：
--
作者：
Ellefsen KO;Mouret JB;Clune J
人工智能的一个长期目标是创造能够为不同问题学习各种不同技能的代理。在神经网络的人工智能子领域，实现这一目标的一个障碍是，当智能体学习一项新技能时，它们通常会失去之前获得的技能，这种问题被称为灾难性遗忘（catastrophic forgetting）。这是因为，为了学习新任务，神经学习算法改变了编码先前获得技能的连接。网络的组织方式严重影响着他们的学习动态。在本文中，我们测试了是否可以通过进化的模块化神经网络来减少灾难性遗忘。模块化应该直观地减少任务之间的学习干扰，方法是将功能分离为物理上不同的模块，在这些模块中可以选择性地打开或关闭学习。模块化可以通过将强化学习模块与感官处理模块分开来进一步改善学习，允许学习仅在对积极或消极奖励做出反应时发生。在本文中，学习是通过神经调节发生的，它允许代理根据环境刺激选择性地改变每个神经连接的学习速率（例如，根据手头的任务改变特定位置的学习）。为了产生模块化，我们以神经连接为代价来进化神经网络。我们表明这种连接成本技术导致模块化，证实了之前的结果，并且这种稀疏连接的模块化网络具有更高的整体性能，因为它们更快地学习新技能，同时更多地保留旧技能，并且因为它们有一个单独的强化学习模块。我们的研究结果表明：(1)鼓励神经网络的模块化可以帮助我们克服长期存在的网络障碍，即不能在不忘记旧技能的情况下学习新技能；(2)自然动物大脑中普遍存在的模块化的一个好处可能是减轻灾难性遗忘的问题。人工智能（AI）的一个长期目标是创建计算大脑模型（神经网络），以便在新情况下学习该做什么。一个障碍是，智能体通常只能通过失去先前获得的技能来学习新技能。在这里，我们测试了进化的模块化神经网络是否会减少这种遗忘，模块化神经网络是指由许多不同的神经元亚群组成的网络。模块化在直观上应该有所帮助，因为学习只能在学习新任务的模块中选择性地开启。我们证实了这一假设：模块化网络具有更高的整体性能，因为它们更快地学习新技能，同时更多地保留旧技能。我们的研究结果表明，自然动物大脑中模块化的一个好处可能是允许学习而不忘记。
A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to alleviate the problem of catastrophic forgetting. A long-standing goal in artificial intelligence (AI) is creating computational brain models (neural networks) that learn what to do in new situations. An obstacle is that agents typically learn new skills only by losing previously acquired skills. Here we test whether such forgetting is reduced by evolving modular neural networks, meaning networks with many distinct subgroups of neurons. Modularity intuitively should help because learning can be selectively turned on only in the module learning the new task. We confirm this hypothesis: modular networks have higher overall performance because they learn new skills faster while retaining old skills more. Our results suggest that one benefit of modularity in natural animal brains may be allowing learning without forgetting.
登录
查看更多内容
影响因子：
32.8
作者：
Bengio, Yoshua
通讯作者：
Bengio, Yoshua
DOI：
10.1098/rspb.2009.0602
发表时间：
2009-09-07
影响因子：
4.7
作者：
Dunlap, Aimee S.;Stephens, David W.
通讯作者：
Stephens, David W.
DOI：
10.1016/s0764-4469(97)82472-9
发表时间：
1997-12-01
期刊：
COMPTES RENDUS DE L ACADEMIE DES SCIENCES SERIE III-SCIENCES DE LA VIE-LIFE SCIENCES
影响因子：
--
作者：
Ans, B;Rousset, S
通讯作者：
Rousset, S
影响因子：
8.6
作者：
Aaltonen, T.;Adelman, J.;Zucchelli, S.
通讯作者：
Zucchelli, S.
DOI：
10.1016/j.physa.2005.12.013
发表时间：
2006-07-15
影响因子：
3.3
作者：
Ahn, Yong-Yeol;Jeong, Hawoong;Kim, Beom Jun
通讯作者：
Kim, Beom Jun