Overcoming Catastrophic Forgetting for Continual Learning via Model Adaptation
Overcoming Catastrophic Forgetting for Continual Learning via Model Adaptation
复制标题
DOI：
--
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
Wenpeng Hu;Zhou Lin-;Bing Liu;Chongyang Tao;Zhengwei Tao;Jinwen Ma;Dongyan Zhao;Rui Yan
中科院分区：
文献类型：
--
作者：
Wenpeng Hu;Zhou Lin-;Bing Liu;Chongyang Tao;Zhengwei Tao;Jinwen Ma;Dongyan Zhao;Rui Yan
顺序学习多个任务对于人工智能和终身学习系统的发展非常重要。然而，标准的神经网络结构遭受灾难性遗忘的困扰，这使得它们很难学习一系列任务。为了解决这个问题，已经提出了几种持续学习的方法。在本文中，我们提出了一种非常不同的方法，称为参数生成和模型自适应(PGMA)，来处理这个问题。提出的方法学习建立一个模型，称为求解器，具有两组参数。第一个集合由到目前为止学习的所有任务共享，第二个集合是动态生成的，以使求解器适应每个测试用例，以便对其进行分类。大量的实验证明了该方法的有效性。
Learning multiple tasks sequentially is important for the development of AI and lifelong learning systems. However, standard neural network architectures suffer from catastrophic forgetting which makes it difficult for them to learn a sequence of tasks. Several continual learning methods have been proposed to address the problem. In this paper, we propose a very different approach, called Parameter Generation and Model Adaptation (PGMA), to dealing with the problem. The proposed approach learns to build a model, called the solver, with two sets of parameters. The first set is shared by all tasks learned so far and the second set is dynamically generated to adapt the solver to suit each test example in order to classify it. Extensive experiments have been carried out to demonstrate the effectiveness of the proposed approach.