Regularization Shortcomings for Continual Learning
Regularization Shortcomings for Continual Learning
复制标题
DOI：
--
复制
发表时间：
2019-12
期刊：
影响因子：
--
通讯作者：
Timothée Lesort;A. Stoian;David Filliat
中科院分区：
文献类型：
--
作者：
Timothée Lesort;A. Stoian;David Filliat
在大多数机器学习算法中，假设训练数据是独立同分布的（iid）。否则，算法的性能受到挑战。非iid数据分布的一个著名现象就是灾难性遗忘。处理它的算法集中在连续学习研究领域。在这篇文章中，我们研究了基于正则化的持续学习方法。我们表明，这些方法不能学习区分类从不同的任务在一个元素的连续基准：类增量设置。本文从理论上论证了这一缺陷，并通过实例和实验加以说明。
In most machine learning algorithms, training data are assumed independent and identically distributed (iid). Otherwise, the algorithms' performances are challenged. A famous phenomenon with non-iid data distribution is known as \say{catastrophic forgetting}. Algorithms dealing with it are gathered in the \textit{Continual Learning} research field. In this article, we study the \textit{regularization} based approaches to continual learning. We show that those approaches can not learn to discriminate classes from different tasks in an elemental continual benchmark: class-incremental setting. We make theoretical reasoning to prove this shortcoming and illustrate it with examples and experiments.