Continual Learning in Linear Classification on Separable Data
Continual Learning in Linear Classification on Separable Data
复制标题
DOI：
10.48550/arxiv.2306.03534
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Itay Evron;E. Moroshko;G. Buzaglo;M. Khriesh;B. Marjieh;N. Srebro;Daniel Soudry
中科院分区：
文献类型：
--
作者：
Itay Evron;E. Moroshko;G. Buzaglo;M. Khriesh;B. Marjieh;N. Srebro;Daniel Soudry
我们分析了一组具有二元标签的可分离线性分类任务的连续学习。我们从理论上证明了弱正则化学习可以简化为解决一个序列最大边界问题，对应于凸集投影（POCS）框架的一个特殊情况。然后，我们在各种循环任务的设置下，包括循环任务和随机任务的顺序，开发了遗忘和其他兴趣量的上限。我们讨论了一些流行的训练实践的实际意义，如正则化调度和加权。我们指出了我们的连续分类设置和最近研究的连续回归设置之间的几个理论差异。
We analyze continual learning on a sequence of separable linear classification tasks with binary labels. We show theoretically that learning with weak regularization reduces to solving a sequential max-margin problem, corresponding to a special case of the Projection Onto Convex Sets (POCS) framework. We then develop upper bounds on the forgetting and other quantities of interest under various settings with recurring tasks, including cyclic and random orderings of tasks. We discuss several practical implications to popular training practices like regularization scheduling and weighting. We point out several theoretical differences between our continual classification setting and a recently studied continual regression setting.