Quick continual kernel learning on bounded memory space based on balancing between adaptation and forgetting
Quick continual kernel learning on bounded memory space based on balancing between adaptation and forgetting
复制标题
DOI：
10.1007/s12530-022-09476-8
复制
发表时间：
2022-12
期刊：
影响因子：
3.2
通讯作者：
K. Yamauchi
中科院分区：
文献类型：
--
作者：
K. Yamauchi
伴随着人工智能的发展，小型计算机系统被要求既执行推理又具有快速增量学习能力。这可以使用一类少次学习方法来实现，该方法包括预定的深度特征提取器，然后是传统的基于实例（最近邻）的学习器。然而，当任务必须在有限存储空间的微型计算机上执行时，其学习能力受到存储容量的限制。我们提出了一种新的基于投影的快速增量学习方法的Nadaraya-Watson内核回归模型作为基于实例的学习器，利用嵌入式系统中的轻量级计算。由于预算增量学习中存储容量的限制，系统中的核数是有界的，当达到上界时，算法修剪部分核以腾出空间记录新的实例。然而，内核修剪会导致系统忘记某些方面。因此，当前实例的遗忘和调整之间的适当平衡对于获得更高的泛化能力是重要的。因此，我们还提出了一种新的方法来获得最佳平衡的类Nadaraya-Watson核回归模型，而不参考过去给定的实例，通过预测遗忘的大小，由于学习和控制学习率的核回归模型。实验结果表明，该方法是上级的性能与现有的最好的原型选择方法，评估他们的结果与所有给定的学习样本与其他连续的基于核的学习方法。
Concomitant with developments in artificial intelligence, small computer systems are being required to both perform reasoning and have quick incremental learning abilities. This can be achieved using a class of few-shot learning methods comprising a predetermined deep-feature extractor followed by a traditional instance-based (nearest neighbor) learner. However, when tasks must be performed on a limited-memory-space microcomputer, its learning ability is constrained by the memory capacity. We propose a new projection-based quick incremental learning method for the Nadaraya–Watson kernel regression model as the instance-based learner utilizing a lightweight calculation in embedded systems. Owing to the limitation of storage capacity in incremental learning on a budget, the number of kernels in the system is bounded; when the upper bound is reached, the algorithm prunes part of the kernels to make space for recording a new instance. However, kernel pruning causes the system to forget certain aspects. Therefore, the appropriate balance between forgetting and adjustment of the current instance is important for obtaining a higher generalization capability. Thus, we also propose a novel method to obtain the optimum balance for the class of Nadaraya–Watson kernel regression models without referring to past given instances by predicting the magnitude of forgetting due to learning and by controlling the learning ratio in the kernel regression model. Experimental results show that the proposed method is superior to other continual kernel-based learning methods with performance on par with the best existing prototype-selection methods, which evaluate their results with all given learning samples.