Incremental learning for ν-Support Vector Regression
Incremental learning for ν-Support Vector Regression
复制标题
DOI：
10.1016/j.neunet.2015.03.013
复制
发表时间：
2015-07-01
期刊：
影响因子：
7.8
通讯作者：
Li, Shuo
中科院分区：
文献类型：
--
作者：
Gu, Bin;Sheng, Victor S.;Li, Shuo
nu-支持向量回归（nu- support Vector Regression，简称nu- svr）是一种有效的回归学习算法，它利用参数nu来控制支持向量的个数和自动调节管的宽度。然而，与nu-支持向量分类（nu-SVC） （Scholkopf et al., 2000）相比，nu-SVR在其目标函数中引入了一个额外的线性项。因此，直接将精确在线nu-SVC算法（AONSVM）应用于nu-SVR将无法生成有效的初始解。设计一种增量式nu-SVR学习算法是当前的主要挑战。为了克服这一挑战，本文提出了一种称为初始调整的特殊程序。该过程根据KKT （Karush-Kuhn-Tucker）条件调整nu-SVC的权重，为增量学习准备初始解。将初始调整与AONSVM的两步相结合，得到了一种精确有效的增量式nu-SVR学习算法（INSVR）。理论分析证明了三个关键逆矩阵的存在性，它们分别是INSVR三步（包括初始调整）的基石。在基准数据集上的实验表明，insr能够尽可能地避免不可行的更新路径，并成功收敛到最优解。结果还表明，无论冷启动还是热启动，insr算法都比批处理nu-SVR算法更快。(C) 2015 Elsevier Ltd.版权所有。
The nu-Support Vector Regression (nu-SVR) is an effective regression learning algorithm, which has the advantage of using a parameter nu on controlling the number of support vectors and adjusting the width of the tube automatically. However, compared to nu-Support Vector Classification (nu-SVC) (Scholkopf et al., 2000), nu-SVR introduces an additional linear term into its objective function. Thus, directly applying the accurate on-line nu-SVC algorithm (AONSVM) to nu-SVR will not generate an effective initial solution. It is the main challenge to design an incremental nu-SVR learning algorithm. To overcome this challenge, we propose a special procedure called initial adjustments in this paper. This procedure adjusts the weights of nu-SVC based on the Karush-Kuhn-Tucker (KKT) conditions to prepare an initial solution for the incremental learning. Combining the initial adjustments with the two steps of AONSVM produces an exact and effective incremental nu-SVR learning algorithm (INSVR). Theoretical analysis has proven the existence of the three key inverse matrices, which are the cornerstones of the three steps of INSVR (including the initial adjustments), respectively. The experiments on benchmark datasets demonstrate that INSVR can avoid the infeasible updating paths as far as possible, and successfully converges to the optimal solution. The results also show that INSVR is faster than batch nu-SVR algorithms with both cold and warm starts. (C) 2015 Elsevier Ltd. All rights reserved.