Nonlinear Regularization Path for Quadratic Loss Support Vector Machines
Nonlinear Regularization Path for Quadratic Loss Support Vector Machines
复制标题
DOI：
10.1109/tnn.2011.2164265
复制
发表时间：
2011-10
影响因子：
--
通讯作者：
Masayuki Karasuyama;I. Takeuchi
中科院分区：
文献类型：
--
作者：
Masayuki Karasuyama;I. Takeuchi
正则化路径算法已被提出来处理模型选择问题的几个机器学习方法。这些算法允许计算的整个路径的解决方案的正则化参数的每个值使用的事实，他们的解决方案路径具有分段线性形式。本文将正则化路径算法推广到一类具有二次损失和二次惩罚项的学习机。该类包含几种重要的学习机，如平方铰链损失支持向量机（SVM）和改进的Huber损失SVM。我们首先证明了这类学习机的解路径具有分段非线性形式，并且两个断点之间的分段由一类有理函数刻画。然后，我们开发了一个算法，可以有效地遵循分段非线性路径，通过解决这些合理的方程。为了求解这些有理方程，我们使用有理逼近技术与二次收敛速度，因此，我们的算法可以遵循非线性路径更精确比现有的方法，如预测-校正型非线性路径近似。我们展示了一些人工和真实的数据集上的算法性能。
Regularization path algorithms have been proposed to deal with model selection problem in several machine learning approaches. These algorithms allow computation of the entire path of solutions for every value of regularization parameter using the fact that their solution paths have piecewise linear form. In this paper, we extend the applicability of regularization path algorithm to a class of learning machines that have quadratic loss and quadratic penalty term. This class contains several important learning machines such as squared hinge loss support vector machine (SVM) and modified Huber loss SVM. We first show that the solution paths of this class of learning machines have piecewise nonlinear form, and piecewise segments between two breakpoints are characterized by a class of rational functions. Then we develop an algorithm that can efficiently follow the piecewise nonlinear path by solving these rational equations. To solve these rational equations, we use rational approximation technique with quadratic convergence rate, and thus, our algorithm can follow the nonlinear path much more precisely than existing approaches such as predictor-corrector type nonlinear-path approximation. We show the algorithm performance on some artificial and real data sets.