SpicyMKL: a fast algorithm for Multiple Kernel Learning with thousands of kernels
SpicyMKL: a fast algorithm for Multiple Kernel Learning with thousands of kernels
复制标题
DOI：
10.1007/s10994-011-5252-9
复制
发表时间：
2011-06
期刊：
影响因子：
7.5
通讯作者：
Taiji Suzuki;Ryota Tomioka
中科院分区：
文献类型：
--
作者：
Taiji Suzuki;Ryota Tomioka
我们提出了一种新的多核学习（MKL）优化算法，称为 SpicyMKL，它适用于一般凸损失函数和一般类型的正则化。所提出的 SpicyMKL 迭代地解决了平滑最小化问题。因此，不需要在内部求解 SVM、LP 或 QP。 SpicyMKL 可以被视为一种近端最小化方法，并且超线性收敛。内部最小化的成本大致与活动内核的数量成正比。因此，当我们的目标是稀疏内核组合时，我们的算法可以很好地适应内核数量的增加。此外，我们给出了 MKL 的通用块范数公式，其中包括非稀疏正则化，例如弹性网络和ℓp-范数正则化。扩展 SpicyMKL，我们提出了一种针对通用正则化框架的有效优化方法。实验结果表明，我们的算法比现有方法更快，特别是当内核数量很大（> 1000）时。
We propose a new optimization algorithm for Multiple Kernel Learning (MKL) called SpicyMKL, which is applicable to general convex loss functions and general types of regularization. The proposed SpicyMKL iteratively solves smooth minimization problems. Thus, there is no need of solving SVM, LP, or QP internally. SpicyMKL can be viewed as a proximal minimization method and converges super-linearly. The cost of inner minimization is roughly proportional to the number of active kernels. Therefore, when we aim for a sparse kernel combination, our algorithm scales well against increasing number of kernels. Moreover, we give a general block-norm formulation of MKL that includes non-sparse regularizations, such as elastic-net andℓp-norm regularizations. Extending SpicyMKL, we propose an efficient optimization method for the general regularization framework. Experimental results show that our algorithm is faster than existing methods especially when the number of kernels is large (>1000).