Fast convergent algorithms for multi-kernel regression
Fast convergent algorithms for multi-kernel regression
复制标题
多核回归的快速收敛算法
DOI：
10.1109/ssp.2016.7551736
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
G. Giannakis
中科院分区：
文献类型：
--
作者：
Liang Zhang;Daniel Romero;G. Giannakis
核脊回归在各种信号处理和机器学习应用中起着核心作用。在正则化约束下，通过优化准则选择合适的核作为“基核”的线性组合。尽管这些方法提供了可靠的泛化性能，但解决相关的最小-最大优化问题面临着重大挑战，特别是在大数据输入的情况下。在分析了凸重公式的关键性质后，提出了一种基于Nesterov加速法的高效算法，该算法在一阶方法之间实现了最优的阶收敛速率。封闭形式的更新是为通用正则表达式派生的。在真实数据集上的实验证实了与竞争算法相比具有相当大的加速优势。
Kernel ridge regression plays a central role in various signal processing and machine learning applications. Suitable kernels are often chosen as linear combinations of “basis kernels” by optimizing criteria under regularization constraints. Although such approaches offer reliable generalization performance, solving the associated min-max optimization problems face major challenges, especially with big data inputs. After analyzing the key properties of a convex reformulation, the present paper introduces an efficient algorithm based on a generalization of Nesterov's acceleration method, which achieves order-optimal convergence rate among first-order methods. Closed-form updates are derived for common regularizers. Experiments on real datasets corroborate considerable speedup advantages over competing algorithms.