An Adaptive Approach to Learning Optimal Neighborhood Kernels
An Adaptive Approach to Learning Optimal Neighborhood Kernels
复制标题
DOI：
10.1109/tsmcb.2012.2207889
复制
发表时间：
2013-02
影响因子：
11.8
通讯作者：
Xinwang Liu;Jianping Yin;Lei Wang;Lingqiao Liu;Jun Liu;Chenping Hou;Jian Zhang
中科院分区：
文献类型：
--
作者：
Xinwang Liu;Jianping Yin;Lei Wang;Lingqiao Liu;Jun Liu;Chenping Hou;Jian Zhang
学习最优核在基于核的方法中起着至关重要的作用。最近，一种称为最优邻域核学习(ONKL)的方法被提出，并显示出良好的分类性能。它假设最优内核位于“预先指定的”内核附近。然而，如何以原则性的方式指定这样一个核心仍然不清楚。为了解决这一问题，本文将预先指定的核作为一个额外变量，与最优邻域核和支持向量机的结构参数联合学习。为了避免无关紧要的解，我们用一个参数化的模型约束预先指定的核。我们首先讨论我们方法的特点，并特别强调它的适应性。然后，通过将预先指定的核分别建模为普通高斯径向基函数核和多核学习(MKL)方式的基核的线性组合，给出了两个实例。我们证明了我们方法中的优化问题是一个极小极大问题，并且可以用扩展水平法和内斯特罗夫法有效地求解。此外，我们对我们的方法进行了概率解释，并将其应用于解释现有的核学习方法，为它们的共性和差异提供了另一种视角。在13个UCI数据集和另外两个真实数据集上的综合实验结果表明，通过联合学习过程，我们的方法不仅能够自适应地识别预先指定的核，而且获得了优于原始ONKL和相关MKL算法的分类性能。
Learning an optimal kernel plays a pivotal role in kernel-based methods. Recently, an approach called optimal neighborhood kernel learning (ONKL) has been proposed, showing promising classification performance. It assumes that the optimal kernel will reside in the neighborhood of a “pre-specified” kernel. Nevertheless, how to specify such a kernel in a principled way remains unclear. To solve this issue, this paper treats the pre-specified kernel as an extra variable and jointly learns it with the optimal neighborhood kernel and the structure parameters of support vector machines. To avoid trivial solutions, we constrain the pre-specified kernel with a parameterized model. We first discuss the characteristics of our approach and in particular highlight its adaptivity. After that, two instantiations are demonstrated by modeling the pre-specified kernel as a common Gaussian radial basis function kernel and a linear combination of a set of base kernels in the way of multiple kernel learning (MKL), respectively. We show that the optimization in our approach is a min-max problem and can be efficiently solved by employing the extended level method and Nesterov's method. Also, we give the probabilistic interpretation for our approach and apply it to explain the existing kernel learning methods, providing another perspective for their commonness and differences. Comprehensive experimental results on 13 UCI data sets and another two real-world data sets show that via the joint learning process, our approach not only adaptively identifies the pre-specified kernel, but also achieves superior classification performance to the original ONKL and the related MKL algorithms.