On the Influence of the Kernel on the Consistency of Support Vector Machines
On the Influence of the Kernel on the Consistency of Support Vector Machines
复制标题
DOI：
10.1162/153244302760185252
复制
发表时间：
2002-03
期刊：
影响因子：
--
通讯作者：
Ingo Steinwart
中科院分区：
文献类型：
--
作者：
Ingo Steinwart
本文利用一类称为泛核的核，研究了几种支持向量机分类器的泛化能力。结果表明，只要选择适当的正则化参数，具有通用核的软边缘算法对于包括噪声任务在内的一大类分类问题是一致的。特别地，在高斯径向基函数核的情况下，我们得到了这个参数的一个简单的充分条件。一方面，我们的考虑是基于对所使用的核的逼近性质-所谓的普适性--的调查，该性质确保所有连续函数都可以用某些核表达式来逼近。这一近似性质也为内核在这些算法和其他算法中的作用提供了新的见解。另一方面，这些结果是通过对分类器的底层优化问题的精确研究来实现的。此外，我们还证明了最大边缘分类器和软边缘支持向量机在存在大边缘的情况下的一致性。在这种情况下，恒定的正则化参数也保证了软边距支持向量机的一致性。最后，我们证明了即使对于简单的无噪声分类问题，具有多项式核的支持向量机也可能表现得很差。
In this article we study the generalization abilities of several classifiers of support vector machine (SVM) type using a certain class of kernels that we call universal. It is shown that the soft margin algorithms with universal kernels are consistent for a large class of classification problems including some kind of noisy tasks provided that the regularization parameter is chosen well. In particular we derive a simple sufficient condition for this parameter in the case of Gaussian RBF kernels. On the one hand our considerations are based on an investigation of an approximation property---the so-called universality---of the used kernels that ensures that all continuous functions can be approximated by certain kernel expressions. This approximation property also gives a new insight into the role of kernels in these and other algorithms. On the other hand the results are achieved by a precise study of the underlying optimization problems of the classifiers. Furthermore, we show consistency for the maximal margin classifier as well as for the soft margin SVM's in the presence of large margins. In this case it turns out that also constant regularization parameters ensure consistency for the soft margin SVM's. Finally we prove that even for simple, noise free classification problems SVM's with polynomial kernels can behave arbitrarily badly.