PAC-Bayesian generalisation error bounds for Gaussian process classification
PAC-Bayesian generalisation error bounds for Gaussian process classification
复制标题
DOI：
10.1162/153244303765208386
复制
发表时间：
2003-02-15
影响因子：
6
通讯作者：
Seeger, M
中科院分区：
文献类型：
--
作者：
Seeger, M
近似贝叶斯高斯过程（GP）分类技术是一种强大的非参数学习方法，在外观和性能上与支持向量机相似。基于简单的概率模型，它们呈现可解释的结果，并可以嵌入到贝叶斯框架中的模型选择，特征选择等，在本文中，通过应用PAC贝叶斯定理McAllester（1999年a），我们证明了分布自由泛化误差界广泛的近似贝叶斯CP分类技术。我们还为这个强大的定理提供了一个新的和更简化的证明，利用凸对偶的概念，这是许多机器学习技术的支柱。我们实例化和测试我们的边界为两个特定的GPC技术，包括最近的稀疏方法，它规避了不利的缩放标准GP算法。正如在真实世界任务的实验中所示，对于中等训练样本大小，界限可能非常紧。据我们所知，这些结果提供了最严格的近似贝叶斯GPC方法已知的分布自由误差界，给出了一个强有力的学习理论的理由，使用这些技术。
Approximate Bayesian Gaussian process (GP) classification techniques are powerful nonparametric learning methods, similar in appearance and performance to support vector machines. Based on simple probabilistic models, they render interpretable results and can be embedded in Bayesian frameworks for model selection, feature selection, etc. In this paper, by applying the PAC-Bayesian theorem of McAllester (1999a), we prove distribution-free generalisation error bounds for a wide range of approximate Bayesian CP classification techniques. We also provide a new and much simplified proof for this powerful theorem, making use of the concept of convex duality which is a backbone of many machine learning techniques. We instantiate and test our bounds for two particular GPC techniques, including a recent sparse method which circumvents the unfavourable scaling of standard GP algorithms. As is shown in experiments on a real-world task, the bounds can be very tight for moderate training sample sizes. To the best of our knowledge, these results provide the tightest known distribution-free error bounds for approximate Bayesian GPC methods, giving a strong learning-theoretical justification for the use of these techniques.