Scaling Multidimensional Inference for Structured Gaussian Processes
Scaling Multidimensional Inference for Structured Gaussian Processes
复制标题
DOI：
10.1109/tpami.2013.192
复制
发表时间：
2015-02-01
影响因子：
23.6
通讯作者：
Cunningham, John P.
中科院分区：
文献类型：
--
作者：
Gilboa, Elad;Saatci, Yunus;Cunningham, John P.
精确高斯过程 (GP) 回归对于数据大小 N 来说具有 O(N-3) 运行时间，这使得它对于大 N 来说变得棘手。许多用于改进 GP 缩放的算法都使用较低秩矩阵来近似协方差。其他工作利用了特定协方差函数固有的结构，包括具有隐含马尔可夫结构的 GP，以及格子上的输入（两者都支持 O(N) 或 O(N log N) 运行时间）。然而，尽管多维应用占优势，这些 GP 的进步还没有很好地扩展到多维输入设置。本文介绍并测试了结构化 GP 对多维输入的三种新颖扩展，适用于具有加法和乘法核的模型。首先，我们提出了一种新的加法 GP 推理方法，展示了经典反向拟合方法和贝叶斯框架之间的新颖联系。我们利用两项进步扩展了该模型：投影追踪回归的变体和非高斯观测的拉普拉斯近似。最后，对于乘法核结构，我们提出了一种在多维网格上输入的 GP 的新方法。我们在多个数据集上展示了这三项进步的力量，以低几个数量级的成本实现了与朴素 GP 相同或非常接近的性能。
Exact Gaussian process (GP) regression has O(N-3) runtime for data size N, making it intractable for large N. Many algorithms for improving GP scaling approximate the covariance with lower rank matrices. Other work has exploited structure inherent in particular covariance functions, including GPs with implied Markov structure, and inputs on a lattice (both enable O(N) or O(N log N) runtime). However, these GP advances have not been well extended to the multidimensional input setting, despite the preponderance of multidimensional applications. This paper introduces and tests three novel extensions of structured GPs to multidimensional inputs, for models with additive and multiplicative kernels. First we present a new method for inference in additive GPs, showing a novel connection between the classic backfitting method and the Bayesian framework. We extend this model using two advances: a variant of projection pursuit regression, and a Laplace approximation for non-Gaussian observations. Lastly, for multiplicative kernel structure, we present a novel method for GPs with inputs on a multidimensional grid. We illustrate the power of these three advances on several data sets, achieving performance equal to or very close to the naive GP at orders of magnitude less cost.