Scalable Exact Inference in Multi-Output Gaussian Processes
Scalable Exact Inference in Multi-Output Gaussian Processes
复制标题
DOI：
--
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
W. Bruinsma;E. Perim;Will Tebbutt;J. Hosking;A. Solin;Richard E. Turner
中科院分区：
文献类型：
--
作者：
W. Bruinsma;E. Perim;Will Tebbutt;J. Hosking;A. Solin;Richard E. Turner
多输出高斯过程(MOGP)利用了GP的灵活性和可解释性，同时捕获了跨输出的结构，这在例如时空建模中是可取的。MOGP的关键问题是它们的计算比例$O(n^3 p^3)$，它在输入$n$(例如，时间点或位置)和输出$p$的数量上都是立方的。出于这个原因，一类流行的MOGP假设数据存在于低维线性子空间中，从而将复杂性降低到$O(n^3 m^3)$。然而，这一代价在子空间$m$的维度上仍然是立方的，对于许多应用来说，这仍然是令人望而却步的昂贵。我们提出利用足够的数据统计量来加速具有正交基的MOGP的推理和学习。该方法在实际中实现了$m$的线性缩放，允许这些模型在不牺牲显著的可表现性或要求近似的情况下扩展到较大的$m$。这一改进开放了广泛的现实世界任务，并可以以即插即用的方式与现有的GP近似相结合。我们在各种合成数据集和真实数据集上演示了该方法的有效性。
Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling. The key problem with MOGPs is their computational scaling $O(n^3 p^3)$, which is cubic in the number of both inputs $n$ (e.g., time points or locations) and outputs $p$. For this reason, a popular class of MOGPs assumes that the data live around a low-dimensional linear subspace, reducing the complexity to $O(n^3 m^3)$. However, this cost is still cubic in the dimensionality of the subspace $m$, which is still prohibitively expensive for many applications. We propose the use of a sufficient statistic of the data to accelerate inference and learning in MOGPs with orthogonal bases. The method achieves linear scaling in $m$ in practice, allowing these models to scale to large $m$ without sacrificing significant expressivity or requiring approximation. This advance opens up a wide range of real-world tasks and can be combined with existing GP approximations in a plug-and-play way. We demonstrate the efficacy of the method on various synthetic and real-world data sets.