Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
复制标题
DOI：
--
复制
发表时间：
2018-06
期刊：
影响因子：
--
通讯作者：
Marton Havasi;José Miguel Hernández-Lobato;J. J. Murillo-Fuentes-J.
中科院分区：
文献类型：
--
作者：
Marton Havasi;José Miguel Hernández-Lobato;J. J. Murillo-Fuentes-J.
深度高斯过程（DGP）是高斯过程的分层概括，其将良好校准的不确定性估计与多层模型的高度灵活性结合联合收割机。这些模型面临的最大挑战之一是精确的推理是棘手的。目前最先进的推理方法，变分推理（VI），采用高斯近似的后验分布。这可能是通常多模态后验的潜在不良单峰近似。在这项工作中，我们提供的非高斯性质的后验的证据，我们应用随机梯度哈密顿蒙特卡罗方法来生成样本。为了有效地优化超参数，我们引入了移动窗口MCEM算法。这导致在一个更低的计算成本比VI对应的显着更好的预测。因此，我们的方法建立了一个新的国家的最先进的DGP推理。
Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian Processes that combine well calibrated uncertainty estimates with the high flexibility of multilayer models. One of the biggest challenges with these models is that exact inference is intractable. The current state-of-the-art inference method, Variational Inference (VI), employs a Gaussian approximation to the posterior distribution. This can be a potentially poor unimodal approximation of the generally multimodal posterior. In this work, we provide evidence for the non-Gaussian nature of the posterior and we apply the Stochastic Gradient Hamiltonian Monte Carlo method to generate samples. To efficiently optimize the hyperparameters, we introduce the Moving Window MCEM algorithm. This results in significantly better predictions at a lower computational cost than its VI counterpart. Thus our method establishes a new state-of-the-art for inference in DGPs.