Direct loss minimization for sparse Gaussian processes
Direct loss minimization for sparse Gaussian processes
复制标题
DOI：
--
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Yadi Wei;Rishit Sheth;R. Khardon
中科院分区：
文献类型：
--
作者：
Yadi Wei;Rishit Sheth;R. Khardon
高斯过程 (GP) 是一种有吸引力的机器学习贝叶斯模型，它将优雅的公式与模型灵活性和不确定性量化相结合。稀疏高斯过程 (sGP) 算法提供了一种近似解决方案，可以减轻 GP 的高计算复杂性，而变分近似是此类近似的当前最佳实践。最近的理论工作表明，另一种方法，即直接损失最小化（DLM），可以直接最小化预测损失，为算法的预期损失提供强有力的保证。在本文中，我们通过实验探索了这种方法。我们为 sGP 开发了 DLM 算法，并表明通过适当的超参数优化，它比变分方法有了显着的改进。特别是，针对对数损失优化 sGP 可以为回归、分类和计数预测提供更好的校准预测，针对平方损失优化 sGP 可以改善回归中的均方误差。
The Gaussian process (GP) is an attractive Bayesian model for machine learning which combines an elegant formulation with model flexibility and uncertainty quantification. Sparse Gaussian process (sGP) algorithms provide an approximate solution that mitigates the high computational complexity of GP and the variational approximation is the current best practice for such approximations. Recent theoretical work has shown that an alternative approach, direct loss minimization (DLM), which directly minimizes predictive loss, comes with strong guarantees on the expected loss of the algorithm. In this paper we explore this approach experimentally. We develop the DLM algorithm for sGP and show that with appropriate hyperparameter optimization it provides a significant improvement over the variational approach. In particular, optimizing sGP for log loss provides better calibrated predictions for regression, classification and count prediction, and optimizing sGP for square loss improves the mean square error in regression.