Uncertainty quantification using martingales for misspecified Gaussian processes
Uncertainty quantification using martingales for misspecified Gaussian processes
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
W. Neiswanger;Aaditya Ramdas
中科院分区：
文献类型：
--
作者：
W. Neiswanger;Aaditya Ramdas
我们解决了在错误指定的先验下高斯过程（GPs）的不确定性量化，并着眼于贝叶斯优化（BO）。GPs由于易于实现基于后验不确定性带的勘探，在BO中得到了广泛的应用。然而，这种便利是以鲁棒性为代价的：在实践中遇到的典型函数不太可能是从数据科学家的先验中得出的，在这种情况下，不确定性估计可能会产生误导，并且最终的探索可能是次优的。这种脆性行为在简单的模拟中得到了令人信服的证明。提出了GP/BO不确定度量化的频率论方法。我们利用GP框架作为工作模型，但不假设先验的正确性。我们使用鞅技术构造未知函数的置信序列（CS）。实现鲁棒性有必要的代价：如果先验是正确的，后验GP带比我们的CS窄。然而，当先验错误时，我们的CS在统计上是有效的，并且在经验上优于标准GP方法，就BO的覆盖率和效用而言。此外，我们证明了动力似然提供了对模型错误规范的鲁棒性。
We address uncertainty quantification for Gaussian processes (GPs) under misspecified priors, with an eye towards Bayesian Optimization (BO). GPs are widely used in BO because they easily enable exploration based on posterior uncertainty bands. However, this convenience comes at the cost of robustness: a typical function encountered in practice is unlikely to have been drawn from the data scientist's prior, in which case uncertainty estimates can be misleading, and the resulting exploration can be suboptimal. This brittle behavior is convincingly demonstrated in simple simulations. We present a frequentist approach to GP/BO uncertainty quantification. We utilize the GP framework as a working model, but do not assume correctness of the prior. We instead construct a confidence sequence (CS) for the unknown function using martingale techniques. There is a necessary cost to achieving robustness: if the prior was correct, posterior GP bands are narrower than our CS. Nevertheless, when the prior is wrong, our CS is statistically valid and empirically outperforms standard GP methods, in terms of both coverage and utility for BO. Additionally, we demonstrate that powered likelihoods provide robustness against model misspecification.