Bayesian Optimization with Conformal Prediction Sets
Bayesian Optimization with Conformal Prediction Sets
复制标题
DOI：
--
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
S. Stanton;Wesley J. Maddox;A. Wilson
中科院分区：
文献类型：
--
作者：
S. Stanton;Wesley J. Maddox;A. Wilson
贝叶斯优化（Bayesian optimization）是一种在不确定性下进行决策的一致、普遍的方法，其应用包括多臂强盗、主动学习和黑盒优化。贝叶斯优化选择决策（即目标函数查询）与最大的预期效用相对于贝叶斯模型的后验分布，量化可减少的，认知的不确定性查询结果。在实践中，主观上不可信的结果可能会经常发生，原因有两个：1）模型错误设定和2）协变量偏移。保形预测是一种不确定性量化方法，即使对于错误指定的模型也具有覆盖保证，并且是校正协变量偏移的简单机制。我们提出了共形贝叶斯优化，它将查询引导到搜索空间的区域，其中模型预测具有保证的有效性，并研究其在一套黑盒优化任务和表格排名任务上的行为。在许多情况下，我们发现查询覆盖率可以显着提高，而不会损害样本效率。
Bayesian optimization is a coherent, ubiquitous approach to decision-making under uncertainty, with applications including multi-arm bandits, active learning, and black-box optimization. Bayesian optimization selects decisions (i.e. objective function queries) with maximal expected utility with respect to the posterior distribution of a Bayesian model, which quantifies reducible, epistemic uncertainty about query outcomes. In practice, subjectively implausible outcomes can occur regularly for two reasons: 1) model misspecification and 2) covariate shift. Conformal prediction is an uncertainty quantification method with coverage guarantees even for misspecified models and a simple mechanism to correct for covariate shift. We propose conformal Bayesian optimization, which directs queries towards regions of search space where the model predictions have guaranteed validity, and investigate its behavior on a suite of black-box optimization tasks and tabular ranking tasks. In many cases we find that query coverage can be significantly improved without harming sample-efficiency.