Regret Bounds for Safe Gaussian Process Bandit Optimization
Regret Bounds for Safe Gaussian Process Bandit Optimization
复制标题
DOI：
10.1109/isit45174.2021.9518176
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Sanae Amani;M. Alizadeh;Christos Thrampoulidis
中科院分区：
文献类型：
--
作者：
Sanae Amani;M. Alizadeh;Christos Thrampoulidis
许多应用程序需要学习者做出顺序的决定，给出关于系统的支付函数和安全约束的不确定性。在安全关键系统中，学习者的行为在学习过程的任何阶段都不违反安全约束是至关重要的。本文研究了一个随机强盗优化问题，其中未知支付和约束函数都是从[1]中首次考虑的高斯过程（GP）中采样的。我们开发了一个安全的变种GP-UCB称为SGP-UCB，在每一轮的安全约束方面进行必要的修改。该算法有两个不同的阶段。第一阶段寻求估计决策集中的安全动作集合，而第二阶段遵循GP-UCB决策规则。我们的主要贡献是推导出这个问题的第一个次线性后悔界。我们数值比较SGP-UCB对现有的安全贝叶斯GP优化算法。
Many applications require a learner to make sequential decisions given uncertainty regarding both the system's payoff function and safety constraints. In safety-critical systems, it is paramount that the learner's actions do not violate the safety constraints at any stage of the learning process. In this paper, we study a stochastic bandit optimization problem where the unknown payoff and constraint functions are sampled from Gaussian Processes (GPs) first considered in [1]. We develop a safe variant of GP-UCB called SGP-UCB, with necessary modifications to respect safety constraints at every round. The algorithm has two distinct phases. The first phase seeks to estimate the set of safe actions in the decision set, while the second phase follows the GP-UCB decision rule. Our main contribution is to derive the first sub-linear regret bounds for this problem. We numerically compare SGP-UCB against existing safe Bayesian GP optimization algorithms.