Safe Online Convex Optimization with Unknown Linear Safety Constraints
Safe Online Convex Optimization with Unknown Linear Safety Constraints
复制标题
DOI：
10.1609/aaai.v36i6.20566
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Sapana Chaudhary;D. Kalathil
中科院分区：
文献类型：
--
作者：
Sapana Chaudhary;D. Kalathil
研究了安全在线凸优化问题，其中每个时间步的行动必须满足一组线性安全约束。我们的目标是选择一系列的行动，以尽量减少遗憾，而不违反安全约束在任何时间步骤（高概率）。指定线性安全约束的参数对于算法是未知的。该算法只能访问所选动作的约束的噪声观测。我们提出了一种算法，称为安全在线投影梯度下降（SO-PGD）算法，来解决这个问题。我们表明，在安全基线动作可用性的假设下，SO-PGD算法实现了后悔O（T^{2/3}）。虽然有许多算法的在线凸优化（OCO）的安全约束问题在文献中，他们允许在学习/优化过程中的约束违反，并一直专注于特征的累积约束违反。据我们所知，我们的工作是第一次提供了一个算法与可证明的保证遗憾，而不违反线性安全约束（高概率）在任何时间步。
We study the problem of safe online convex optimization, where the action at each time step must satisfy a set of linear safety constraints. The goal is to select a sequence of actions to minimize the regret without violating the safety constraints at any time step (with high probability). The parameters that specify the linear safety constraints are unknown to the algorithm. The algorithm has access to only the noisy observations of constraints for the chosen actions. We propose an algorithm, called the Safe Online Projected Gradient Descent (SO-PGD) algorithm, to address this problem. We show that, under the assumption of availability of a safe baseline action, the SO-PGD algorithm achieves a regret O(T^{2/3}). While there are many algorithms for online convex optimization (OCO) problems with safety constraints available in the literature, they allow constraint violations during learning/optimization, and the focus has been on characterizing the cumulative constraint violations. To the best of our knowledge, ours is the first work that provides an algorithm with provable guarantees on the regret, without violating the linear safety constraints (with high probability) at any time step.