Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions
Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions
复制标题
DOI：
10.15607/rss.2020.xvi.088
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Jason J. Choi;F. Castañeda;C. Tomlin;K. Sreenath
中科院分区：
文献类型：
--
作者：
Jason J. Choi;F. Castañeda;C. Tomlin;K. Sreenath
本文用数据驱动的方法解决了安全关键控制中的模型不确定性问题。为此，我们利用了基于标称模型的输入输出线性化控制器以及基于控制屏障函数和基于控制李亚普诺夫函数的二次规划(CBF-CLF-QP)的结构。具体地说，我们提出了一种新的强化学习框架，它学习了CBF和CLF约束中存在的模型不确定性，以及二次规划中的其他控制-仿射动态约束。将训练后的策略与基于名义模型的CBF-CLF-QP相结合，得到了基于强化学习的CBF-CLF-QP(RL-CBF-CLF-QP)，解决了安全约束中模型的不确定性问题。通过对欠驱动非线性两足机器人在随机间距踏板上行走的一步预览实验，验证了该方法的有效性，实现了模型不确定性下的稳定安全行走。
In this paper, the issue of model uncertainty in safety-critical control is addressed with a data-driven approach. For this purpose, we utilize the structure of an input-ouput linearization controller based on a nominal model along with a Control Barrier Function and Control Lyapunov Function based Quadratic Program (CBF-CLF-QP). Specifically, we propose a novel reinforcement learning framework which learns the model uncertainty present in the CBF and CLF constraints, as well as other control-affine dynamic constraints in the quadratic program. The trained policy is combined with the nominal model-based CBF-CLF-QP, resulting in the Reinforcement Learning-based CBF-CLF-QP (RL-CBF-CLF-QP), which addresses the problem of model uncertainty in the safety constraints. The performance of the proposed method is validated by testing it on an underactuated nonlinear bipedal robot walking on randomly spaced stepping stones with one step preview, obtaining stable and safe walking under model uncertainty.