Distributionally Robust Adaptive Control: Enabling Safe and Robust Reinforcement Learning

数据驱动算法可以自主控制自动驾驶汽车和无人机等复杂系统。然而，这种强大算法的使用仍然主要局限于受控的实验室环境。对于安全关键型系统，很少采用数据驱动方法的主要原因是，在试图像使用成熟的控制理论方法那样建立安全性和可预测性保证时遇到的困难。该奖项支持基础研究，以确定整合数据驱动和控制理论工具的最佳方法，从而使整体方法安全、稳健和高性能。新方法将控制工具提升为与数据驱动方法使用相同的语言。在这样做时，数据驱动方法的性能不会受到损害，但是，可以构建控制理论工具的安全保证。复杂系统的安全、可预测的自主操作可以通过其在医疗机器人、自主物流、运输和地外探索等领域的应用带来巨大的社会经济效益。这项研究涉及多个学科，包括机器人、控制理论、统计学习和数学。跨学科的性质将有助于代表性不足的群体更广泛地参与STEM和影响工程教育。为了在安全关键型系统中采用依赖于强化学习（RL）算法的数据驱动方法，我们需要保证安全性和鲁棒性。为具有参数不确定性的经典系统开发的鲁棒和自适应控制方法不能直接与RL结合使用，因为后者在数据驱动的模型上操作，对于这些模型，识别参数和确定性不确定性是困难的，如果不是不可能的话。本研究将构建一类新的鲁棒自适应控制器，该控制器对学习分布中的误差具有鲁棒性，从而允许强化学习算法直接与这些控制器交互，而无需进一步的限制。由于分布级别的鲁棒性，可以以一种直接的方式包含风险感知安全性的概念。本研究将首先致力于构建具有统一边界的跟踪随时间变化的状态分布的控制器。然后，引入认知不确定性，提出一种新的自适应控制方案，定量控制不确定性对分布空间的影响。通过这种努力产生的结果将把数据驱动控制和经典控制这两个截然不同的世界结合在一起，在一个自然的交叉点上，分布的轨迹，而不是样本路径，被考虑。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data-driven algorithms can autonomously control complex systems like autonomous cars and drones. However, the use of such powerful algorithms remains relegated primarily to controlled laboratory environments. The main reason for the minimal adoption of data-driven methods for safety-critical systems is the difficulty one encounters when attempting to establish safety and predictability guarantees as one would do with well-established control theoretical methods. This award supports fundamental research to identify the best methodologies to consolidate data-driven and control-theoretic tools so that the overall methodology is safe, robust, and high-performing. The new approach lifts control tools to speak the same language as the data-driven methods. In doing so, the performance of the data-driven methods is not compromised, and yet, the safety guarantees of control-theoretic tools can be constructed. Safe and predictable autonomous operation of complex systems can bring immense socio-economic benefits through its application in medical robotics, autonomous logistics, transportation, and extra-terrestrial exploration, to name a few. This research involves multiple disciplines, including robotics, control theory, statistical learning, and mathematics. The cross-disciplinary nature will assist underrepresented groups' broader participation in STEM and impact engineering education. To adopt data-driven methods that rely on reinforcement learning (RL) algorithms in safety-critical systems, we need guarantees on safety and robustness. Robust and adaptive control methodologies developed for classical systems with parametric uncertainties cannot be used directly in conjunction with RL because the latter operates on data-driven models for which identifying parametric and deterministic uncertainties is difficult, if not impossible. This research will construct a new class of robust adaptive controllers that are robust to errors in the learned distributions, thus allowing RL algorithms to directly interact with these controllers without further restrictions. Due to robustness at the level of distributions, notions of risk-aware safety can be included in a straightforward manner. This research will first aim to construct controllers that track temporally evolving state distributions with uniform bounds. Then, the epistemic uncertainties will be introduced with a novel adaptive control scheme to quantifiably control the effect of the uncertainties in the space of distributions. The results produced through this effort will bring the two distinct worlds of data-driven control and classical control together at a natural intersection point where trajectories of distributions, not of sample paths, are considered.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.