Distributionally Robust Adaptive Control: Enabling Safe and Robust Reinforcement Learning

数据驱动的算法可以自主控制自动驾驶汽车和无人机等复杂系统。然而，这种强大的算法的使用仍然主要局限于受控的实验室环境。对安全关键系统采用数据驱动方法最少的主要原因是，当人们试图建立安全和可预测性保证时遇到困难，就像人们使用成熟的控制理论方法所做的那样。该奖项支持基础研究，以确定整合数据驱动和控制理论工具的最佳方法，从而使总体方法安全、可靠和高性能。新方法提升了控制工具，使其与数据驱动方法具有相同的语言。这样做，数据驱动方法的性能不会受到影响，但可以构建控制理论工具的安全保障。通过在医疗机器人、自主物流、运输和外星探索等领域的应用，复杂系统的安全和可预测的自主运行可以带来巨大的社会经济效益。这项研究涉及多个学科，包括机器人学、控制论、统计学习和数学。跨学科的性质将有助于代表不足的群体更广泛地参与STEM和影响工程教育。为了在安全关键系统中采用依赖于强化学习(RL)算法的数据驱动方法，我们需要安全和健壮性的保证。为具有参数不确定性的经典系统开发的鲁棒和自适应控制方法不能直接与RL结合使用，因为RL处理的是数据驱动模型，对于这些模型，识别参数和确定性不确定性是困难的，如果不是不可能的话。这项研究将构造一类新的鲁棒自适应控制器，它们对学习分布中的错误具有鲁棒性，从而允许RL算法直接与这些控制器交互，而不需要进一步的限制。由于分布级别的健壮性，风险感知安全的概念可以以一种简单的方式包含在内。这项研究首先旨在构造控制器，以一致的界跟踪时间演变的状态分布。然后，将认知不确定性引入一种新的自适应控制方案，以定量地控制分布空间中不确定性的影响。通过这一努力产生的结果将在一个自然的交叉点将数据驱动控制和经典控制这两个截然不同的世界结合在一起，在这个交叉点上，考虑的是分布的轨迹，而不是样本路径的轨迹。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Data-driven algorithms can autonomously control complex systems like autonomous cars and drones. However, the use of such powerful algorithms remains relegated primarily to controlled laboratory environments. The main reason for the minimal adoption of data-driven methods for safety-critical systems is the difficulty one encounters when attempting to establish safety and predictability guarantees as one would do with well-established control theoretical methods. This award supports fundamental research to identify the best methodologies to consolidate data-driven and control-theoretic tools so that the overall methodology is safe, robust, and high-performing. The new approach lifts control tools to speak the same language as the data-driven methods. In doing so, the performance of the data-driven methods is not compromised, and yet, the safety guarantees of control-theoretic tools can be constructed. Safe and predictable autonomous operation of complex systems can bring immense socio-economic benefits through its application in medical robotics, autonomous logistics, transportation, and extra-terrestrial exploration, to name a few. This research involves multiple disciplines, including robotics, control theory, statistical learning, and mathematics. The cross-disciplinary nature will assist underrepresented groups' broader participation in STEM and impact engineering education. To adopt data-driven methods that rely on reinforcement learning (RL) algorithms in safety-critical systems, we need guarantees on safety and robustness. Robust and adaptive control methodologies developed for classical systems with parametric uncertainties cannot be used directly in conjunction with RL because the latter operates on data-driven models for which identifying parametric and deterministic uncertainties is difficult, if not impossible. This research will construct a new class of robust adaptive controllers that are robust to errors in the learned distributions, thus allowing RL algorithms to directly interact with these controllers without further restrictions. Due to robustness at the level of distributions, notions of risk-aware safety can be included in a straightforward manner. This research will first aim to construct controllers that track temporally evolving state distributions with uniform bounds. Then, the epistemic uncertainties will be introduced with a novel adaptive control scheme to quantifiably control the effect of the uncertainties in the space of distributions. The results produced through this effort will bring the two distinct worlds of data-driven control and classical control together at a natural intersection point where trajectories of distributions, not of sample paths, are considered.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.