CPS: SMALL: Formal Methods for Safe, Efficient, and Transferable Learning-enabled Autonomy

深度强化学习(RL)已成为控制具有高度非线性、随机性和未知性的网络物理系统(CP)的重要工具。然而，我们目前对RL何时工作、如何工作以及为什么工作缺乏了解，因此有必要为由RL控制器驱动的安全关键CP提供新的综合和分析工具；这是本项目的主要范围。本研究的主要焦点是移动机器人系统。这类CP通常由RL控制器驱动，因为它们固有的复杂--可能是不确定/未知--动态、未知的外部干扰或实时决策的需要。通常，基于RL的控制设计方法是数据低效的，它们不能安全地转移到新的任务和安全要求或新的环境中，而且它们往往缺乏性能保证。这项研究旨在解决这些局限性，从而为具有RL控制器的CP提供一种新的安全自主范例。广泛使用开发的自动驾驶方法可以使CP的安全关键型应用程序在环境监测、基础设施检查、自动驾驶和医疗保健等方面产生重大社会影响。为了达到安全、高效、可迁移的研究目标，本研究追求三个紧耦合的研究方向：(I)用于时态逻辑控制目标的加速和安全强化学习；(Ii)用于时态逻辑控制目标的安全迁移学习；(Iii)具有神经网络控制器的CPS的时态逻辑性质的成分验证。这些推进中的技术方法依赖于从形式方法、机器学习和控制理论中提取的工具，并需要克服与计算、控制和传感集成相关的智力挑战。开发的自主方法将在移动空中和地面机器人上进行验证和演示，执行自主监视、交付和移动操作任务。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep Reinforcement Learning (RL) has emerged as prominent tool to control cyber-physical systems (CPS) with highly non-linear, stochastic, and unknown dynamics. Nevertheless, our current lack of understanding of when, how, and why RL works necessitates the need for new synthesis and analysis tools for safety-critical CPS driven by RL controllers; this is the main scope of this project. The primary focus of this research is on mobile robot systems. Such CPS are often driven by RL controllers due to their inherent complex - and possibly uncertain/unknown - dynamics, unknown exogenous disturbances, or the need for real-time decision making. Typically, RL-based control design methods are data inefficient, they cannot be safely transferred to new mission & safety requirements or new environments, while they often lack performance guarantees. This research aims to address these limitations resulting in a novel paradigm in safe autonomy for CPS with RL controllers. Wide availability of the developed autonomy methods can enable safety-critical applications for CPS with significant societal impact on, e.g., environmental monitoring, infrastructure inspection, autonomous driving, and healthcare. The broader impacts of this research include its educational agenda involving K-12, undergraduate and graduate level education.To achieve the research goal of safe, efficient, and transferable RL, three tightly coupled research thrusts are pursued: (i) accelerated & safe reinforcement learning for temporal logic control objectives; (ii) safe transfer learning for temporal logic control objectives; (iii) compositional verification of temporal logic properties for CPS with NN controllers. The technical approach in these thrusts relies on tools drawn from formal methods, machine learning, and control theory and requires overcoming intellectual challenges related to integration of computation, control, and sensing. The developed autonomy methods will be validated and demonstrated on mobile aerial and ground robots in autonomous surveillance, delivery, and mobile manipulation tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.