Data-efficient Safe Control with Recovery-to-Optimality Guarantees

数据高效的安全控制，并保证恢复最佳性

• 批准号: 2227311
• 负责人: Bahare Kiumarsi
• 金额: $ 40万
• 依托单位: Michigan State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-15 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2227311&HistoricalAwards=false
• 关键词: Data; efficient; Safe; Control; Recovery

As rapid developments on learning-enabled systems in recent years have been advancing autonomy capabilities of systems, their safety certification becomes exceedingly important. While the recent progress on safe reinforcement learning (RL) algorithms for autonomous control design has been promising, these algorithms are accountable only in stable environments and under the availability of comprehensive and high-quality data sets. However, many systems must operate in unpredictable environments under which dangerous divergence might arise between safety and performance. In these environments, adaptation of safety and performance specifications to the context is required. Besides, RL agent must perform learning under realistic data quantity and quality. Current RL practice assumes availability of rich and high-quality data with full observability of the entire system’s states. These assumptions can be violated in many practical systems. This award supports research to create low-complexity safe learning-enabled algorithms for partially observable systems that are equipped with highly-efficient conflict management mechanisms to deliver as much performance as possible safely. Advances will have broad implications in applications of autonomous systems, robots, manufacturing, smart grids, and more.This research project aims to develop low-complexity, safe learning-enabled algorithms for partially observable systems equipped with highly efficient conflict management mechanisms. The objectives of this project are two-fold: 1) Proposing direct data-driven learning approaches for backup safe control policies in partially observable nonlinear systems with uncertain dynamics. The utilization of concepts such as L-extra sample dynamics, probabilistic contractivity, and convex lifting will enable the learning of safe control policies for nonlinear systems with nonconvex safe sets using only measured noisy input-output data. 2) Introducing novel merging approaches to proactively manage conflicts by merging learned backup safe control policies with learning-enabled control policies. Instead of providing reactive quick fixes to conflicts as they arise, these approaches will enable proactive conflict management to avoid destructive future conflicts. Towards conflict management, the level sets of the RL agent will be adapted to the situation to make the agent align with the safety constraint. That is, safety-shaped value functions will be learned to effectively resolve conflicts by considering safety and optimality concerns across the relevant domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着近年来对支持学习的系统的快速发展正在提高系统的自主能力，因此其安全认证变得越来越重要。虽然已承诺最近在安全加固学习（RL）算法上的进展，但这些算法仅在稳定的环境中以及在综合和高质量的数据集的可用性下才能负责。但是，许多系统必须在不可预测的环境中运行，在这种环境中，安全性和性能之间可能存在危险的差异。在这些环境中，需要将安全性和绩效规范适应上下文。此外，RL代理必须在现实的数据数量和质量下进行学习。当前的RL实践假设可以充分观察整个系统状态的富裕和高质量数据。这些假设在许多实用系统中可能会违反。该奖项支持研究，为部分可观察到的系统创建低复杂性安全学习算法，这些算法配备了高效的冲突管理机制，以提供尽可能多的性能。进步将对自主系统，机器人，制造，智能电网等的应用具有广泛的影响。本研究项目旨在为配备有高效的冲突管理机制的部分可观察到的部分可观察到的低复杂性，安全学习的算法。该项目的目标是两个方面：1）在具有不确定动力学的部分可观察到的非线性系统中，提出了直接数据驱动的学习方法，以备份安全控制策略。利用诸如L-Extra样本动力学，概率合同和凸起的概念，将使使用仅使用测量的噪声输入输出数据的非convex安全集的非线性系统来学习安全控制策略。 2）引入新颖的合并方法，通过将学习的备份安全控制政策与支持学习的控制政策合并，以主动管理冲突。这些方法不会在发生冲突的情况下提供反应性快速修复，而是使积极的冲突管理能够避免破坏性的未来冲突。在冲突管理方面，RL代理的级别集将适应情况，以使代理与安全限制保持一致。也就是说，将通过考虑在相关领域的安全性和最佳问题来有效地解决安全形的价值功能。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子和更广泛的影响审查标准来评估，被认为是宝贵的支持。