Collaborative Research: SLES: Foundations of Qualitative and Quantitative Safety Assessment of Learning-enabled Systems

在陌生或前所未有的环境中运行的学习型自主系统为其安全评估和后续的风险管理提出了新的基础性挑战。在此背景下，系统级安全是指多个学习组件与物理世界相互作用产生的复杂行为满足安全要求，保护系统免受意外故障的影响，避免碰撞其他车辆、自行车和行人等危险。所设想的定性和定量方法相辅相成，提供“是”或“否”的二元安全决定和数字安全措施，从而能够透彻地了解安全问题，并能够在不确定的环境中进行有效的安全核查。该项目针对的是开发定性和定量安全评估方法的基本挑战，这些方法能够捕获环境中的不确定性，并在系统级别提供及时、全面和准确的安全评估。预计这一结果将提高学习系统对未知世界的可信度和适应性，并促进它们安全地集成到各个领域，如自动驾驶汽车、机器人或工业自动化。教育和外展活动被很好地整合到研究中，包括课程开发、K-12 STEM外展和产业参与活动。所设计的活动具有独特的定位，通过优先考虑、指导和与代表人数较少的少数群体的学生合作，在整个项目中促进多样性。拟议的研究工作将旨在为支持学习的自主系统的端到端定性和定量安全评估奠定基础。该项目将开发概率星形时态逻辑规范语言。新的规范语言为学习过程不确定性和复杂时间行为的表达建模提供了一种形式化方法，并支持定性和定量推理。将开发有效的计算方法和工具来验证支持学习的深度神经网络组件的概率星时序逻辑规范。验证方法和工具的核心是增强其可伸缩性和资源效率。该项目将开发系统一级的定性和定量安全评估方法和工具，这些方法和工具可以处理在不同环境信息可获得性下系统中各种可学习组成部分的相互作用。支持学习的F1Tten试验台是一个由真实自动驾驶车辆及其模拟器组成的小规模系统，将用于创建多个真实世界的自动驾驶场景，以验证和评估建议的方法和工具的适用性、可扩展性和可靠性。这项研究由国家科学基金会和开放慈善机构之间的合作伙伴关系支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Learning-enabled autonomous systems operating in unfamiliar or unprecedented environments pose new foundational challenges for their safety assessment and subsequent risk management. In this context, the system-level safety means the complicated behaviors created by the interactions between multiple learning components and the physical world satisfy the safety requirements, protecting the system from accidental failures to avoid hazards such as collisions to other vehicles, bicycles and pedestrians. The qualitative and quantitative methodologies envisioned to complement each other by providing both 'yes' or 'no' binary decisions and numerical measures of safety, which allow for a thorough understanding of safety concerns and enable effective safety verification in uncertain environments. This project targets the foundational challenges of developing qualitative and quantitative safety assessment methods capable of capturing uncertainties from environments and providing timely, comprehensive, and accurate safety evaluations at the system level. The outcomes are expected to boost the trustworthiness and adaptability of learning-enabled systems to the unknown world and facilitate their safe integration into various domains, such as autonomous vehicles, robotics, or industrial automation. Educational and outreach activities are well-integrated into the research, including curriculum development, K-12 STEM outreach, and industrial engagement activities. The designed activities are uniquely positioned to promote diversity throughout this project by giving priority consideration, mentoring, and working with students in underrepresented minority groups. The proposed research efforts will be directed toward building the foundations of end-to-end qualitative and quantitative safety assessment of learning-enabled autonomous systems. This project will develop the probabilistic star temporal logic specification language. The new specification language offers a formalism for expressive modeling of learning process uncertainty and complex temporal behaviors, and supports both qualitative and quantitative reasoning. Efficient computation methods and tools will be developed to verify probabilistic star temporal logic specifications for learning-enabled deep neural network components. The verification methods and tools are centered on enhancing their scalability and resource efficiency. This project will develop system-level qualitative and quantitative safety assessment methods and tools that can handle the interplay of various learning-enabled components in a system under different availability of environment information. Learning-enabled F1Tenth testbed, a small-scale system of real autonomous vehicles and its simulator, will be used to create multiple real-world autonomous driving scenarios to validate and evaluate the applicability, scalability and reliability of the proposed methods and tools.This research is supported by a partnership between the National Science Foundation and Open Philanthropy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.