Collaborative Research: SLES: Foundations of Qualitative and Quantitative Safety Assessment of Learning-enabled Systems

在不熟悉或前所未有的环境中运行的自主学习系统为其安全评估和后续风险管理带来了新的基础挑战。在这种情况下，系统级安全意味着多个学习组件与物理世界之间的交互产生的复杂行为满足安全要求，保护系统免受意外故障的影响，以避免与其他车辆、自行车和行人发生碰撞等危险。定性和定量方法设想通过提供“是”或“否”的二元决策和安全的数值测量来相互补充，这允许彻底了解安全问题，并在不确定的环境中进行有效的安全验证。该项目针对开发定性和定量安全评估方法的基本挑战，这些方法能够捕捉环境中的不确定性，并在系统层面提供及时、全面和准确的安全评估。研究结果有望提高学习型系统对未知世界的可信度和适应性，并促进它们安全地集成到自动驾驶汽车、机器人或工业自动化等各个领域。教育和推广活动很好地融入了研究，包括课程开发，K-12 STEM推广和工业参与活动。通过优先考虑、指导和与代表性不足的少数群体的学生合作，设计的活动具有独特的定位，以促进整个项目的多样性。拟议的研究工作将针对建立端到端的定性和定量安全评估的基础学习自主系统。本项目将开发概率星型时序逻辑规范语言。新的规范语言为学习过程的不确定性和复杂的时间行为的表达建模提供了一种形式，并支持定性和定量推理。将开发有效的计算方法和工具来验证支持学习的深度神经网络组件的概率星型时间逻辑规范。验证方法和工具以增强其可扩展性和资源效率为中心。该项目将开发系统级定性和定量安全评估方法和工具，这些方法和工具可以处理在不同环境信息可用性下系统中各种具有学习能力的组件的相互作用。f110th测试平台是一个小规模的真实自动驾驶车辆及其模拟器系统，将用于创建多个真实世界的自动驾驶场景，以验证和评估所提出的方法和工具的适用性、可扩展性和可靠性。这项研究得到了美国国家科学基金会和开放慈善机构的合作支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Learning-enabled autonomous systems operating in unfamiliar or unprecedented environments pose new foundational challenges for their safety assessment and subsequent risk management. In this context, the system-level safety means the complicated behaviors created by the interactions between multiple learning components and the physical world satisfy the safety requirements, protecting the system from accidental failures to avoid hazards such as collisions to other vehicles, bicycles and pedestrians. The qualitative and quantitative methodologies envisioned to complement each other by providing both 'yes' or 'no' binary decisions and numerical measures of safety, which allow for a thorough understanding of safety concerns and enable effective safety verification in uncertain environments. This project targets the foundational challenges of developing qualitative and quantitative safety assessment methods capable of capturing uncertainties from environments and providing timely, comprehensive, and accurate safety evaluations at the system level. The outcomes are expected to boost the trustworthiness and adaptability of learning-enabled systems to the unknown world and facilitate their safe integration into various domains, such as autonomous vehicles, robotics, or industrial automation. Educational and outreach activities are well-integrated into the research, including curriculum development, K-12 STEM outreach, and industrial engagement activities. The designed activities are uniquely positioned to promote diversity throughout this project by giving priority consideration, mentoring, and working with students in underrepresented minority groups. The proposed research efforts will be directed toward building the foundations of end-to-end qualitative and quantitative safety assessment of learning-enabled autonomous systems. This project will develop the probabilistic star temporal logic specification language. The new specification language offers a formalism for expressive modeling of learning process uncertainty and complex temporal behaviors, and supports both qualitative and quantitative reasoning. Efficient computation methods and tools will be developed to verify probabilistic star temporal logic specifications for learning-enabled deep neural network components. The verification methods and tools are centered on enhancing their scalability and resource efficiency. This project will develop system-level qualitative and quantitative safety assessment methods and tools that can handle the interplay of various learning-enabled components in a system under different availability of environment information. Learning-enabled F1Tenth testbed, a small-scale system of real autonomous vehicles and its simulator, will be used to create multiple real-world autonomous driving scenarios to validate and evaluate the applicability, scalability and reliability of the proposed methods and tools.This research is supported by a partnership between the National Science Foundation and Open Philanthropy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.