Collaborative Research: SLES: Foundations of Qualitative and Quantitative Safety Assessment of Learning-enabled Systems

在不熟悉或前所未有的环境中运行的学习型自主系统对其安全评估和后续风险管理提出了新的基本挑战。在这种情况下，系统级安全意味着多个学习组件与物理世界之间的交互所产生的复杂行为满足安全要求，保护系统免受意外故障的影响，以避免对其他车辆，自行车和行人的碰撞等危险。通过提供“是”或“否”的二元决策和数字安全措施，设想了相互补充的定性和定量方法，这允许对安全问题的全面理解，并在不确定的环境中进行有效的安全验证。该项目的目标是开发定性和定量安全评估方法的基本挑战，这些方法能够捕获环境中的不确定性，并在系统层面提供及时，全面和准确的安全评估。这些成果有望提高学习系统对未知世界的可信度和适应性，并促进其安全集成到各个领域，如自动驾驶汽车、机器人或工业自动化。教育和推广活动很好地融入了研究，包括课程开发，K-12 STEM推广和工业参与活动。所设计的活动具有独特的地位，通过优先考虑，指导和与代表性不足的少数群体的学生合作，促进整个项目的多样性。拟议的研究工作将致力于为学习型自主系统的端到端定性和定量安全评估奠定基础。本计画将发展机率型星星时序逻辑规范语言。新的规范语言提供了一个形式主义的学习过程的不确定性和复杂的时间行为的表达建模，并支持定性和定量推理。将开发有效的计算方法和工具，以验证支持学习的深度神经网络组件的概率星星时序逻辑规范。验证方法和工具的核心是提高其可扩展性和资源效率。该项目将开发系统一级的定性和定量安全评估方法和工具，以便在环境信息的不同可用性下处理系统中各种学习型组件的相互作用。F1 Tenth测试平台是一个小规模的真实的自动驾驶汽车及其模拟器系统，将用于创建多个真实的自动驾驶场景，以验证和评估其适用性，所提出的方法和工具的可扩展性和可靠性。这项研究得到了美国国家科学基金会和开放慈善机构之间的合作伙伴关系的支持。该奖项反映了NSF的法定使命，并被认为是值得的通过使用基金会的知识价值和更广泛的影响审查标准进行评估来提供支持。

Learning-enabled autonomous systems operating in unfamiliar or unprecedented environments pose new foundational challenges for their safety assessment and subsequent risk management. In this context, the system-level safety means the complicated behaviors created by the interactions between multiple learning components and the physical world satisfy the safety requirements, protecting the system from accidental failures to avoid hazards such as collisions to other vehicles, bicycles and pedestrians. The qualitative and quantitative methodologies envisioned to complement each other by providing both 'yes' or 'no' binary decisions and numerical measures of safety, which allow for a thorough understanding of safety concerns and enable effective safety verification in uncertain environments. This project targets the foundational challenges of developing qualitative and quantitative safety assessment methods capable of capturing uncertainties from environments and providing timely, comprehensive, and accurate safety evaluations at the system level. The outcomes are expected to boost the trustworthiness and adaptability of learning-enabled systems to the unknown world and facilitate their safe integration into various domains, such as autonomous vehicles, robotics, or industrial automation. Educational and outreach activities are well-integrated into the research, including curriculum development, K-12 STEM outreach, and industrial engagement activities. The designed activities are uniquely positioned to promote diversity throughout this project by giving priority consideration, mentoring, and working with students in underrepresented minority groups. The proposed research efforts will be directed toward building the foundations of end-to-end qualitative and quantitative safety assessment of learning-enabled autonomous systems. This project will develop the probabilistic star temporal logic specification language. The new specification language offers a formalism for expressive modeling of learning process uncertainty and complex temporal behaviors, and supports both qualitative and quantitative reasoning. Efficient computation methods and tools will be developed to verify probabilistic star temporal logic specifications for learning-enabled deep neural network components. The verification methods and tools are centered on enhancing their scalability and resource efficiency. This project will develop system-level qualitative and quantitative safety assessment methods and tools that can handle the interplay of various learning-enabled components in a system under different availability of environment information. Learning-enabled F1Tenth testbed, a small-scale system of real autonomous vehicles and its simulator, will be used to create multiple real-world autonomous driving scenarios to validate and evaluate the applicability, scalability and reliability of the proposed methods and tools.This research is supported by a partnership between the National Science Foundation and Open Philanthropy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.