RII Track-4:NSF: Safety Validation of Autonomous Systems from Multiple Sources of Information

随着人们对人工智能(AI)在自治领域的使用兴趣的快速增长，革命性的安全验证方法至关重要，这些方法可以解释复杂的AI启用的自治系统的安全行为。该项目的目标是为安全关键型应用程序建立对启用人工智能的复杂系统的信任。这种信任可以通过一些离线或在线验证过程来建立。在线核查方法的缺点是，它们需要某种形式的现实部署，这可能是不安全和有风险的。通常，在将基于人工智能的决策系统部署到现实世界之前，在模拟环境中揭示可能的故障场景是非常有意义的。由于复杂系统中故障事件和转角情况的空间很广，因此验证过程可能非常耗时，因为安全验证需要大量的实验。该项目旨在开发从多个来源获取信息的方法，以显著加快验证过程并降低总体计算成本。通过与斯坦福人工智能安全中心的合作，PI和研究生实习生将获得宝贵的培训机会，这将有助于在西弗吉尼亚大学(WVU)和斯坦福大学之间建立强大的STEM研究和教育合作伙伴关系。该项目的总体目标是开发自主系统的安全验证算法，从多个信息来源推断自主系统的安全行为。这项工作背后的核心理念是，网络物理系统(CPS)或机器人可以从多个来源查询数据，包括模拟、离线或在线真实世界数据和/或人类专家输入的不同粒度。目前，没有严格的机制来推理支持学习的决策系统的安全行为，该系统以最佳方式考虑来自不同信息源的数据。这项研究将结合斯坦福人工智能安全中心的不确定性决策和正式方法专业知识，以及PI在机器学习和数据驱动优化技术方面的专业知识，以得出利用来自多个来源的数据的安全验证框架。PI和他的学生将开发来自数据驱动的优化和强化学习算法的工具，以从多种信息来源识别故障事件。所提出的算法将应用于一套自动驾驶的模拟环境。这项研究将极大地扩展自主系统安全验证的工具和开源软件。通过与斯坦福人工智能安全中心的合作，PI将保持学术界和产业界的联系，为斯坦福大学和西弗吉尼亚大学之间的联合提案撰写、联合期刊出版和学生交换项目开辟新的途径。这项提议将被整合到一项教育计划中，让本科生和研究生参与研究，并丰富西弗吉尼亚大学(WVU)的机器人学和工程学课程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

With the rapid growth of interest in the use of artificial intelligence (AI) in autonomy, it is critical to revolutionize safety validation approaches that reason about the safety behaviors of a complex AI-enabled autonomous system. The goal of this project is to build trust in AI-enabled complex systems for safety-critical applications. This trust could be built by means of some offline or online validation processes. The drawback of online verification methods is that they require some form of real-world deployments which could be unsafe and risky. Typically, it is of great interest to reveal possible failure scenarios in a simulated environment before deploying an AI-based decision-making system into the real world. Since the space of failure events and corner cases is extensive in complex systems, the validation process might be very time-consuming as a huge number of experiments are required for safety validation. This project aims to develop approaches that capture information from multiple sources to significantly speed up the validation process and reduce the overall computational cost. Through the collaboration with the Stanford Center for AI Safety, the PI and graduate trainee will gain invaluable training opportunities that will help to build a strong STEM research and education partnership between West Virginia University (WVU) and Stanford.The overarching objective of this project is to develop algorithms for safety validation of autonomous systems that reason about the safety behaviors of autonomous systems from multiple sources of information. The central philosophy behind this work is that a cyber-physical system (CPS) or a robot can query data from multiple sources, including different levels of granularity in simulation, offline or online real-world data, and/or human expert inputs. Currently, there is no rigorous mechanism to reason about the safety behaviors of a learning-enabled decision-making system that optimally considers data from different sources of information. This research will combine the decision making under uncertainty and formal methods expertise at the Stanford Center for AI Safety and the PI's expertise in machine learning and data-driven optimization techniques to arrive at safety validation frameworks that leverage data from multiple sources. The PI and his students will develop tools from data-driven optimization and reinforcement learning algorithms to identify failure events from multiple sources of information. The proposed algorithms will be applied to a suite of simulated environments for autonomous driving. This research will significantly extend the tools and open-source software for the safety validation of autonomous systems. Through collaboration with the Stanford Center for AI Safety, the PI will maintain ties between academia and industry, open new avenues for joint proposal writing, joint journal publications, and student exchange programs between Stanford and WVU. The proposal will be integrated into an educational plan that involves undergraduate and graduate students in research and enriches the curriculum of robotics and engineering at West Virginia University (WVU).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.