Foundation of Safe Autonomous Systems

点击翻译按钮获取中文摘要

Autonomous systems are about to make the jump to main stream commercial products that will be encounter by the public in their daily lives in the form of autonomous vehicles, delivery drones, etc. Semi autonomous vehicle features such as Tesla's autopilot and GM's Supercruise are already in widespread use. Elon Musk has gone so far as to claim that the Tesla ``Autopilot'' is safer than human drivers based upon the fact that over 100 million autopilot miles were driven before the first recorded fatality. While the safety of autonomous vehicles relative to human drivers is currently debatable, there is no doubt that such autonomous systems have the potential to deliver tremendous benefits. That said, they will only gain the public acceptance that would allow their widespread use if they are designed to be acceptably safe and reliable.******Using the first recorded North American fatality involving an autonomous driving system as a motivational example, the proposal outlines a program of research to provide the foundations to affordably design safe, reliable autonomous systems. Using the applicants previous experience in safety critical software systems and control of probabilistic discrete event systems, the proposal develops three main themes:******(1) Fault tolerant architectures that incorporate a standardized monitoring concept for subsystems that use machine learning (ML) and artificial intelligence (AI) techniques for environment sensing and control such as vision processing and path planning. Initially these subsystems will be treated as "black box'' systems and then eventually as "gray box" systems that can continuously have key aspects of their internal health and decisions monitored. ******(2) Methods and tools to provide a better observability of the current state of ML and AI systems in order to better understand when, how and why these systems fail. This improved ability to estimate the state or status of the ML and AI systems can then be used to provide fault diagnosis, improve reliability, and gain confidence in ML and AI decisions. ******(3) Standardized safety arguments in the form of assurance (safety) case templates for autonomous systems to allow these systems to be developed more reliably and affordably together with tool support to formally verify and analyze the dependability of the designs. ******The proposed research will provide results and HQP that will support Canadian companies in developing the coming wave of autonomous systems that will have to safely interact with the general public. With the global driver assist market alone estimated to be a $102 billion market by 2030, the work is of critical importance.