Foundation of Safe Autonomous Systems

自动驾驶系统即将跃升为主流商业产品，公众将在日常生活中以自动驾驶汽车、送货无人机等形式遇到这些产品。特斯拉的自动驾驶和通用汽车的Suercruise等半自动驾驶汽车功能已经得到广泛使用。埃隆·马斯克甚至声称，特斯拉的自动驾驶比人类驾驶更安全，因为在第一起有记录的死亡事件之前，自动驾驶的行驶里程超过1亿英里。虽然自动驾驶汽车相对于人类司机的安全性目前仍有争议，但毫无疑问，这种自动驾驶系统有可能带来巨大的好处。尽管如此，只有在它们被设计成可接受的安全可靠的情况下，它们才能获得公众的接受，才能得到广泛的使用。*以北美第一起有记录的涉及自动驾驶系统的死亡事件为例，提案概述了一个研究计划，为以负担得起的价格设计安全、可靠的自动驾驶系统提供基础。利用申请人以前在安全关键软件系统和概率离散事件系统控制方面的经验，该提案开发了三个主要主题：*(1)容错体系结构，该体系结构纳入了对使用机器学习(ML)和人工智能(AI)技术进行环境感知和控制(如视觉处理和路径规划)的子系统的标准化监测概念。最初，这些子系统将被视为“黑箱”系统，然后最终被视为“灰箱”系统，可以持续监控其内部运行状况和决策的关键方面。*(2)方法和工具，以便更好地观察ML和AI系统的当前状态，以便更好地了解这些系统何时、如何和为什么发生故障。这种估计ML和AI系统状态或状态的改进能力随后可用于提供故障诊断、提高可靠性并获得对ML和AI决策的信心。*(3)以自主系统的保证(安全)案例模板的形式进行标准化的安全论证，以使这些系统的开发更加可靠且经济实惠，同时提供工具支持以正式验证和分析设计的可靠性。*拟议的研究将提供结果和HQP，支持加拿大公司开发即将到来的自主系统浪潮，这些系统将不得不与普通公众安全互动。据估计，到2030年，仅全球驾驶员辅助市场就将达到1020亿美元的规模，因此这项工作至关重要。

Autonomous systems are about to make the jump to main stream commercial products that will be encounter by the public in their daily lives in the form of autonomous vehicles, delivery drones, etc. Semi autonomous vehicle features such as Tesla's autopilot and GM's Supercruise are already in widespread use. Elon Musk has gone so far as to claim that the Tesla ``Autopilot'' is safer than human drivers based upon the fact that over 100 million autopilot miles were driven before the first recorded fatality. While the safety of autonomous vehicles relative to human drivers is currently debatable, there is no doubt that such autonomous systems have the potential to deliver tremendous benefits. That said, they will only gain the public acceptance that would allow their widespread use if they are designed to be acceptably safe and reliable.******Using the first recorded North American fatality involving an autonomous driving system as a motivational example, the proposal outlines a program of research to provide the foundations to affordably design safe, reliable autonomous systems. Using the applicants previous experience in safety critical software systems and control of probabilistic discrete event systems, the proposal develops three main themes:******(1) Fault tolerant architectures that incorporate a standardized monitoring concept for subsystems that use machine learning (ML) and artificial intelligence (AI) techniques for environment sensing and control such as vision processing and path planning. Initially these subsystems will be treated as "black box'' systems and then eventually as "gray box" systems that can continuously have key aspects of their internal health and decisions monitored. ******(2) Methods and tools to provide a better observability of the current state of ML and AI systems in order to better understand when, how and why these systems fail. This improved ability to estimate the state or status of the ML and AI systems can then be used to provide fault diagnosis, improve reliability, and gain confidence in ML and AI decisions. ******(3) Standardized safety arguments in the form of assurance (safety) case templates for autonomous systems to allow these systems to be developed more reliably and affordably together with tool support to formally verify and analyze the dependability of the designs. ******The proposed research will provide results and HQP that will support Canadian companies in developing the coming wave of autonomous systems that will have to safely interact with the general public. With the global driver assist market alone estimated to be a $102 billion market by 2030, the work is of critical importance.