Design and Assurance Techniques for Critical Autonomous Software-Intensive Systems

动机和意义：关键自主软件密集型系统（CASIS）包括自动驾驶汽车、无人机、工业机器人或智能工厂，其中CASIS的故障可能导致严重损害甚至伤亡。在现代CASIS中，机器学习（ML）组件补充了传统软件（SW）组件，以持续与复杂，不确定和动态变化的环境进行交互。 在这样一个开放和动态的环境中，用ML和SW组件保证CASIS的可信性是一个重大的长期科学挑战。 研究背景：由软件组件控制的安全关键系统的保证由安全标准进行管理，但它们通常无法解决不确定性。虽然先进的机器学习技术在适应复杂环境方面表现出色，但其安全保障仍处于早期阶段。我的研究将研究如何在CASIS中语义集成和联合收割机ML模型和SW模型，以符合安全需求。长期目标：我的研究计划的长期目标是提供值得信赖的设计和保证CASIS与混合ML和SW组件，以确保符合相关的安全标准。研究计划和结果：我的长期研究计划将开发新的技术，软件工具和开放基准的设计和混合CASIS的保证。重点将放在提出具有精确语义基础的可扩展解决方案（适用于工业规模的系统）上。在接下来的5年里，我的团队将专注于四个短期目标，作为直接的研究挑战：1）继续开发自动图生成技术，以合成一组不同的现实图模型，用于罕见的边缘情况; 2）提出（近）最优算法，用于在运行时不确定性下的决策，特别是当过去决策的结果不能立即知道时; 3）为具有ML组件的自主CASIS开发新的测试技术，通过对不确定语义模型的推理来推导关键场景; 4）为数据密集型软件提供静态分析技术，以早期揭示ML程序中的关键错误。研究团队：该研究计划将有助于培养6名博士，2名硕士和10名本科生，他们将同时获得关键系统，软件工程和机器学习保证方面的专业知识。影响：作为关键的科学影响，拟议的研究将为CASIS提供更好的安全保证技术，以纳入极其罕见的事件和意外情况。作为一种社会影响，这些技术可能有助于避免严重事故，减少城市地区的交通堵塞，从而增加公众的信任，促进更可持续的交通。作为技术影响，图形生成器和静态分析器将揭示各种ML应用程序中难以检测的缺陷，从而为加拿大公司节省大量资金。

Motivation and Significance: Critical autonomous software-intensive systems (CASIS) include self-driving vehicles, drones, industrial robots or smart factories where the failure of a CASIS may result in severe damage or even casualties. In a modern CASIS, machine learning (ML) components complement traditional software (SW) components to continuously interact with a complex, uncertain and dynamically changing environment.  Guaranteeing the trustworthiness of CASIS with both ML and SW components in such an open and dynamic environment is a major long-term scientific challenge.  Research Context: The assurance of safety-critical systems controlled by SW components is regulated by safety standards, but they often fail to address uncertainty. While advanced ML techniques excel at adapting to complex environments, their safety assurance is still in an early stage. My research will investigate how to semantically integrate and combine ML models and SW models in CASIS to justifiably comply with safety requirements. Long-term Goal: The long-term goal of my research program is to provide trustworthy design and assurance of CASIS with mixed ML and SW components to justifiably comply with relevant safety standards. Research Plan and Outcome: My long-term research program will develop novel techniques, software tools and open benchmarks for the design and assurance of mixed CASIS. Significant focus will be placed to come up with scalable solutions (applicable to industrial size systems) with precise semantic foundations. In the next 5 years, my team will focus on four short-term objectives as direct research challenges: 1) continue to develop automated graph generation techniques to synthesize a diverse set of realistic graph models for rare edge cases; 2) propose (near-)optimal algorithms for decision making under uncertainty at runtime, in particular, when the outcome of past decisions is not known immediately; 3) develop novel testing techniques for autonomous CASIS with ML components with inference over uncertain semantic models to derive critical scenarios; 4) provide static analysis techniques for data-intensive software to reveal critical bugs early e.g. in ML programs. Research Team: The research program will contribute to the training of 6 PhD, 2 MSc and 10 undergraduate students who will simultaneously gain expertise in assurance of critical systems, software engineering and machine learning. Impact: As key scientific impact, the proposed research will provide better safety assurance techniques for CASIS to incorporate extremely rare events and unexpected situations. As a social impact, such techniques may help avoid serious accidents and reduce traffic jams in urban areas, thus increasing public trust and contributing to a more sustainable transportation. As technological impact, the graph generator and the static analyzers will reveal hard-to-detect flaws in various ML applications, thus providing substantial savings for Canadian companies.