Design and Assurance Techniques for Critical Autonomous Software-Intensive Systems

动机和意义：关键的自主软件密集型系统(CASIS)包括自动驾驶车辆、无人机、工业机器人或智能工厂，在这些系统中，CASIS的故障可能导致严重的破坏甚至人员伤亡。在现代CASIS中，机器学习(ML)组件补充传统的软件(SW)组件，以与复杂、不确定和动态变化的环境持续交互。在这样一个开放和动态的环境中，如何用ML和SW组件保证CASIS的可信性是一个重大的长期科学挑战。研究背景：由软件组件控制的安全关键系统的保证由安全标准来规范，但它们往往无法解决不确定性问题。虽然先进的ML技术擅长适应复杂的环境，但其安全保证仍处于早期阶段。我的研究将探讨如何在CASIS中从语义上集成和组合ML模型和SW模型，以证明其符合安全要求。长期目标：我的研究计划的长期目标是使用混合的ML和SW组件为CASIS提供可靠的设计和保证，以合理地符合相关安全标准。研究计划和结果：我的长期研究计划将为混合型CASIS的设计和保证开发新的技术、软件工具和开放的基准。重点将放在提出具有精确语义基础的可扩展解决方案(适用于工业规模系统)上。在接下来的5年里，我的团队将把重点放在四个短期目标上，作为直接的研究挑战：1)继续开发自动化图形生成技术，以合成罕见边缘情况下的各种现实图形模型；2)为运行时的不确定情况下的决策提出(接近)最佳算法，特别是在过去的决策结果不立即知道的情况下；3)为自治的CASIS开发新颖的测试技术，通过对不确定语义模型的推理来得出关键场景；4)为数据密集型软件提供静态分析技术，以早期发现关键错误，例如在ML程序中。研究团队：该研究计划将有助于培训6名博士、2名硕士和10名本科生，他们将同时获得关键系统保证、软件工程和机器学习方面的专业知识。影响：作为关键的科学影响，拟议的研究将为CASIS提供更好的安全保障技术，以纳入极其罕见的事件和意外情况。作为一种社会影响，这种技术可能有助于避免严重事故，减少城市地区的交通拥堵，从而增加公众信任，并为更可持续的交通做出贡献。作为技术影响，图形生成器和静态分析器将揭示各种ML应用程序中难以检测的缺陷，从而为加拿大公司提供大量节省。

Motivation and Significance: Critical autonomous software-intensive systems (CASIS) include self-driving vehicles, drones, industrial robots or smart factories where the failure of a CASIS may result in severe damage or even casualties. In a modern CASIS, machine learning (ML) components complement traditional software (SW) components to continuously interact with a complex, uncertain and dynamically changing environment.  Guaranteeing the trustworthiness of CASIS with both ML and SW components in such an open and dynamic environment is a major long-term scientific challenge.  Research Context: The assurance of safety-critical systems controlled by SW components is regulated by safety standards, but they often fail to address uncertainty. While advanced ML techniques excel at adapting to complex environments, their safety assurance is still in an early stage. My research will investigate how to semantically integrate and combine ML models and SW models in CASIS to justifiably comply with safety requirements. Long-term Goal: The long-term goal of my research program is to provide trustworthy design and assurance of CASIS with mixed ML and SW components to justifiably comply with relevant safety standards. Research Plan and Outcome: My long-term research program will develop novel techniques, software tools and open benchmarks for the design and assurance of mixed CASIS. Significant focus will be placed to come up with scalable solutions (applicable to industrial size systems) with precise semantic foundations. In the next 5 years, my team will focus on four short-term objectives as direct research challenges: 1) continue to develop automated graph generation techniques to synthesize a diverse set of realistic graph models for rare edge cases; 2) propose (near-)optimal algorithms for decision making under uncertainty at runtime, in particular, when the outcome of past decisions is not known immediately; 3) develop novel testing techniques for autonomous CASIS with ML components with inference over uncertain semantic models to derive critical scenarios; 4) provide static analysis techniques for data-intensive software to reveal critical bugs early e.g. in ML programs. Research Team: The research program will contribute to the training of 6 PhD, 2 MSc and 10 undergraduate students who will simultaneously gain expertise in assurance of critical systems, software engineering and machine learning. Impact: As key scientific impact, the proposed research will provide better safety assurance techniques for CASIS to incorporate extremely rare events and unexpected situations. As a social impact, such techniques may help avoid serious accidents and reduce traffic jams in urban areas, thus increasing public trust and contributing to a more sustainable transportation. As technological impact, the graph generator and the static analyzers will reveal hard-to-detect flaws in various ML applications, thus providing substantial savings for Canadian companies.