Systematic Evaluation of Assurance Cases

保证案例的系统评估

• 批准号: RGPIN-2019-06022
• 负责人: Wassyng, Alan
• 金额: $ 2.48万
• 依托单位: McMaster University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750441
• 关键词: Systematic; Evaluation; Assurance; Cases

Background The possibility of complex software-intensive systems contributing to accidents has increased dramatically. In recent years we have seen Assurance Cases (ACs) assume a prominent role in assuring safety of these systems. An AC presents a claim about what we want to assure, and then presents explicit reasoning, backed-up by evidence, that we have adequate confidence that the claim is true. In practice, these claims are not precise and rely on societal norms of what is "good enough". It is thus vital that we develop sound ways of evaluating ACs. There is ongoing research related to "confidence" in ACs, which primarily relates to a "measure" of how sound the conclusion is. "Evaluation" of an AC, in the proposed work, is more general than measuring confidence, and considers multiple aspects of the quality of an AC. Evaluation of ACs has not received the attention it warrants, and should significantly improve system safety . Overarching Goal The 5 year goal of my research is to develop and test evaluation criteria and evaluation processes for ACs. It fits within my research group's mission to develop effective methods for certifying safety-critical software-intensive systems. The current state of the practice for ACs is to use a notation such as Goal Structuring Notation as the basis for the AC. My research on evaluation of the quality of ACs will consider current ACs as well as a new, more formal method for building ACs, that my research group is developing outside of the work in this proposal. Challenges i) ACs are enormous and complex. ii) Most reasoning in ACs is informal. iii) We have few metrics to evaluate ACs. iv) The fundamental claim in an AC is not precise, and relies on societal norms. v) We must achieve incremental assurance, i.e., modify existing ACs to reflect all kinds of changes, without redoing the complete AC - and without "weakening" the AC. vi) An AC may be subject to "confirmation bias". Research Outcomes & Direction We will develop and test criteria that will form the basis of our evaluation methods. In evaluating an AC, we need to distinguish between: 1) the content of the AC; and 2) the documentation structure/notation of the AC. Initial criteria for "content" include: i) convincing basis for the AC, ii) rigour of the argument, iii) support for completeness arguments, iv) repeatability, v) satisfaction of acceptance criteria for evidence. Criteria for "structure" include: i) traceability, ii) robustness with respect to change, iii) understandability, iv) efficiency. External evaluators of an AC may use different evaluation criteria compared with the authors of the AC. We will deal with both cases. For authors of the AC, we will emphasize the use of pair-wise comparison.  Impact Effective evaluation of ACs is a necessary step to improve and maintain safety of complex, software-intensive systems. This research will save lives. It will also enhance Canada's reputation in certification of software-intensive systems.

复杂的软件密集型系统导致事故的可能性急剧增加。近年来，我们已经看到保证案例（AC）在确保这些系统的安全性方面发挥了重要作用。AC提出了一个关于我们想要保证的东西的主张，然后提出了明确的推理，并得到了证据的支持，我们有足够的信心相信这个主张是正确的。实际上，这些说法并不精确，而是依赖于什么是“足够好”的社会规范。因此，至关重要的是，我们制定了合理的评估AC的方法。目前正在进行的研究与AC中的“信心”有关，这主要涉及到一个“衡量”结论的合理性。在拟议的工作中，对AC的“评估”比测量置信度更一般，并考虑AC质量的多个方面。AC的评估没有得到应有的重视，应该显着提高系统的安全性。总体目标我研究的5年目标是开发和测试AC的评估标准和评估流程。它符合我的研究小组的使命，即开发有效的方法来认证安全关键的软件密集型系统。AC的实践的当前状态是使用诸如目标结构化表示法之类的表示法作为AC的基础。我对评估AC质量的研究将考虑当前的AC以及一种新的，更正式的构建AC的方法，我的研究小组正在开发本提案中的工作之外。（i）空调是巨大而复杂的。2. AC中的大多数推理都是非正式的。iii）我们几乎没有评估AC的指标。（iv）AC中的基本主张并不精确，并且依赖于社会规范。（五）我们必须实现增量保证，即：修改现有的AC以反映所有类型的更改，而无需重做完整的AC -也不会“削弱”AC。 vi）AC可能受到“确认偏差”的影响。研究成果和方向我们将制定和测试标准，这些标准将成为我们评估方法的基础。在评估AC时，我们需要区分：1）AC的内容; 2）AC的文档结构/符号。“内容”的初始标准包括：i）AC的令人信服的基础，ii）论证的严谨性，iii）对完整性论证的支持，iv）可重复性，v）满足证据的验收标准。“结构”的标准包括：i）可追溯性，ii）相对于变化的鲁棒性，iii）可理解性，iv）效率。AC的外部评价者可能使用与AC作者不同的评价标准。我们将处理这两个案件。对于AC的作者，我们将强调使用成对比较。影响AC的有效评估是提高和维护复杂的软件密集型系统安全性的必要步骤。它还将提高加拿大在软件密集型系统认证方面的声誉。