Systematic Evaluation of Assurance Cases

点击翻译按钮获取中文摘要

Background The possibility of complex software-intensive systems contributing to accidents has increased dramatically. In recent years we have seen Assurance Cases (ACs) assume a prominent role in assuring safety of these systems. An AC presents a claim about what we want to assure, and then presents explicit reasoning, backed-up by evidence, that we have adequate confidence that the claim is true. In practice, these claims are not precise and rely on societal norms of what is "good enough". It is thus vital that we develop sound ways of evaluating ACs. There is ongoing research related to "confidence" in ACs, which primarily relates to a "measure" of how sound the conclusion is. "Evaluation" of an AC, in the proposed work, is more general than measuring confidence, and considers multiple aspects of the quality of an AC. Evaluation of ACs has not received the attention it warrants, and should significantly improve system safety . Overarching Goal The 5 year goal of my research is to develop and test evaluation criteria and evaluation processes for ACs. It fits within my research group's mission to develop effective methods for certifying safety-critical software-intensive systems. The current state of the practice for ACs is to use a notation such as Goal Structuring Notation as the basis for the AC. My research on evaluation of the quality of ACs will consider current ACs as well as a new, more formal method for building ACs, that my research group is developing outside of the work in this proposal. Challenges i) ACs are enormous and complex. ii) Most reasoning in ACs is informal. iii) We have few metrics to evaluate ACs. iv) The fundamental claim in an AC is not precise, and relies on societal norms. v) We must achieve incremental assurance, i.e., modify existing ACs to reflect all kinds of changes, without redoing the complete AC - and without "weakening" the AC. vi) An AC may be subject to "confirmation bias". Research Outcomes & Direction We will develop and test criteria that will form the basis of our evaluation methods. In evaluating an AC, we need to distinguish between: 1) the content of the AC; and 2) the documentation structure/notation of the AC. Initial criteria for "content" include: i) convincing basis for the AC, ii) rigour of the argument, iii) support for completeness arguments, iv) repeatability, v) satisfaction of acceptance criteria for evidence. Criteria for "structure" include: i) traceability, ii) robustness with respect to change, iii) understandability, iv) efficiency. External evaluators of an AC may use different evaluation criteria compared with the authors of the AC. We will deal with both cases. For authors of the AC, we will emphasize the use of pair-wise comparison.  Impact Effective evaluation of ACs is a necessary step to improve and maintain safety of complex, software-intensive systems. This research will save lives. It will also enhance Canada's reputation in certification of software-intensive systems.