SCADNet - Structural Causal Analysis of Deep Neural Networks

基于反例计算和反事实因果推理的因果解释是(安全)关键软件驱动系统(CS)设计和工程的基石。它们是迭代系统设计周期中系统调试的基础，形成了诉讼中失败取证的基础，并且是系统安全性认证以及社会接受的核心。在这个项目中，我们关注的是依赖于使用机器学习(ML)技术实现的决策制定组件的CS，特别是深度学习(DL)。正如广泛承认的那样，获得对这种类型的CS的解释充满了巨大的挑战，这主要是由于缺乏易于解释的逻辑代码结构，因此，现有的软件验证技术不适用。我们将为深度神经网络(DNN)开发新的结构抽象和分析技术，允许在基于规范的安全验证的上下文中计算反例。然后，我们将解决安全违规行为的反事实证据的引出问题，接下来将转向建议对违反特定安全属性的DNN进行自动修复，从而有助于解释DNN的行为。最后，我们将把这些结果放在基于ML的CS的反事实因果分析的一般理论的背景下。

Causal explanations based on counterexample computation and counterfactual causal reasoning are a cornerstone of the design and engineering of (Safety-)Critical Software-Driven Systems (CSS). They are the foundation of system debugging during iterative system design cycles, form the basis of failure forensics in litigation and are central to the certification of system safety as well as societal acceptance. In this project we focus on CSS that rely on decision-making components which are implemented using Machine Learning (ML) technology, in particular DeepLearning (DL). As widely acknowledged, obtaining explanations for this type of CSS is fraught with significant challenges, largely due to the absence of easily interpretable logical code structure and, as a consequence, the non-applicability of existing software verification technology. We will develop novel structural abstraction and analysis techniques for Deep Neural Networks (DNNs) that allow for the computation of counterexamples in the context of specification-based safety verification. We will then address the elicitation of counterfactual evidence for safety violations and next turn to proposing automatic repairs for DNNs that violate specified safety properties, thus contributing to the explanation of the behavior of a DNN. Finally, we will place these results in the context of a general theory of counterfactual causal analysis for ML-based CSS.