Concept-Level Model Interpretation From the Causal Aspect
Concept-Level Model Interpretation From the Causal Aspect
复制标题
DOI：
10.1109/tkde.2022.3209997
复制
发表时间：
2023-09
影响因子：
8.9
通讯作者：
Liuyi Yao;Yaliang Li;Sheng Li;Jinduo Liu;Mengdi Huai;Aidong Zhang;Jing Gao
中科院分区：
文献类型：
--
作者：
Liuyi Yao;Yaliang Li;Sheng Li;Jinduo Liu;Mengdi Huai;Aidong Zhang;Jing Gao
随着数据的增长和学习能力的增强，机器学习模型被广泛应用于各个领域。然而，很少有机器学习模型能够对它们的预测进行推理，这限制了它们在现实任务中的进一步应用。随着解决这一困境的潜力，模型解释已经成为一个重要的研究课题，因为它能够在特征水平或概念水平上为模型预测提供潜在的原因。概念层面的模型解释侧重于探索概念在模型预测中的作用，这使得解释更简洁、更容易理解。概念级模型解释需要确定有助于模型预测的概念，并探索这些概念下的规则。为了实现这两个目标，我们从因果关系的角度提出了一个概念级模型解释框架。CMIC可以自动检测数据中的概念，并发现检测到的概念与模型的预测标签之间的因果关系。此外，CMIC根据概念对模型预测的因果影响对概念的贡献进行排序，反映了检测到的概念的重要性。我们在人工数据集和真实数据集上对所提出的CMIC框架进行了评估，以证明所提供的解释的质量。
With the increasing growth of data and the ability of learning with them, machine learning models are adopted in various domains. However, few of machine learning models are able to reason their prediction, which limits their further applications in real-world tasks. With the potential to address this dilemma, model interpretation has become an important research topic because of the ability to provide the underlying reasons for model predictions at the feature level or concept level. Model interpretation at the concept level focuses on exploring the roles of concepts in model prediction, which enables more compact and understandable interpretations. Concept-level model interpretation requires the identification of the concepts that contribute to model prediction and the exploration of the rules underneath these concepts. To achieve the two objectives, we propose a Concept-level Model Interpretation framework (CMIC) from the perspective of causality. CMIC can automatically detect concepts in data and discover the causal relation between the detected concepts and the model's predicted labels. Furthermore, CMIC ranks the contributions of concepts by their causal effect on the model prediction, reflecting the detected concepts’ importance. We evaluate the proposed CMIC framework on both synthetic and real-world datasets to demonstrate the quality of the provided interpretation.