A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation
A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation
复制标题
DOI：
10.48550/arxiv.2306.07304
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Thomas Fel;Victor Boutin;Mazda Moayeri;Rémi Cadène;Louis Béthune;L'eo And'eol;Mathieu Chalvidal;Thomas Serre
中科院分区：
文献类型：
--
作者：
Thomas Fel;Victor Boutin;Mazda Moayeri;Rémi Cadène;Louis Béthune;L'eo And'eol;Mathieu Chalvidal;Thomas Serre
近年来，基于概念的方法已经成为一些最有前途的解释性方法，以帮助我们解释人工神经网络（ANN）的决策。这些方法试图通过两个关键步骤发现埋藏在人工神经网络激活的复杂模式中的可理解的视觉“概念”：（1）概念提取，然后是（2）重要性估计。虽然这两个步骤在方法之间共享，但它们在具体实现上都有所不同。在这里，我们介绍一个统一的理论框架，全面定义和澄清这两个步骤。这个框架提供了几个优点，因为它允许我们：（i）提出新的评价指标，比较不同的概念提取方法;（ii）利用现代属性方法和评价指标，以扩展和系统地评估国家的最先进的概念为基础的方法和重要性估计技术;（iii）获得理论保证，这些方法的最优性。我们进一步利用我们的框架来尝试解决可解释性中的一个关键问题：如何有效地识别基于类似共享策略分类的数据点集群。为了说明这些发现并突出模型的主要策略，我们引入了一种称为策略聚类图的可视化表示。最后，我们介绍了https：//serre-lab.github.io/Lens，这是一个专门的网站，为ImageNet数据集的所有类提供了这些可视化的完整编译。
In recent years, concept-based approaches have emerged as some of the most promising explainability methods to help us interpret the decisions of Artificial Neural Networks (ANNs). These methods seek to discover intelligible visual 'concepts' buried within the complex patterns of ANN activations in two key steps: (1) concept extraction followed by (2) importance estimation. While these two steps are shared across methods, they all differ in their specific implementations. Here, we introduce a unifying theoretical framework that comprehensively defines and clarifies these two steps. This framework offers several advantages as it allows us: (i) to propose new evaluation metrics for comparing different concept extraction approaches; (ii) to leverage modern attribution methods and evaluation metrics to extend and systematically evaluate state-of-the-art concept-based approaches and importance estimation techniques; (iii) to derive theoretical guarantees regarding the optimality of such methods. We further leverage our framework to try to tackle a crucial question in explainability: how to efficiently identify clusters of data points that are classified based on a similar shared strategy. To illustrate these findings and to highlight the main strategies of a model, we introduce a visual representation called the strategic cluster graph. Finally, we present https://serre-lab.github.io/Lens, a dedicated website that offers a complete compilation of these visualizations for all classes of the ImageNet dataset.