ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspective
ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspective
复制标题
DOI：
10.1109/tvcg.2022.3209384
复制
发表时间：
2022-04
影响因子：
5.2
通讯作者：
Jinbin Huang;Aditi Mishra;Bum Chul Kwon;Chris Bryan
中科院分区：
文献类型：
--
作者：
Jinbin Huang;Aditi Mishra;Bum Chul Kwon;Chris Bryan
适合模型用户的传统深度学习可解释性方法无法在全局层面解释网络行为，并且在提供细粒度解释方面不灵活。作为一种解决方案，基于概念的解释由于其人类的直观性和描述全局和局部模型行为的灵活性而受到关注。概念是一组类似的有意义的像素，它们表达了一个概念，嵌入在网络的潜在空间中，通常是手工生成的，但最近被自动化方法发现。不幸的是，所发现的概念的数量和多样性使得难以导航和理解概念空间。可视化分析可以通过支持概念空间的结构化导航和探索，为用户提供基于概念的模型行为见解，从而在弥合这些差距方面发挥重要作用。为此，我们设计，开发和验证ConceptExplainer，一个可视化的分析系统，使人们能够交互式地探测和探索概念空间，解释模型的行为在实例/类/全局级别。该系统是通过迭代原型开发的，以解决建模用户在解释深度学习模型的行为时面临的许多设计挑战。通过严格的用户研究，我们验证了ConceptExplainer如何支持这些挑战。同样，我们进行了一系列的使用场景，以展示系统如何支持跨各种任务和解释粒度的模型行为的交互式分析，例如识别对分类重要的概念，识别训练数据中的偏差，以及理解概念如何在不同的和看似不同的类中共享。
Traditional deep learning interpretability methods which are suitable for model users cannot explain network behaviors at the global level and are inflexible at providing fine-grained explanations. As a solution, concept-based explanations are gaining attention due to their human intuitiveness and their flexibility to describe both global and local model behaviors. Concepts are groups of similarly meaningful pixels that express a notion, embedded within the network's latent space and have commonly been hand-generated, but have recently been discovered by automated approaches. Unfortunately, the magnitude and diversity of discovered concepts makes it difficult to navigate and make sense of the concept space. Visual analytics can serve a valuable role in bridging these gaps by enabling structured navigation and exploration of the concept space to provide concept-based insights of model behavior to users. To this end, we design, develop, and validate ConceptExplainer, a visual analytics system that enables people to interactively probe and explore the concept space to explain model behavior at the instance/class/global level. The system was developed via iterative prototyping to address a number of design challenges that model users face in interpreting the behavior of deep learning models. Via a rigorous user study, we validate how ConceptExplainer supports these challenges. Likewise, we conduct a series of usage scenarios to demonstrate how the system supports the interactive analysis of model behavior across a variety of tasks and explanation granularities, such as identifying concepts that are important to classification, identifying bias in training data, and understanding how concepts can be shared across diverse and seemingly dissimilar classes.