NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks
NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks
复制标题
DOI：
10.1109/tvcg.2021.3114858
复制
发表时间：
2021-08
影响因子：
5.2
通讯作者：
Haekyu Park;Nilaksh Das;Rahul Duggal;Austin P. Wright;Omar Shaikh;Fred Hohman;Duen Horng Chau
中科院分区：
文献类型：
--
作者：
Haekyu Park;Nilaksh Das;Rahul Duggal;Austin P. Wright;Omar Shaikh;Fred Hohman;Duen Horng Chau
现有的关于理解深度神经网络的研究通常集中在神经元级别的解释上，这可能无法充分捕获多个神经元如何集体编码概念的更大图景。我们提出了 NEUROCARTOGRAPHY，这是一种交互式系统，可以大规模地总结和可视化神经网络学习的概念。它自动发现并分组检测相同概念的神经元，并描述这些神经元组如何相互作用以形成更高级别的概念和后续预测。神经制图引入了两种可扩展的概括技术：（1）神经元聚类根据神经元检测到的概念的语义相似性对神经元进行分组（例如，将检测到不同品种的“狗脸”的神经元进行分组）； （2）神经元嵌入根据相关概念同时出现的频率对它们之间的关联进行编码（例如，检测“狗脸”和“狗尾巴”的神经元在嵌入空间中放置得更近）。我们可扩展技术的关键是能够有效计算所有神经元对的关系，时间与神经元数量呈线性关系，而不是二次时间。 NEUROCARTOGRAPHY 可扩展到大数据，例如具有 120 万张图像的 ImageNet 数据集。该系统紧密协调的视图集成了可扩展技术，以可视化概念及其关系，将概念关联投影到神经元投影视图中的二维空间，并在图形视图中总结神经元簇及其关系。通过大规模的人类评估，我们证明我们的技术可以发现代表连贯的、对人类有意义的概念的神经元组。通过使用场景，我们描述了我们的方法如何实现有趣和令人惊讶的发现，例如相关和孤立概念的概念级联。 NEUROCARTOGRAPHY 可视化在现代浏览器中运行并且是开源的。
Existing research on making sense of deep neural networks often focuses on neuron-level interpretation, which may not adequately capture the bigger picture of how concepts are collectively encoded by multiple neurons. We present NEUROCARTOGRAPHY, an interactive system that scalably summarizes and visualizes concepts learned by neural networks. It automatically discovers and groups neurons that detect the same concepts, and describes how such neuron groups interact to form higher-level concepts and the subsequent predictions. NEUROCARTOGRAPHY introduces two scalable summarization techniques: (1) neuron clustering groups neurons based on the semantic similarity of the concepts detected by neurons (e.g., neurons detecting “dog faces” of different breeds are grouped); and (2) neuron embedding encodes the associations between related concepts based on how often they co-occur (e.g., neurons detecting “dog face” and “dog tail” are placed closer in the embedding space). Key to our scalable techniques is the ability to efficiently compute all neuron pairs' relationships, in time linear to the number of neurons instead of quadratic time. NEUROCARTOGRAPHY scales to large data, such as the ImageNet dataset with 1.2M images. The system's tightly coordinated views integrate the scalable techniques to visualize the concepts and their relationships, projecting the concept associations to a 2D space in Neuron Projection View, and summarizing neuron clusters and their relationships in Graph View. Through a large-scale human evaluation, we demonstrate that our technique discovers neuron groups that represent coherent, human-meaningful concepts. And through usage scenarios, we describe how our approaches enable interesting and surprising discoveries, such as concept cascades of related and isolated concepts. The NEUROCARTOGRAPHY visualization runs in modern browsers and is open-sourced.