Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
复制标题
DOI：
10.1007/s11263-019-01228-7
复制
发表时间：
2020-02-01
影响因子：
19.5
通讯作者：
Batra, Dhruv
中科院分区：
文献类型：
--
作者：
Selvaraju, Ramprasaath R.;Cogswell, Michael;Batra, Dhruv
我们提出了一种用于从一大批基于卷积神经网络（CNN）模型中决策的“视觉解释”的技术，使它们更加透明和可以解释。我们的接近级级加权类激活映射（GRAD-CAM）使用任何目标概念的梯度（在分类网络中说“狗”或字幕网络中的一系列单词），流向最终卷积层产生粗糙的卷积层本地化映射强调图像中重要区域以预测概念。与以前的方法不同，GRAD-CAM适用于多种CNN模型 - 含户：（1）具有完全连接层（例如VGG）的CNN，（2）用于结构化输出（例如接收）的CNN，（3）CNNS用于具有多模式输入的任务（例如，视觉问题的回答）或强化学习，所有这些都没有建筑变化或重新训练。我们将GRAD-CAM与现有的细粒可视化结合在一起，以创建高分辨率类别歧视性可视化，引导的Grad-CAM，并将其应用于图像分类，图像字幕和视觉询问答案（VQA）模型，包括基于Resnet的模型体系结构。在图像分类模型的背景下，我们的可视化（a）对这些模型的故障模式提供了洞察力（表明看似不合理的预测具有合理的解释），（b）在ILSVRC-15弱弱化的本地化任务上的先前方法优于先前的方法，（（ c）对对抗性扰动是强大的，（d）更忠实于基础模型，（e）通过识别数据集偏差来帮助实现模型概括。对于图像字幕和VQA，我们的可视化表明，即使是基于非注意的模型，也可以学习本地化输入图像的区分区域。我们设计了一种通过Grad-CAM识别重要神经元的方法，并将其与神经元名称（Bau等人的计算机视觉和模式识别中，2017年）相结合，以提供模型决策的文本解释。最后，我们设计和进行人体研究，以衡量Grad-CAM的解释是否有助于用户对深层网络的预测建立适当的信任，并表明Grad-CAM可以帮助未经训练的用户成功地从“较弱”的网络中辨别出“更强”的深网络，即使两者都做出相同的预测。我们的代码可在CloudCV上的演示（Agrawal等人，in：移动云视觉媒体计算，pp265-290。Springer，2015）和视频中获得。
We propose a technique for producing 'visual explanations' for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach-Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say 'dog' in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are robust to adversarial perturbations, (d) are more faithful to the underlying model, and (e) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show that even non-attention based models learn to localize discriminative regions of input image. We devise a way to identify important neurons through Grad-CAM and combine it with neuron names (Bau et al. in Computer vision and pattern recognition, 2017) to provide textual explanations for model decisions. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a 'stronger' deep network from a 'weaker' one even when both make identical predictions. Our code is available at , along with a demo on CloudCV (Agrawal et al., in: Mobile cloud visual media computing, pp 265-290. Springer, 2015) () and a video at .