This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition
复制标题
看起来像那样，因为......解释可解释图像识别的原型
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
C. Seifert
中科院分区：
文献类型：
--
作者：
Meike Nauta;Annemarie Jutte;Jesper C. Provoost;C. Seifert
带原型的图像识别被认为是黑盒深度学习模型的一种可解释的替代方案。分类取决于测试图像“看起来”像原型的程度。然而，人类的知觉相似性可能与模型学到的相似性不同。用户不知道基本的分类策略，并且不知道哪个图像特征(例如，颜色或形状)是用于决策的主要特征。我们解决了这种模棱两可的问题，并认为原型应该得到解释。只有可视化的原型不足以理解原型到底代表什么，以及为什么原型和图像被认为是相似的。我们通过使用模型认为重要的视觉特征的额外信息自动增强原型来提高可解释性。具体地说，我们的方法量化了原型中颜色色调、形状、纹理、对比度和饱和度的影响。我们将我们的方法应用于现有的原型零件网络(ProtoPNet)，并表明我们的解释澄清了原型的含义，否则可能会被错误地解释。我们还发现，视觉上相似的原型可以有相同的解释，这表明存在冗余。由于我们方法的通用性，它可以提高任何基于相似度的原型图像识别方法的可解释性。
Image recognition with prototypes is considered an interpretable alternative for black box deep learning models. Classification depends on the extent to which a test image "looks like" a prototype. However, perceptual similarity for humans can be different from the similarity learnt by the model. A user is unaware of the underlying classification strategy and does not know which image characteristics (e.g., color or shape) is the dominant characteristic for the decision. We address this ambiguity and argue that prototypes should be explained. Only visualizing prototypes can be insufficient for understanding what a prototype exactly represents, and why a prototype and an image are considered similar. We improve interpretability by automatically enhancing prototypes with extra information about visual characteristics considered important by the model. Specifically, our method quantifies the influence of color hue, shape, texture, contrast and saturation in a prototype. We apply our method to the existing Prototypical Part Network (ProtoPNet) and show that our explanations clarify the meaning of a prototype which might have been interpreted incorrectly otherwise. We also reveal that visually similar prototypes can have the same explanations, indicating redundancy. Because of the generality of our approach, it can improve the interpretability of any similarity-based method for prototypical image recognition.