Match them up: visually explainable few-shot image classification
Match them up: visually explainable few-shot image classification
复制标题
DOI：
10.1007/s10489-022-04072-4
复制
发表时间：
2020-11
影响因子：
5.3
通讯作者：
Bowen Wang;Liangzhi Li;Manisha Verma;Yuta Nakashima;R. Kawasaki;H. Nagahara
中科院分区：
文献类型：
--
作者：
Bowen Wang;Liangzhi Li;Manisha Verma;Yuta Nakashima;R. Kawasaki;H. Nagahara
少样本学习（FSL）方法主要基于神经网络，假设预先训练的知识可以从基础（已见）类中获得，并转移到新的（未见）类中。然而，神经网络的黑盒性质使得很难理解实际传输的内容，这可能会阻碍 FSL 在一些风险敏感领域的应用。在本文中，我们揭示了一种执行 FSL 进行图像分类的新方法，使用主干模型的视觉表示和基于自注意力的可解释模块生成的模式。按模式加权的表示仅包括最少数量的可区分特征，并且可视化模式可以作为所转移知识的信息提示。在三个主流数据集上的实验结果证明，该方法能够实现令人满意的可解释性并取得较高的分类结果。代码可在 https://github.com/wbw520/MTUNet 获取。
Few-shot learning (FSL) approaches, mostly neural network-based, assume that pre-trained knowledge can be obtained from base (seen) classes and transferred to novel (unseen) classes. However, the black-box nature of neural networks makes it difficult to understand what is actually transferred, which may hamper FSL application in some risk-sensitive areas. In this paper, we reveal a new way to perform FSL for image classification, using a visual representation from the backbone model and patterns generated by a self-attention based explainable module. The representation weighted by patterns only includes a minimum number of distinguishable features and the visualized patterns can serve as an informative hint on the transferred knowledge. On three mainstream datasets, experimental results prove that the proposed method can enable satisfying explainability and achieve high classification results. Code is available at https://github.com/wbw520/MTUNet.