AttCAT: Explaining Transformers via Attentive Class Activation Tokens
AttCAT: Explaining Transformers via Attentive Class Activation Tokens
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Yao Qiang;Deng Pan;Chengyin Li;X. Li;Rhongho Jang;D. Zhu
中科院分区：
文献类型：
--
作者：
Yao Qiang;Deng Pan;Chengyin Li;X. Li;Rhongho Jang;D. Zhu
变形金刚在各种自然语言处理和计算机视觉任务中改进了最先进的技术。然而，变形金刚模型的成功还没有得到适当的解释。由于以下原因，现有的解释技术要么剖析自我注意机制，要么基于梯度归因，不一定能真实地解释变形金刚的内部工作机制：第一，没有考虑特征值大小的单独注意权重不足以揭示自我注意机制；第二，尽管大多数变形金刚解释技术利用自我注意模块，但跳过连接模块作为变形金刚中信息流的重要组成部分，尚未被充分利用；第三，基于梯度的个体特征归因在解释模型输出时没有考虑特征之间的交互作用。为了解决上述问题，我们提出了一种新的Transformer解释技术，该技术通过注意力类激活令牌(AttCAT)，利用编码的特征、它们的梯度和它们的注意力权重来生成对Transformer输出的忠实和自信的解释。大量的实验证明了AttCAT的优越性能，它很好地概括了不同的Transformer架构、评估指标、数据集和任务，以及基线方法。我们的代码请访问：https://github.com/qiangyao1988/AttCAT.
Transformers have improved the state-of-the-art in various natural language processing and computer vision tasks. However, the success of the Transformer model has not yet been duly explained. Current explanation techniques, which dissect either the self-attention mechanism or gradient-based attribution, do not necessarily provide a faithful explanation of the inner workings of Transformers due to the following reasons: first, attention weights alone without considering the magnitudes of feature values are not adequate to reveal the self-attention mechanism; second, whereas most Transformer explanation techniques utilize self-attention module, the skip-connection module, contributing a significant portion of information flows in Transformers, has not yet been sufficiently exploited in explanation; third, the gradient-based attribution of individual feature does not incorporate interaction among features in explaining the model’s output. In order to tackle the above problems, we propose a novel Transformer explanation technique via attentive class activation tokens, aka, AttCAT, leveraging encoded features, their gradients, and their attention weights to generate a faithful and confident explanation for Transformer’s output. Extensive experiments are conducted to demonstrate the superior performance of AttCAT, which generalizes well to different Transformer architectures, evaluation metrics, datasets, and tasks, to the baseline methods. Our code is available at: https://github.com/qiangyao1988/AttCAT.