Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluation
Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluation
复制标题
DOI：
10.48550/arxiv.2302.12893
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
N. Jethani;A. Saporta;R. Ranganath
中科院分区：
文献类型：
--
作者：
N. Jethani;A. Saporta;R. Ranganath
特征属性方法识别输入的哪些特征对模型的输出影响最大。大多数广泛使用的特征属性方法（如SHAP、LIME和Grad-CAM）都是“类依赖“方法，因为它们生成的特征属性向量是类的函数。在这项工作中，我们证明了类相关的方法可以“泄漏”有关所选类的信息，使该类看起来比实际情况更有可能。因此，最终用户在解释依赖于类的方法生成的解释时会冒得出错误结论的风险。相比之下，我们引入了“分布感知”的方法，这些方法有利于解释保持标签的分布接近其分布，因为输入的所有特征都是如此。我们介绍SHAP-KL和FastSHAP-KL，这两种基线分布感知方法可以计算Shapley值。最后，我们对不同高维数据类型（图像、生物信号和文本）的三个临床数据集进行了七种类依赖和三种分布感知方法的综合评估。
Feature attribution methods identify which features of an input most influence a model's output. Most widely-used feature attribution methods (such as SHAP, LIME, and Grad-CAM) are"class-dependent"methods in that they generate a feature attribution vector as a function of class. In this work, we demonstrate that class-dependent methods can"leak"information about the selected class, making that class appear more likely than it is. Thus, an end user runs the risk of drawing false conclusions when interpreting an explanation generated by a class-dependent method. In contrast, we introduce"distribution-aware"methods, which favor explanations that keep the label's distribution close to its distribution given all features of the input. We introduce SHAP-KL and FastSHAP-KL, two baseline distribution-aware methods that compute Shapley values. Finally, we perform a comprehensive evaluation of seven class-dependent and three distribution-aware methods on three clinical datasets of different high-dimensional data types: images, biosignals, and text.