Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification
Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification
复制标题
DOI：
10.18653/v1/d19-1046
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Vivian Lai;Zheng Jon Cai;Chenhao Tan
中科院分区：
文献类型：
--
作者：
Vivian Lai;Zheng Jon Cai;Chenhao Tan
特征重要性通常用于解释机器预测。虽然可以通过各种方法从机器学习模型中导出特征重要性，但通过不同方法获得的特征重要性的一致性仍然没有得到充分研究。在这项工作中，我们系统地比较了模型中内置机制的特征重要性，如注意力值和近似模型行为的事后方法，如LIME。使用文本分类作为测试平台，我们发现：1）无论使用哪种方法，传统模型（如SVM和XGBoost）的重要特征都比深度学习模型更相似; 2）事后方法倾向于为两个模型生成比内置方法更相似的重要特征。我们进一步展示了这种相似性如何在不同的实例中变化。值得注意的是，当两个模型在预测标签上一致时，重要特征并不总是比它们不一致时更相似。
Feature importance is commonly used to explain machine predictions. While feature importance can be derived from a machine learning model with a variety of methods, the consistency of feature importance via different methods remains understudied. In this work, we systematically compare feature importance from built-in mechanisms in a model such as attention values and post-hoc methods that approximate model behavior such as LIME. Using text classification as a testbed, we find that 1) no matter which method we use, important features from traditional models such as SVM and XGBoost are more similar with each other, than with deep learning models; 2) post-hoc methods tend to generate more similar important features for two models than built-in methods. We further demonstrate how such similarity varies across instances. Notably, important features do not always resemble each other better when two models agree on the predicted label than when they disagree.