Sparse oblique decision trees: a tool to understand and manipulate neural net features
Sparse oblique decision trees: a tool to understand and manipulate neural net features
复制标题
稀疏倾斜决策树：理解和操纵神经网络特征的工具
DOI：
10.1007/s10618-022-00892-7
复制
发表时间：
2023
影响因子：
4.8
通讯作者：
Zharmagambetov, Arman
中科院分区：
文献类型：
--
作者：
Hada, Suryabhan Singh;Carreira-Perpiñán, Miguel Á.;Zharmagambetov, Arman
随着深度网络在实际应用中的广泛应用，人们越来越希望了解这种黑箱方法是如何以及为什么进行预测的。很多工作都集中在理解输入模式的哪一部分（比如图像）负责预测特定的类，以及如何操纵输入来预测不同的类。相反，我们专注于理解由神经网络计算的哪些内部特征负责特定的类。我们通过在决策节点处具有稀疏权向量的倾斜决策树来模拟神经网络的一部分来实现这一点。使用最近提出的树交替优化（TAO）算法，我们能够学习既高度准确又可解释的树。这样的树可以忠实地模仿它们所取代的神经网络部分，因此它们可以提供对深度网络黑箱的洞察。此外，我们表明我们可以很容易地操纵神经网络特征，以使网络预测或不预测给定的类，从而表明有可能在特征级别上进行对抗性攻击。这些洞察和操作适用于整个训练和测试集，而不仅仅是局部（单实例）级别。我们在使用LeNet5和VGG网络的MNIST和ImageNet数据集中有力地证明了这一点。
The widespread deployment of deep nets in practical applications has lead to a growing desire to understand how and why such black-box methods perform prediction. Much work has focused on understanding what part of the input pattern (an image, say) is responsible for a particular class being predicted, and how the input may be manipulated to predict a different class. We focus instead on understanding which of the internal features computed by the neural net are responsible for a particular class. We achieve this by mimicking part of the neural net with an oblique decision tree having sparse weight vectors at the decision nodes. Using the recently proposed Tree Alternating Optimization (TAO) algorithm, we are able to learn trees that are both highly accurate and interpretable. Such trees can faithfully mimic the part of the neural net they replaced, and hence they can provide insights into the deep net black box. Further, we show we can easily manipulate the neural net features in order to make the net predict, or not predict, a given class, thus showing that it is possible to carry out adversarial attacks at the level of the features. These insights and manipulations apply globally to the entire training and test set, not just at a local (single-instance) level. We demonstrate this robustly in the MNIST and ImageNet datasets with LeNet5 and VGG networks.
倾斜决策树的反事实解释：精确、高效的算法
DOI：
10.1609/aaai.v35i8.16851
发表时间：
2021
期刊：
ArXiv
影响因子：
--
作者：
Miguel 'A. Carreira;Suryabhan Singh Hada
通讯作者：
Suryabhan Singh Hada
DOI：
10.1109/5.784232
发表时间：
1999
期刊：
Proc. IEEE
影响因子：
--
作者：
C. A. Jensen;R. Reed;R. Marks;M. El;Jae;R. Miyamoto;G. Anderson;C. Eggen
通讯作者：
C. Eggen
影响因子：
2.9
作者：
Montavon, Gregoire;Samek, Wojciech;Mueller, Klaus-Robert
通讯作者：
Mueller, Klaus-Robert