Scalable Rule-Based Representation Learning for Interpretable Classification
Scalable Rule-Based Representation Learning for Interpretable Classification
复制标题
用于可解释分类的可扩展的基于规则的表示学习
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Jianyong Wang
中科院分区：
文献类型：
--
作者：
Zhuo Wang;Wei Zhang;Ning Liu;Jianyong Wang
基于规则的模型，例如决策树，由于其透明的内部结构和良好的模型表达能力，被广泛应用于要求模型可解释性高的场景中。然而，基于规则的模型很难优化，特别是在大数据集上，因为它们的参数和结构离散。集成方法和模糊/软规则通常用于提高性能，但它们牺牲了模型的可解释性。为了获得良好的可扩展性和可解释性，我们提出了一种新的分类器，称为基于规则的表示学习器(RRL)，它自动学习用于数据表示和分类的可解释的非模糊规则。为了有效地训练不可微RRL，我们将其投影到一个连续的空间，并提出了一种新的训练方法，称为梯度嫁接，该方法可以直接使用梯度下降来优化离散模型。设计了一种改进的逻辑激活函数设计，以增加RRL的可扩展性，使其能够端到端地离散连续特征。在9个小数据集和4个大数据集上的穷举实验表明，RRL的性能优于竞争的可解释方法，并且可以很容易地进行调整，以获得不同场景下分类精度和模型复杂性之间的权衡。我们的代码请访问：https://github.com/12wang3/rrl.
Rule-based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a new classifier, named Rule-based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation and classification. To train the non-differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability of RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on nine small and four large data sets show that RRL outperforms the competitive interpretable approaches and can be easily adjusted to obtain a trade-off between classification accuracy and model complexity for different scenarios. Our code is available at: https://github.com/12wang3/rrl.