Neural Generators of Sparse Local Linear Models for Achieving both Accuracy and Interpretability
Neural Generators of Sparse Local Linear Models for Achieving both Accuracy and Interpretability
复制标题
DOI：
10.1016/j.inffus.2021.11.009
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
Yuya Yoshikawa;Tomoharu Iwata
中科院分区：
文献类型：
--
作者：
Yuya Yoshikawa;Tomoharu Iwata
为了可靠性，机器学习方法所做的预测必须是人类可以解释的。一般来说，深度神经网络（DNN）可以提供准确的预测，尽管很难解释为什么DNN会获得这样的预测。另一方面，线性模型的解释很容易，尽管它们的预测性能很低，因为真实世界的数据通常本质上是非线性的。为了将DNN的高预测性能和线性模型的高可解释性的优点联合收割机结合到单个模型中，我们提出了稀疏局部线性模型（NGSLL）的神经生成器。稀疏局部线性模型具有很高的灵活性，因为它们可以近似非线性函数。NGSLL使用DNN为每个样本生成稀疏线性权重，DNN采用每个样本的原始表示（例如，字序列）及其简化表示（例如，词袋）作为输入。通过从原始表示中提取特征，权重可以包含丰富的信息并实现高预测性能。此外，预测是可解释的，因为它是通过简化表示和稀疏权重之间的内积获得的，其中NGSLL中的门模块仅选择少量权重。在图像，文本和表格数据集的实验中，我们证明了NGSLL的有效性定量和定性评估的预测性能和可视化生成的权重。
For reliability, it is important for the predictions made by machine learning methods to be interpretable by humans. In general, deep neural networks (DNNs) can provide accurate predictions, although it is difficult to interpret why such predictions are obtained by the DNNs. On the other hand, interpretation of linear models is easy, although their predictive performance is low because real-world data are often intrinsically non-linear. To combine both the benefits of the high predictive performance of DNNs and the high interpretability of linear models into a single model, we propose neural generators of sparse local linear models (NGSLL). Sparse local linear models have high flexibility because they can approximate non-linear functions. NGSLL generates sparse linear weights for each sample using DNNs that take the original representations of each sample (e.g., word sequence) and their simplified representations (e.g., bag-of-words) as input. By extracting features from the original representations, the weights can contain rich information and achieve a high predictive performance. In addition, the prediction is interpretable because it is obtained through the inner product between the simplified representations and the sparse weights, where only a small number of weights are selected by our gate module in NGSLL. In experiments on image, text and tabular datasets, we demonstrate the effectiveness of NGSLL quantitatively and qualitatively by evaluating the prediction performance and visualizing generated weights.