VtNet: A neural network with variable importance assessment
VtNet: A neural network with variable importance assessment
复制标题
DOI：
10.1002/sta4.325
复制
发表时间：
2020-10
期刊：
影响因子：
1.7
通讯作者：
Lixiang Zhang;Lin Lin-Lin;Jia Li
中科院分区：
文献类型：
--
作者：
Lixiang Zhang;Lin Lin-Lin;Jia Li
许多神经网络的结构在很大程度上依赖于与变量相关的底层网格，例如，图像中的像素网格。对于没有网格结构的一般生物医学数据，通常使用多层感知器(MLP)和深度信念网络(DBN)。然而，在这些网络中，变量在网络结构意义上是同质的，很难评估它们各自的重要性。在本文中，我们提出了一种新的神经网络，称为可变块树网络(VtNet)，其结构由一棵底层树决定，每个节点对应一个变量子集。树是从数据中学习的，以最好地捕捉变量之间的因果关系。VtNet为每个树节点包含一个类似长短期记忆(LSTM)的单元。每个单元的输入和忘记门控制着通过节点的信息流，它们被用来定义变量的重要性分数。为了验证定义的重要性分数，VtNet使用较小的树进行训练，其中删除了低分数的变量。假设检验表明，得分越高的变量对分类的影响越大。从变量选择的角度与《随机森林》中定义的变量重要性分数进行了比较。我们的实验表明，VtNet在分类准确率方面具有很强的竞争力，并且经常可以通过删除重要性分数较低的变量来提高准确率。
The architectures of many neural networks rely heavily on the underlying grid associated with the variables, for instance, the lattice of pixels in an image. For general biomedical data without a grid structure, the multi‐layer perceptron (MLP) and deep belief network (DBN) are often used. However, in these networks, variables are treated homogeneously in the sense of network structure; and it is difficult to assess their individual importance. In this paper, we propose a novel neural network called Variable‐block tree Net (VtNet) whose architecture is determined by an underlying tree with each node corresponding to a subset of variables. The tree is learned from the data to best capture the causal relationships among the variables. VtNet contains a long short‐term memory (LSTM)‐like cell for every tree node. The input and forget gates of each cell control the information flow through the node, and they are used to define a significance score for the variables. To validate the defined significance score, VtNet is trained using smaller trees with variables of low scores removed. Hypothesis tests are conducted to show that variables of higher scores influence classification more strongly. Comparison is made with the variable importance score defined in Random Forest from the aspect of variable selection. Our experiments demonstrate that VtNet is highly competitive in classification accuracy and can often improve accuracy by removing variables with low significance scores.