Learning and understanding based on neural network trees

在机器学习中，大致有两种类型的模型，即符号模型和非符号模型。前者是可以理解的，但不善于在变化的环境中学习。另一方面，后者善于学习，但学习的结果是不可理解的。本研究的目的是提出一种将神经网络(NN)和决策树(DT)相结合的同时学习和理解的方法。本研究使用的学习模型称为神经网络树(NNTree)。针对NNTrees存在的主要问题是诱导成本高、结果不能理解等问题，在本项目的第一年，我们提出了一种启发式方法来定义分配给树内部节点的数据的教师信号。基于这种方法，内部节点中的神经网络可以使用监督学习来训练，而不是像以前那样使用进化学习。这可以大大降低诱导成本。提高…的可理解性在NNTrees的基础上，我们提出了在每个内部节点中使用最近邻分类器(NNC)来代替NN。我们称这种模型为NNC-Tree。事实上，如果我们把每个原型都看作是一个“先例”，NNC就可以提供非常容易理解的决策规则。为了设计内部节点中的NNC，我们可以使用赵之前提出的R4规则。实验结果表明，该方法能够生成准确、紧凑、易理解的NNC树。第二年，我们提出了两种降低NNC树诱导成本的方法。第一种方法是“注意学习法”，第二种是“降维方法”。在第一种方法中，在基于R4规则的学习中，我们更加关注困难的数据，而跳过容易的数据。这可以降低80%以上的NNC-Tree诱导成本。在第二种方法中，我们尝试使用主成分分析对问题进行降维。如果原始问题空间的维度很高(如图像识别)，该方法也可以大大降低成本。未来，我们希望应用NNC-树来解决数据不完整(属性缺失的数据)的问题。我们还想考虑在树的归纳过程中属性的重要性和成本。此外，我们希望将归纳结果可视化，并试图使NNC-Tree更易于理解。较少

In machine learning, there are roughly two types of models, namely symbolic and non-symbolic. The former is comprehensible but not good at learning in changing environment. The latter, on the other hand, is good at learning but the learned results are not comprehensible. The purpose of this research is to propose a way that can learn and understand simultaneously, by combining neural network (NN) and decision tree (DT). The learning model used in this research is called the neural network tree (NNTree). The main problems in using NNTrees are that the induction cost is high and the results are not comprehensible.In the first year of this project, we proposed a heuristic method for defining the teacher signals for the data assigned to an internal node of the tree. Based on this method, NNs in the internal nodes can be trained using supervised learning, rather than evolutionary learning, as we did before. This can reduce the cost for induction greatly. To increase the comprehensibility of … More the NNTrees, we proposed to use a nearest neighbor classifier (NNC) in each internal node instead of an NN. We call this model the NNC-Tree. In fact, an NNC can provide very comprehensible decision rules if we consider each prototype as a "precedent". To design the NNCs in the internal nodes, we can use the R4-rule proposed by Zhao earlier. Experimental results show that the proposed method can induce accurate, compact, and comprehensible NNC-Trees.In the second year, we proposed two methods for reducing the induction cost. The first method is the "attentional learning method", and the second is the "dimensionality reduction method". In the first method, we pay more attention to difficult data and skip easy data during the R4-rule based learning. This can reduce more than 80% of the cost for NNC-Tree induction. In the second method, we try to reduce the dimensionality of the problem using principal component analysis. If the dimensionality of the original problem space is very high (e.g., image recognition), this method can also reduce the cost greatly.In the future, we would like to apply NNC-Trees to solve problems with incomplete data (data with missing attributes). We would also like to consider the importance and costs of the attributes during induction of the tree. Further, we would like to visualize the induction results, and try to make the NNC-Tree more comprehensible. Less