A less-greedy two-term Tsallis Entropy Information Metric approach for decision tree classification
A less-greedy two-term Tsallis Entropy Information Metric approach for decision tree classification
复制标题
用于决策树分类的不太贪婪的两项 Tsallis 熵信息度量方法
DOI：
10.1016/j.knosys.2016.12.021
复制
发表时间：
2017-03-15
影响因子：
8.8
通讯作者：
Wu, Jia
中科院分区：
文献类型：
--
作者：
Wang, Yisen;Xia, Shu-Tao;Wu, Jia
高效决策树的构造由于其简单性和灵活性仍然是机器学习中的一个关键主题。许多启发式算法被提出来构造近似最优的决策树。然而，它们中的大多数都是贪婪算法，其缺点是只能获得局部最优值。此外，他们使用的传统分裂标准，如香农熵，增益比和基尼指数，是基于一个术语，缺乏适应性的不同数据集。针对上述问题，提出了一种新的分割准则和决策树构造方法，并提出了一种不太贪婪的两项Tsallis熵信息度量（TEIM）算法。首先，新的分裂准则是基于两项Tsallis条件熵，这是优于传统的一项分裂准则。其次，新的树的构建是基于一个两阶段的方法，减少了贪婪，并在一定程度上避免局部最优。TEIM算法利用了两项Tsallis熵的泛化能力和两阶段算法的低贪婪性。在UCI数据集上的实验结果表明，与现有的决策树算法相比，TEIM算法在统计学上具有更好的决策树性能，并且对噪声具有更强的鲁棒性. (C)2016爱思唯尔B.V.保留所有权利。
The construction of efficient and effective decision trees remains a key topic in machine learning because of their simplicity and flexibility. A lot of heuristic algorithms have been proposed to construct near optimal decision trees. Most of them, however, are greedy algorithms that have the drawback of obtaining only local optimums. Besides, conventional split criteria they used, e.g. Shannon entropy, Gain Ratio and Gini index, are based on one-term that lack adaptability to different datasets. To address the above issues, we propose a less-greedy two-term Tsallis Entropy Information Metric (TEIM) algorithm with a new split criterion and a new construction method of decision trees. Firstly, the new split criterion is based on two-term Tsallis conditional entropy, which is better than conventional one-term split criteria. Secondly, the new tree construction is based on a two-stage approach that reduces the greediness and avoids local optimum to a certain extent. The TEIM algorithm takes advantages of the generalization ability of two term Tsallis entropy and the low greediness property of two-stage approach. Experimental results on UCI datasets indicate that, compared with the state-of-the-art decision trees algorithms, the TEIM algorithm yields statistically significantly better decision trees and is more robust to noise. (C) 2016 Elsevier B.V. All rights reserved.