Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning
Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning
复制标题
DOI：
10.1501/0000000035
复制
发表时间：
2011-01-01
影响因子：
36.5
通讯作者：
Konukoglu, Ender
中科院分区：
文献类型：
--
作者：
Criminisil, Antonio;Shotton, Jamie;Konukoglu, Ender
本文提出了一个统一的、有效的随机决策森林模型，它可以应用于许多非线性学习、计算机视觉和医学图像分析任务，我们的模型扩展了现有的基于森林的技术，因为它在同一个决策森林框架下统一了分类、回归、密度估计、显式学习、半监督学习和主动学习。本文的主要贡献在于：（1）提出了一个适用于各种学习任务的统一的、概率性的、有效的模型，该模型可以应用于离散或连续的、有标签或无标签的数据，也可以应用于判别式或生成式的学习任务;（2）证明了分类森林的边际最大化性质;（3）讨论了概率回归森林与其他非线性回归算法的比较;（4）引入密度森林估计概率密度函数;（5）提出了一种有效的密度森林抽样算法;（6）引入流形森林进行非线性降维;（7）提出了新的转换学习和主动学习算法。最后，我们讨论了随机蕨类植物和极端随机树等替代方案是如何从我们更一般的森林模型中产生的。本文针对希望学习决策森林基础知识的学生以及对新贡献感兴趣的研究人员。它以结构化的方式提出了基本和新颖的概念，并提供了许多说明性的示例和实际应用。与最先进的算法，如支持向量机，提升和高斯过程的彻底比较，并讨论了相对的优点和缺点。大量的合成实例和现有的商业应用证明了该模型的有效性和灵活性。
This review presents a unified, efficient model of random decision forests which can be applied to a number of nlaclrine learning, computer vision', and medical image analysis tasks.Our model extends existing forest-based techniques as it unifies classification, regression, density estimation, manifOld learning, semi supervised learning, and active learning under the same decision forest framework. This gives us the opportunity to write and optimize the core implementation only once, with application to many diverse tasks.The proposed model may be used both in a discriminative or generative way and may be applied to discrete or continuous, labeled or unlabeled data.The main contributions of this review are: (1) Proposing a unified, probabilistic and efficient model for a variety of learning tasks; (2) Demonstrating margin-maximizing properties of classification forests; (3) Discussing probabilistic regression forests in comparison with other nonlinear regression algorithms; (4) Introducing density forests for estimating probability density functions; (5) Proposing an efficient algorithm for sampling from a density forest; (6) Introducing manifold forests for nonlinear dimensionality reduction; (7) Proposing new algorithms for transductive learning and active learning. Finally, we discuss how alternatives such as random ferns and extremely randomized trees stem from our more general forest model.This document is directed at both students who wish to learn the basics of decision forests, as well as researchers interested in the new contributions. It presents both fundamental and novel concepts in a structured way, with many illustrative examples and real-world applications. Thorough comparisons with state-of-the-art algorithms such as support vector machines, boosting and Gaussian processes are presented and relative advantages and disadvantages discussed. The many synthetic examples and existing commercial applications demonstrate the validity of the proposed model and its flexibility.