Variance reduction in purely random forests
Variance reduction in purely random forests
复制标题
DOI：
10.1080/10485252.2012.677843
复制
发表时间：
2012-01-01
影响因子：
1.2
通讯作者：
Genuer, Robin
中科院分区：
文献类型：
--
作者：
Genuer, Robin
Leo Breiman 于 2001 年提出的随机森林 (RF) 是一种非常有效的统计方法。该方法机理复杂，给理论分析带来困难。因此，人们考虑了 RF 的简化版本，称为纯 RF (PRF)，理论上更容易处理。在本文中，我们研究了此类森林的方差。首先，我们展示了一个通用的上限，它强调了森林减少了方差的事实。然后我们介绍 PRF 的一个简单变体，我们称之为纯一致 RF。对于这种变体以及在一维预测器空间的回归问题的背景下，我们表明随机树和 RF 都达到了极小极大收敛率。此外，我们证明，与随机树相比，RF 通过将估计方差减少四分之三来提高准确性。
Random forests (RFs), introduced by Leo Breiman in 2001, are a very effective statistical method. The complex mechanism of the method makes theoretical analysis difficult. Therefore, simplified versions of RF, called purely RFs (PRF), which can be theoretically handled more easily, have been considered. In this paper, we study the variance of such forests. First, we show a general upper bound which emphasises the fact that a forest reduces the variance. We then introduce a simple variant of PRFs, that we call purely uniformly RFs. For this variant and in the context of regression problems with a one-dimensional predictor space, we show that both random trees and RFs reach minimax rate of convergence. In addition, we prove that compared with random trees, RFs improve accuracy by reducing the estimator variance by a factor of three-fourths.