Confidence intervals for random forests: the jackknife and the infinitesimal jackknife
Confidence intervals for random forests: the jackknife and the infinitesimal jackknife
复制标题
DOI：
10.5555/2627435.2638587
复制
发表时间：
2013-11
期刊：
影响因子：
--
通讯作者：
Stefan Wager;T. Hastie;B. Efron
中科院分区：
文献类型：
--
作者：
Stefan Wager;T. Hastie;B. Efron
我们研究了袋装学习者和随机森林的预测的变异性，并展示了如何估计这些方法的标准误差。我们的工作建立在Efron（1992，2013）提出的基于折刀和无穷小折刀（IJ）的装袋方差估计的基础上。在实践中，袋装预测因子是使用有限数量的自举重复B来计算的，并且使用大的B来工作可能在计算上是昂贵的。将折刀和IJ估计直接应用于装袋需要B = Θ（n1.5）自助重复才能收敛，其中n是训练集的大小。我们提出了改进的版本，只需要B = Θ（n）重复。此外，我们表明，IJ估计需要1.7倍以下的自举重复比刀切，以达到给定的精度。最后，我们研究了刀切和IJ方差估计本身的抽样分布。我们用多个实验和模拟研究来说明我们的发现。
We study the variability of predictions made by bagged learners and random forests, and show how to estimate standard errors for these methods. Our work builds on variance estimates for bagging proposed by Efron (1992, 2013) that are based on the jackknife and the infinitesimal jackknife (IJ). In practice, bagged predictors are computed using a finite number B of bootstrap replicates, and working with a large B can be computationally expensive. Direct applications of jackknife and IJ estimators to bagging require B = Θ(n1.5) bootstrap replicates to converge, where n is the size of the training set. We propose improved versions that only require B = Θ(n) replicates. Moreover, we show that the IJ estimator requires 1.7 times less bootstrap replicates than the jackknife to achieve a given accuracy. Finally, we study the sampling distributions of the jackknife and IJ variance estimates themselves. We illustrate our findings with multiple experiments and simulation studies.