Adaptive Concentration of Regression Trees, with Application to Random Forests
Adaptive Concentration of Regression Trees, with Application to Random Forests
复制标题
DOI：
--
复制
发表时间：
2015-03
期刊：
影响因子：
--
通讯作者：
Stefan Wager;G. Walther
中科院分区：
文献类型：
--
作者：
Stefan Wager;G. Walther
研究了回归树和回归林的预测面收敛性。为了支持我们的分析，我们为回归树引入了自适应集中的概念。这种方法将树训练分解为一个模型选择阶段，在这个阶段我们选择树的分裂，然后是一个模型拟合阶段，在这个阶段我们找到与这些分裂一致的最佳回归模型。然后，我们表明，拟合的回归树以相同的分割集中在最优预测器周围：随着d和n变大，差异在整个回归表面上均匀地以sqrt(log（d) log(n)/k）的阶为界，其中d是特征空间的维数，n是训练样例的数量，k是每棵树的最小叶大小。我们还为这种自适应浓度声明提供了速率匹配的下限。从实际角度来看，我们的结果使我们能够证明高维自适应生长森林的一致性结果，并对树叶定义的亚群进行Berk等[2013]意义上的有效后选择推理。
We study the convergence of the predictive surface of regression trees and forests. To support our analysis we introduce a notion of adaptive concentration for regression trees. This approach breaks tree training into a model selection phase in which we pick the tree splits, followed by a model fitting phase where we find the best regression model consistent with these splits. We then show that the fitted regression tree concentrates around the optimal predictor with the same splits: as d and n get large, the discrepancy is with high probability bounded on the order of sqrt(log(d) log(n)/k) uniformly over the whole regression surface, where d is the dimension of the feature space, n is the number of training examples, and k is the minimum leaf size for each tree. We also provide rate-matching lower bounds for this adaptive concentration statement. From a practical perspective, our result enables us to prove consistency results for adaptively grown forests in high dimensions, and to carry out valid post-selection inference in the sense of Berk et al. [2013] for subgroups defined by tree leaves.