Bayesian Regression Trees for High-Dimensional Prediction and Variable Selection
Bayesian Regression Trees for High-Dimensional Prediction and Variable Selection
复制标题
DOI：
10.1080/01621459.2016.1264957
复制
发表时间：
2018-01-01
影响因子：
3.7
通讯作者：
Linero, Antonio R.
中科院分区：
文献类型：
--
作者：
Linero, Antonio R.
决策树集成是一种非常流行的工具，用于在非参数回归问题中获得高质量的预测。然而，未经修改，许多常用的决策树集成方法不适应稀疏的制度，其中预测的数量大于观察的数量。最近的一个研究流涉及决策树集成的生成概率模型的动机，最有影响力的方法是贝叶斯加性回归树（BART）框架的建设。在这篇文章中，我们对这个问题采取贝叶斯的观点，并展示了如何构建先验的决策树集成，能够适应稀疏的预测，通过放置一个稀疏诱导Dirichlet超先验的回归树先验的分裂比例。我们描述了模型中预测因子的数量的渐近分布，并展示了如何将此先验可以很容易地纳入现有的马尔可夫链蒙特卡罗方案。我们证明了我们的方法为每个预测因子产生了有用的后验包含概率，并说明了我们的方法相对于模拟和真实的数据集上的其他决策树集成方法的有用性。
Decision tree ensembles are an extremely popular tool for obtaining high-quality predictions in nonparametric regression problems. Unmodified, however, many commonly used decision tree ensemble methods do not adapt to sparsity in the regime in which the number of predictors is larger than the number of observations. A recent stream of research concerns the construction of decision tree ensembles that are motivated by a generative probabilistic model, the most influential method being the Bayesian additive regression trees (BART) framework. In this article, we take a Bayesian point of view on this problem and show how to construct priors on decision tree ensembles that are capable of adapting to sparsity in the predictors by placing a sparsity-inducing Dirichlet hyperprior on the splitting proportions of the regression tree prior. We characterize the asymptotic distribution of the number of predictors included in the model and show how this prior can be easily incorporated into existing Markov chain Monte Carlo schemes. We demonstrate that our approach yields useful posterior inclusion probabilities for each predictor and illustrate the usefulness of our approach relative to other decision tree ensemble approaches on both simulated and real datasets.