Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance
Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance
复制标题
DOI：
--
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
L. Mentch;Siyu Zhou
中科院分区：
文献类型：
--
作者：
L. Mentch;Siyu Zhou
随着数据的规模、复杂性和可用性不断增长，科学家越来越依赖黑盒学习算法，这些算法通常可以用最少的先验模型规范提供准确的预测。像随机森林这样的工具已经拥有现成的成功记录，甚至提供了各种策略来分析变量之间的潜在关系。在这里，受最近对随机森林行为的见解的启发，我们引入了增强装袋（AugBagg）的简单想法，该过程的运行方式与经典装袋和随机森林相同，但它在包含额外随机生成的噪声特征的更大的增强空间上运行。令人惊讶的是，我们证明了在模型中包含额外噪声变量的简单行为可以显着提高样本外预测准确性，有时甚至优于经过优化调整的传统随机森林。因此，基于改进的模型精度的可变重要性的直观概念可能存在严重缺陷，因为即使是纯粹的随机噪声也通常会被认为具有统计显着性。提供了有关真实数据和合成数据的大量演示以及建议的解决方案。
As the size, complexity, and availability of data continues to grow, scientists are increasingly relying upon black-box learning algorithms that can often provide accurate predictions with minimal a priori model specifications. Tools like random forests have an established track record of off-the-shelf success and even offer various strategies for analyzing the underlying relationships among variables. Here, motivated by recent insights into random forest behavior, we introduce the simple idea of augmented bagging (AugBagg), a procedure that operates in an identical fashion to classical bagging and random forests, but which operates on a larger, augmented space containing additional randomly generated noise features. Surprisingly, we demonstrate that this simple act of including extra noise variables in the model can lead to dramatic improvements in out-of-sample predictive accuracy, sometimes outperforming even an optimally tuned traditional random forest. As a result, intuitive notions of variable importance based on improved model accuracy may be deeply flawed, as even purely random noise can routinely register as statistically significant. Numerous demonstrations on both real and synthetic data are provided along with a proposed solution.