Extensions of the random forest algorithm and a simple inference procedure for machine learning approaches

该方案是对随机森林（RF）算法的几个扩展方案的更新方案，该方案开发了解决实际相关问题的算法。新提案由三部分组成。射频方法学是前两部分的核心，在第三部分中所起的作用较小。在第一部分中，我们将开发针对多类结果量身定制的RF变体。虽然后者可能会提高RFs的预测性能，但这种变体的一个明显优势是其可变重要性度量将更好地解释结果的多类别性质。这填补了一个重要的空白，因为到目前为止，似乎没有针对多类别结果的既定可变重要性措施。所提出的射频变体采用了开发的多样性森林算法。我们将在第二部分开发另一个RF变体，即全球森林。全球森林中的树木将通过考虑相互依存的分裂来改进经典树木的结构，这可以更好地利用协变量之间的相互作用效应。预计这将导致通过与其他协变量的相互作用产生强烈影响的协变量的改进变量重要性测量值，并且还可能提高预测性能。第三部分将为机器学习（ML）算法开发一个简单的通用推理程序。这一程序的提出是基于越来越多的关注，即从ML模型中得出的结论通常被视为固定的，而不质疑其统计意义。该方法具有保守性、计算可行性、适用于任何机器学习方法、易于实现、直观易懂等特点。它使用自举抽样，但与经典的自举分析相比，计算成本要低得多。对于本提案的第一部分和第二部分，我们将进行广泛的模拟研究和真实数据分析，以研究所提出的RF变体的特性。这两个变体都将在我们的R包“多样性森林”中实现。第三部分提出的推理方法的关键性质可以很容易地解析推导出来。因此，本部分仅进行说明性分析。在这里，我们将演示所提出的方法对ML中通常未被经典推理技术涵盖的各种概念的适用性。我们将在所有这些说明性分析中使用rf，只有一个例外。

This proposal is a renewal proposal of the project in which several extensions of the random forest (RF) algorithm solving practically relevant problems were developed. The new proposal consists of three parts. RF methodology is the core of the first two parts and plays a less important role in the third part. In the first part, we will develop an RF variant tailored to multi-class outcomes. While the latter may improve the predictive performance of RFs, a clear advantage of this variant will be that its variable importance measure will better account for the multi-class nature of the outcome. This fills an important gap, as to date there appear to be no established variable importance measures tailored to multi-class outcomes. The proposed RF variant uses the diversity forest algorithm developed. We will develop another RF variant, global forests, in the second part. The trees in global forests will improve on the structure of classical trees by considering interdependent splits, which allows to better exploit interaction effects between the covariates. This is expected to lead to improved variable importance measure values for covariates that have a strong effect through their interaction with other covariates, and it may also improve predictive performance. The third part will develop a simple general inference procedure for machine learning (ML) algorithms. This procedure is proposed in light of growing concern that conclusions drawn from ML models are often treated as fixed without questioning their statistical significance. The proposed procedure is conservative, computationally feasible, applicable to any ML method, very easy to implement and intuitively understandable. It uses bootstrap sampling, but is dramatically less computationally expensiv than classical bootstrap analysis. For the first and second part of this proposal, we will perform extensive simulation studies and real data analyses to study the properties of the proposed RF variants. Both variants will be implemented in our R package 'diversityForest'. The key properties of the inference approach proposed in the third part can be easily derived analytically. Therefore, only illustrative analyses will be performed in this part. Here, we will demonstrate the applicability of the proposed approach to various concepts in ML that are usually not covered by classical inference techniques. We will use RFs in all but one of these illustrative analyses.