Provable Boolean interaction recovery from tree ensemble obtained via random forests.
Provable Boolean interaction recovery from tree ensemble obtained via random forests.
复制标题
DOI：
10.1073/pnas.2118636119
复制
发表时间：
2022-05-31
影响因子：
11.1
通讯作者：
中科院分区：
文献类型：
--
作者：
随机森林（RF）是在预测准确性方面最成功的机器学习算法之一。然而，在许多领域问题中，主要目标不是预测，而是理解数据生成过程，特别是找到重要的特征和特征交互。有强有力的经验证据表明，基于RF的方法，特别是迭代RF（iRF），是非常成功的检测功能的相互作用。在这项工作中，我们提出了一个生物动机，布尔交互模型。使用这个模型，我们补充了现有的经验证据与理论证据的能力，iRF型的方法来选择理想的相互作用。我们的理论分析也产生了更深的见解决策树算法的一般交互选择机制和特征子采样的重要性。随机森林（RF）在预测性能方面处于监督机器学习的前沿，特别是在基因组学方面。迭代RF（iRF）使用来自迭代修改的RF的树集成来获得特征的预测性和稳定的非线性或布尔交互。他们在布尔生物相互作用发现方面表现出了巨大的希望，这对推进功能基因组学和精准医学至关重要。然而，理论研究如何基于树的方法发现布尔特征的相互作用是缺失的。受许多生物过程中的阈值行为的启发，我们首先引入了一个不连续的非线性回归模型，称为“局部尖峰稀疏”（LSS）模型。具体而言，LSS模型假设回归函数是分段常数布尔交互项的线性组合。给定RF树集合，我们为一组有符号特征定义了一个称为“深度加权患病率”（DWP）的量。直观地说，DWP（）测量RF树集合中的特征一起出现的频率。我们证明，以高概率，DWP（）达到一个普遍的上限，不涉及任何模型系数，当且仅当对应于一个联盟的布尔相互作用下的LSS模型。因此，我们表明，一个理论上易于处理的版本的iRF程序，称为LSSFind，产生一致的相互作用发现下的LSS模型的样本量趋于无穷大。最后，仿真结果表明，LSSFind恢复LSS模型下的相互作用，即使在一些假设被违反。
Random Forests (RFs) are among the most successful machine-learning algorithms in terms of prediction accuracy. In many domain problems, however, the primary goal is not prediction, but to understand the data-generation process—in particular, finding important features and feature interactions. There exists strong empirical evidence that RF-based methods—in particular, iterative RF (iRF)—are very successful in terms of detecting feature interactions. In this work, we propose a biologically motivated, Boolean interaction model. Using this model, we complement the existing empirical evidence with theoretical evidence for the ability of iRF-type methods to select desirable interactions. Our theoretical analysis also yields deeper insights into the general interaction selection mechanism of decision-tree algorithms and the importance of feature subsampling. Random Forests (RFs) are at the cutting edge of supervised machine learning in terms of prediction performance, especially in genomics. Iterative RFs (iRFs) use a tree ensemble from iteratively modified RFs to obtain predictive and stable nonlinear or Boolean interactions of features. They have shown great promise for Boolean biological interaction discovery that is central to advancing functional genomics and precision medicine. However, theoretical studies into how tree-based methods discover Boolean feature interactions are missing. Inspired by the thresholding behavior in many biological processes, we first introduce a discontinuous nonlinear regression model, called the “Locally Spiky Sparse” (LSS) model. Specifically, the LSS model assumes that the regression function is a linear combination of piecewise constant Boolean interaction terms. Given an RF tree ensemble, we define a quantity called “Depth-Weighted Prevalence” (DWP) for a set of signed features . Intuitively speaking, DWP() measures how frequently features in appear together in an RF tree ensemble. We prove that, with high probability, DWP() attains a universal upper bound that does not involve any model coefficients, if and only if corresponds to a union of Boolean interactions under the LSS model. Consequentially, we show that a theoretically tractable version of the iRF procedure, called LSSFind, yields consistent interaction discovery under the LSS model as the sample size goes to infinity. Finally, simulation results show that LSSFind recovers the interactions under the LSS model, even when some assumptions are violated.
登录
查看更多内容
影响因子：
4.4
作者：
Chen, Xi;Ishwaran, Hemant
通讯作者：
Ishwaran, Hemant
影响因子：
5.4
作者：
Levine, Erel;Hwa, Terence
通讯作者：
Hwa, Terence
影响因子：
1.6
作者：
Janitza, Silke;Celik, Ender;Boulesteix, Anne-Laure
通讯作者：
Boulesteix, Anne-Laure
影响因子：
13.8
作者：
Ferrell, JE
通讯作者：
Ferrell, JE
DOI：
10.1073/pnas.0500670102
发表时间：
2005-03-22
影响因子：
11.1
作者：
Kobiler, O;Rokney, A;Oppenheim, AB
通讯作者：
Oppenheim, AB