Parallel Ensemble Learning and Feature Interaction Discovery: High Volume Dynamic Data

并行集成学习和特征交互发现：大量动态数据

• 批准号: 1953191
• 负责人: Bin Yu
• 金额: $ 45.2万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-15 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1953191&HistoricalAwards=false
• 关键词: Parallel; Ensemble; Learning; Feature; Interaction

In the digital age, the advancement of technology has enabled data collection at an unprecedented pace including the collection of a variety of dynamic data over time. Such dynamic data potentially holds the key to unlock many mysteries in science, such as how genes interact with each other in the developments of Drosophila and Human alike. However, dynamic data is notoriously challenging to analyze due to its changing nature as well as its massive data size. The PI plans to enhance the modeling toolbox for dynamic data by designing scalable parallel algorithms that aim at both high prediction accuracy and high interpretability through decision-tree based methods. Their applications range across many fields including computational biology and precision medicine. During the course of the proposed research, graduate students will receive training in domain-driven data science and open-source software development. Further dissemination of the proposed research will be through an upcoming book, undergraduate- and graduate-level courses, and presentations at workshops and conferences.The high-volume dynamic data poses challenges to the model training process because the underlying data distribution is varying with time. Algorithms or models have to adapt to the changing dynamic as well as their interpretations. Among statistics and machine learning methods, decision-tree based ensembles are especially favorable for dealing with a large volume of dynamic data because tree ensembles can capture flexible non-linear relationships in the data and are easily interpretable for people to extract useful narratives and information. PI’s prior work, such as iterative Random Forests (iRF) and signed iterative Random Forests (siRF), identifies stable and high-order biomolecule interactions that explain its high predictive accuracy but it only focuses on cross-sectional data at a fixed time point. The proposed research will build on the iRF and siRF algorithms to develop enhanced Random Forest and iRF algorithms for modeling high-volume and dynamic data with interpretable high-order feature interactions. The PI will 1) develop a communication-efficient parallel RF training algorithm (pRF) that can efficiently take advantage of a large number of machines. 2) propose a novel method that discovers feature interactions in the dynamic data with the presence of concept drift: dynamic iterative Random Forests (diRF). 3) carry out a theoretical analysis of pRF and diRF algorithm under time-varying change-detection models where local stationarity conditions are satisfied.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在数字时代，技术的进步使数据收集以前所未有的速度进行，包括随着时间的推移收集各种动态数据。这种动态数据潜在地掌握着解开科学中许多谜团的钥匙，例如在果蝇和人类的发育过程中，基因是如何相互作用的。然而，动态数据由于其不断变化的性质和海量数据大小而难以分析，这是出了名的。PI计划通过设计可扩展的并行算法来增强动态数据的建模工具箱，这些算法旨在通过基于决策树的方法实现高预测精度和高可解释性。它们的应用范围涉及许多领域，包括计算生物学和精确医学。在拟议的研究过程中，研究生将接受领域驱动的数据科学和开源软件开发方面的培训。将通过即将出版的一本书、本科生和研究生级别的课程以及在研讨会和会议上的演讲来进一步传播拟议的研究。大量的动态数据对模型培训过程构成了挑战，因为潜在的数据分布随时间而变化。算法或模型必须适应不断变化的动态及其解释。在统计学和机器学习方法中，基于决策树的集成方法特别适合处理大量的动态数据，因为树集成可以捕获数据中灵活的非线性关系，并且易于解释，便于人们提取有用的叙述和信息。PI以前的工作，如迭代随机森林(IRF)和符号迭代随机森林(SiRF)，确定了稳定的和高阶生物分子相互作用，解释了其高预测精度，但它只关注固定时间点的横截面数据。建议的研究将建立在IRF和SiRF算法的基础上，以开发增强型随机森林和IRF算法，用于建模具有可解释的高阶特征交互的大容量和动态数据。PI将1)开发一种通信高效的并行RF训练算法(PRF)，该算法可以有效地利用大量机器。2)提出了一种在存在概念漂移的动态数据中发现特征交互的新方法：动态迭代随机森林(DIRF)。3)在满足局部平稳性条件的时变变化检测模型下，对PRF和DRF算法进行了理论分析。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Stable Discovery of Interpretable Subgroups via Calibration in Causal Studies

• DOI: 10.1111/insr.12427
• 发表时间: 2020-12-22
• 期刊: INTERNATIONAL STATISTICAL REVIEW
• 影响因子: 2
• 作者: Dwivedi, Raaz;Tan, Yan Shuo;Yu, Bin
• 通讯作者: Yu, Bin