FRG: Collaborative Research: Quantile-Based Modeling for Large-Scale Heterogeneous Data

技术的快速发展导致了科学、经济、工程、医疗保健和许多其他学科中大规模异构数据的巨大增长。例如，在现代健康信息系统中，电子健康记录常规地收集关于来自不同疾病类别的异质群体的许多患者的大量信息。这些数据提供了独特的机会，以了解不同亚群之间的特征和结果之间的关联。现有的方法还没有完全解决可怕的计算和统计的挑战。为了挖掘信息丰富的数据的真正潜力，该项目将开发一种新的计算和统计范式，并为分析大规模异构数据奠定坚实的理论基础。此外，该项目还将为研究生提供研究培训机会。该项目将建立一个统一的，基于分位数建模的框架，其总体目标是在分析异构数据时实现有效性和可靠性，特别是当潜在的解释变量和样本量都很大时。具体目标是：（1）为大规模异质数据开发基于重采样的推理;（2）开发贝叶斯算法和可扩展和可解释的结构感知方法，以实现更好的推理;（3）开发具有许多协变量的分位数最优决策规则估计和推理;（4）开发新的估计和推理过程，用于大规模删失下的分位数回归。该项目将解决数据大小和维度的可扩展性、异质性和结构的探索、鲁棒性需求以及利用不完整观测结果的能力方面的一些关键障碍。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

The rapid development of technology has led to the tremendous growth of large-scale heterogeneous data in science, economics, engineering, healthcare, and many other disciplines. For example, in a modern health information system, electronic health records routinely collect a large amount of information on many patients from heterogeneous populations across different disease categories. Such data provide unique opportunities to understand the association between features and outcomes across different subpopulations. Existing approaches have not fully addressed the formidable computational and statistical challenges. To tap into the true potential of information-rich data, this project will develop a new computational and statistical paradigm and solid theoretical foundation for analyzing large-scale heterogeneous data. In addition the project will also provide research training opportunities for graduate students. The project will build a unified, quantile-modeling based framework with an overarching goal of achieving effectiveness and reliability in analyzing heterogeneous data, especially when both the number of potential explanatory variables and the sample size are large. The specific goals are (1) to develop resampling-based inference for large-scale heterogeneous data; (2) to develop Bayesian algorithms and scalable and interpretable structure-aware approach for better inference; (3) to develop quantile-optimal decision rule estimation and inference with many covariates; (4) to develop novel estimation and inference procedure for large-scale quantile regression under censoring. The project will address some of the key barriers in scalability to data size and dimensionality, exploration of heterogeneity and structures, need for robustness, and the ability to make use of incomplete observations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.