FRG: Collaborative Research: Quantile-Based Modeling for Large-Scale Heterogeneous Data

技术的快速发展导致了科学、经济、工程、医疗等许多学科的大规模异质数据的巨大增长。例如，在现代健康信息系统中，电子健康记录例行公事地收集来自不同疾病类别的不同人群的许多患者的大量信息。这些数据提供了独特的机会来了解不同亚群之间的特征和结果之间的关联。现有的办法没有完全解决计算和统计方面的巨大挑战。为了挖掘信息丰富的数据的真正潜力，该项目将开发一种新的计算和统计范式，并为分析大规模不同种类的数据奠定坚实的理论基础。此外，该项目还将为研究生提供研究培训机会。该项目将建立一个统一的、基于分位数模型的框架，其首要目标是在分析不同种类的数据时实现有效性和可靠性，特别是在潜在解释变量的数量和样本量都很大的情况下。具体目标是：(1)开发基于重采样的大规模异质数据推理；(2)开发贝叶斯算法以及可扩展和可解释的结构感知方法，以更好地进行推理；(3)开发分位数最优决策规则估计和多协变量推理；(4)开发新的截尾条件下大规模分位数回归的估计和推理程序。该项目将解决数据大小和维度的可扩展性、异质性和结构的探索、健壮性需求以及利用不完整观测的能力方面的一些关键障碍。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The rapid development of technology has led to the tremendous growth of large-scale heterogeneous data in science, economics, engineering, healthcare, and many other disciplines. For example, in a modern health information system, electronic health records routinely collect a large amount of information on many patients from heterogeneous populations across different disease categories. Such data provide unique opportunities to understand the association between features and outcomes across different subpopulations. Existing approaches have not fully addressed the formidable computational and statistical challenges. To tap into the true potential of information-rich data, this project will develop a new computational and statistical paradigm and solid theoretical foundation for analyzing large-scale heterogeneous data. In addition the project will also provide research training opportunities for graduate students. The project will build a unified, quantile-modeling based framework with an overarching goal of achieving effectiveness and reliability in analyzing heterogeneous data, especially when both the number of potential explanatory variables and the sample size are large. The specific goals are (1) to develop resampling-based inference for large-scale heterogeneous data; (2) to develop Bayesian algorithms and scalable and interpretable structure-aware approach for better inference; (3) to develop quantile-optimal decision rule estimation and inference with many covariates; (4) to develop novel estimation and inference procedure for large-scale quantile regression under censoring. The project will address some of the key barriers in scalability to data size and dimensionality, exploration of heterogeneity and structures, need for robustness, and the ability to make use of incomplete observations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.