CAREER: Super-Quantile Based Methods for Analyzing Large-Scale Heterogenous Data

数据驱动的决策已经在许多科学学科和生活的各个方面变得无处不在。由于社会越来越依赖统计建模来做决策，因此统计方法必须能够对异构数据进行建模，以便为不同的子群体做出不同的决策，从而产生更好的结果。这在大数据时代很有吸引力，因为数据丰富的设置允许统计模型经过训练，对异构数据更加灵活和健壮。该项目侧重于开发一类新的统计方法，用于基于超分位数（尾平均）对大规模异构数据进行建模，也称为预期不足或条件风险值。正在开发的方法将有可能回答以前使用不同领域（如气候科学、神经科学、金融和健康差异研究）的现有工具无法直接回答的问题。该项目将允许本科生和研究生研究数据异构建模的前沿方法。此外，还将开设一门关于数据异质性的研究生课程。研究者还将参与与密歇根大学教育推广中心合作的K-12教育推广。随着数据的丰富，对大规模异构数据建模用于决策的兴趣越来越大。分位数回归是为异构数据建模的一种代表性统计工具，但它的一个限制是，它关注于特定的分位数水平，对于回答涉及兴趣分布的下尾/上尾的汇总信息的科学问题可能不是最佳方法。研究者研究了一类新的基于超分位数的工具的发展，以帮助从业者回答这些问题。本项目有三个目标：(i)建立理论基础，开发可扩展的计算算法，用于拟合大数据、高维和异常值数据中的超分位数回归；（ii）在存在未测量混杂因素的情况下开发超分位数回归方法；（iii）开发一系列基于超分位数的方法来分析不同的数据类型。研究者将创建一个综合平台，包括软件和电子书教程，以鼓励使用基于超级分位数的方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data-driven decision-making has become ubiquitous across many scientific disciplines and in every aspect of life. As society relies more heavily on statistical modeling to make decisions, it becomes imperative for statistical methods to be able to model heterogenous data so that different decisions that will lead to better outcomes for different subgroups can be made. This is appealing in the big data era as data-rich settings allow statistical models to be trained to be more flexible and robust to heterogeneous data. This project focuses on developing a new class of statistical methods for modeling large-scale heterogenous data based on the super-quantile (tail average), also referred to as the expected shortfall or conditional value-at-risk. The methods under development will have the potential to answer questions that cannot be directly answered previously using existing tools in different fields such as climate science, neuroscience, finance, and health disparity research. The project will allow undergraduate and graduate students to work on cutting-edge methods for modeling data heterogeneity. In addition, a graduate-level course on data heterogeneity will be developed. The investigator will also engage in K-12 educational outreach in collaboration with the Center for Educational Outreach at the University of Michigan.With the abundance of data, there has been growing interest in modeling large-scale heterogeneous data for decision-making. Quantile regression is one representative statistical tool for modeling heterogeneous data, but one limitation is that it focuses on a specific quantile level and may not be the best approach for answering scientific questions that involve aggregate information of the lower/upper tail of the distribution of interest. The investigator studies the development of a new class of super-quantile-based tools to help practitioners answer such questions. There are three aims in this project: (i) establish theoretical foundations and develop scalable computational algorithms for fitting super-quantile regression in the big-data regime, high-dimensional regime, and data with outliers; (ii) develop super-quantile regression methods in the presence of unmeasured confounders; (iii) develop a series of super-quantile-based methods for analyzing different data types. The investigator will create a comprehensive platform, including software and e-book tutorials, to encourage using super-quantile-based methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.