CAREER: Super-Quantile Based Methods for Analyzing Large-Scale Heterogenous Data

数据驱动的决策已经在许多科学学科和生活的方方面面变得无处不在。随着社会越来越多地依赖统计建模来做出决策，统计方法必须能够对异质数据进行建模，以便能够做出不同的决策，从而为不同的分组带来更好的结果。这在大数据时代很有吸引力，因为数据丰富的设置允许对统计模型进行培训，使其对异类数据更加灵活和稳健。这个项目的重点是开发一类新的统计方法，用于基于超分位数(尾部平均值)对大规模异质数据进行建模，也被称为预期缺口或条件风险值。正在开发的方法将有可能回答以前无法使用气候科学、神经科学、金融和健康差距研究等不同领域的现有工具直接回答的问题。该项目将允许本科生和研究生研究建模数据异质性的尖端方法。此外，还将开发一门关于数据异质性的研究生课程。该调查员还将与密歇根大学教育外展中心合作进行K-12教育外展。随着数据的丰富，人们对为决策而建模的大规模异质数据越来越感兴趣。分位数回归是建模异质数据的一种代表性统计工具，但它的一个局限性是它侧重于特定的分位数水平，并且可能不是回答涉及兴趣分布的下尾/上尾的聚集信息的科学问题的最佳方法。研究人员研究了一类新的基于超分位数的工具的开发，以帮助从业者回答这些问题。这个项目有三个目标：(I)建立理论基础并开发可扩展的计算算法，用于在大数据、高维和有异常值的数据中拟合超分位数回归；(Ii)在存在不可测量混杂因素的情况下开发超分位数回归方法；(Iii)开发一系列基于超分位数的方法来分析不同的数据类型。研究人员将创建一个综合平台，包括软件和电子书教程，以鼓励使用基于超级分位数的方法。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Data-driven decision-making has become ubiquitous across many scientific disciplines and in every aspect of life. As society relies more heavily on statistical modeling to make decisions, it becomes imperative for statistical methods to be able to model heterogenous data so that different decisions that will lead to better outcomes for different subgroups can be made. This is appealing in the big data era as data-rich settings allow statistical models to be trained to be more flexible and robust to heterogeneous data. This project focuses on developing a new class of statistical methods for modeling large-scale heterogenous data based on the super-quantile (tail average), also referred to as the expected shortfall or conditional value-at-risk. The methods under development will have the potential to answer questions that cannot be directly answered previously using existing tools in different fields such as climate science, neuroscience, finance, and health disparity research. The project will allow undergraduate and graduate students to work on cutting-edge methods for modeling data heterogeneity. In addition, a graduate-level course on data heterogeneity will be developed. The investigator will also engage in K-12 educational outreach in collaboration with the Center for Educational Outreach at the University of Michigan.With the abundance of data, there has been growing interest in modeling large-scale heterogeneous data for decision-making. Quantile regression is one representative statistical tool for modeling heterogeneous data, but one limitation is that it focuses on a specific quantile level and may not be the best approach for answering scientific questions that involve aggregate information of the lower/upper tail of the distribution of interest. The investigator studies the development of a new class of super-quantile-based tools to help practitioners answer such questions. There are three aims in this project: (i) establish theoretical foundations and develop scalable computational algorithms for fitting super-quantile regression in the big-data regime, high-dimensional regime, and data with outliers; (ii) develop super-quantile regression methods in the presence of unmeasured confounders; (iii) develop a series of super-quantile-based methods for analyzing different data types. The investigator will create a comprehensive platform, including software and e-book tutorials, to encourage using super-quantile-based methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.