Collaborative Research: Fine-Grained Statistical Inference in High Dimension: Actionable Information, Bias Reduction, and Optimality

新兴的数据科学应用需要从庞大而混乱的数据集中有效提取可操作的见解。相关特征的数量通常会超过可用数据量，这使得统计推断任务和后续决策变得非常复杂。在现有的统计文献中，大多数理论旨在理解高维统计估计量的平均或全局行为。然而，在许多应用中，通常的情况是，目标不是探索参数估计器的全局行为，而是对其局部但重要的操作属性进行推理和推理。 该项目开发的技术和方法将进一步促进高维统计、调和分析、统计物理、优化、复杂分析和统计机器学习等广泛领域之间的相互作用。该项目为研究生提供研究培训机会。该项目追求细粒度的推理程序和理论，旨在扩大高维各种低复杂度模型的不确定性评估工具箱。该研究项目聚焦于一些典型问题，包括四个主要方向：（1）构建低秩矩阵估计中特征向量线性函数的最佳置信区间； (2) 为一般设计下的稀疏回归设计细粒度的假设检验程序； (3) 开发用于缺失数据的主成分分析的逐项推理方案； (4) 在最小特征间隙下进行可靠且自适应的统计特征分析。重点放在与模型无关且完全适应数据异方差性的算法上。解决这些问题需要开发新的统计理论，从而能够对未知参数下的大量局部属性进行可靠的推断。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Emerging data science applications require efficient extraction of actionable insights from large and messy datasets. The number of relevant features often overwhelms the volume of data that is available, which dramatically complicates the statistical inference tasks and subsequent decision making. In the existing statistical literature, most of theory aims at understanding the average or global behavior of a statistical estimator in high dimensions. In many applications, however, it is often the case that the goal is not to explore the global behavior of a parameter estimator, but rather to perform inference and reasoning on its local, yet important, operational properties. The techniques and methods developed in the project will further advance the interplay between a broad range of areas including high-dimensional statistics, harmonic analysis, statistical physics, optimization, complex analysis, and statistical machine learning. The project provides research training opportunities for graduate students.This project pursues fine-grained inferential procedures and theory, aimed at enlarging the uncertainty assessment toolbox for various low-complexity models in high dimensions. Focusing on a few stylized problems, this research program consists of four major thrusts: (1) construct optimal confidence intervals for linear functionals of eigenvectors in low-rank matrix estimation; (2) design fine-grained hypothesis testing procedures for sparse regression under general designs; (3) develop entry-wise inference schemes for principal component analysis with missing data; and (4) conduct reliable and adaptive statistical eigen-analysis under minimal eigen-gaps. Emphasis is placed on algorithms that are model-agnostic and fully adaptive to data heteroscedasticity. Addressing these issues calls for the development of new statistical theory that enables reliable inference for a broad class of local properties underlying the unknown parameters.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.