Complexity of High-Dimensional Statistical Models: An Information-Based Approach

高维统计模型的复杂性：基于信息的方法

• 批准号: 2015285
• 负责人: Ming Yuan
• 金额: $ 30万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2015285&HistoricalAwards=false
• 关键词: Complexity; Dimensional; Statistical; Models; Information

With big data come bigger goals – many modern statistical applications not only involve large datasets that may offer new insights but also require deciphering highly intricate relationships among a large number of variables, often characterized by very complex and high-dimensional functions. As more data are acquired, these functions inevitably become more complex. Oftentimes the most fundamental challenge for these applications is how to quantify the complexity of such tasks and learn these functions from data in an efficient way, both statistically and computationally. Despite impressive progress made in recent years, the current approach towards this goal is limited by the discrete nature of the classical notion of computational complexity and is not suitable for statistical problems that are continuous. This project aims to develop an information-based approach that better accounts for both statistical and computational efficiencies. This new framework of complexity is expected to offer insights into the potential trade-off between statistical and computational efficiencies and to reveal the role of experimental design in alleviating computational burden. The project provides training for graduate students through involvement in the research.Traditional nonparametric techniques based solely on smoothness are known to suffer from the so-called "curse of dimensionality." But in many scientific and engineering applications, the underlying high-dimensional object may have additional structures which, if appropriately accounted for, could help lift this barrier and allow for efficient methods to handle it. This project aims to develop a coherent framework to quantify the complexity of high dimensional models that appropriately accounts for both statistical accuracy and computational cost and helps better understand the role of these additional structures. The project will use this notion of complexity to examine several common yet notoriously difficult high-dimensional nonparametric regression problems: one based solely on smoothness, another based on smoothness and sparsity, and finally, one based on low-rank tensors. The exercise is designed to reveal interesting relationships between statistical and computational aspects of these problems and lead to the development of novel and optimal sampling and estimation strategies. The research will develop the new framework of complexity in more general statistical contexts as well and investigate its role in characterizing statistically and computationally optimal inference schemes. This will be achieved by developing new statistical methods and computational algorithms, theoretical study of their performance and fundamental limits, and the development of related mathematical tools and computational software.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

大数据带来了更大的目标——许多现代统计应用不仅涉及可能提供新见解的大型数据集，而且还需要破译大量变量之间高度复杂的关系，这些变量通常以非常复杂和高维的函数为特征。随着获取的数据越来越多，这些功能不可避免地变得越来越复杂。通常，这些应用程序最基本的挑战是如何量化这些任务的复杂性，并以有效的方式从数据中学习这些功能，包括统计和计算。尽管近年来取得了令人印象深刻的进展，但目前实现这一目标的方法受到经典计算复杂性概念的离散性的限制，并且不适合连续的统计问题。这个项目旨在发展一种基于信息的方法，更好地说明统计和计算效率。这一新的复杂性框架有望为统计和计算效率之间的潜在权衡提供见解，并揭示实验设计在减轻计算负担方面的作用。该项目通过参与研究为研究生提供培训。传统的基于平滑的非参数技术被称为“维数诅咒”。但在许多科学和工程应用中，潜在的高维物体可能有额外的结构，如果适当地解释，可以帮助消除这个障碍，并允许有效的方法来处理它。该项目旨在开发一个连贯的框架，以量化高维模型的复杂性，适当地考虑统计准确性和计算成本，并有助于更好地理解这些附加结构的作用。该项目将使用这个复杂性的概念来检查几个常见的但众所周知的困难的高维非参数回归问题：一个仅仅基于平滑，另一个基于平滑和稀疏，最后，一个基于低秩张量。该练习旨在揭示这些问题的统计和计算方面之间的有趣关系，并导致新的和最佳的采样和估计策略的发展。该研究将在更一般的统计背景下发展复杂性的新框架，并研究其在描述统计和计算最优推理方案中的作用。这将通过开发新的统计方法和计算算法，对其性能和基本限制的理论研究以及相关数学工具和计算软件的开发来实现。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Higher-Order Least Squares: Assessing Partial Goodness of Fit of Linear Causal Models

• DOI: 10.1080/01621459.2022.2157728
• 发表时间: 2021-09
• 期刊: Journal of the American Statistical Association
• 影响因子: 3.7
• 作者: C. Schultheiss;P. Bühlmann;Ming Yuan

On the Optimality of Kernel-Embedding Based Goodness-of-Fit Tests

• 发表时间: 2017-09
• 期刊: J. Mach. Learn. Res.
• 作者: K. Balasubramanian;Tong Li;M. Yuan

Comments on “Factor Models for High-Dimensional Tensor Time Series”

对“高维张量时间序列的因子模型”的评论

• DOI: 10.1080/01621459.2022.2028630
• 发表时间: 2022
• 期刊: Journal of the American Statistical Association
• 影响因子: 3.7
• 作者: Ouyang, Jialin;Yuan, Ming
• 通讯作者: Yuan, Ming

Effective Tensor Sketching via Sparsification

• DOI: 10.1109/tit.2021.3049174
• 发表时间: 2017-10
• 期刊: IEEE Transactions on Information Theory
• 影响因子: 2.5
• 作者: Dong Xia;M. Yuan

On Estimating Rank-One Spiked Tensors in the Presence of Heavy Tailed Errors

存在重尾误差时估计一阶尖峰张量

• DOI: 10.1109/tit.2022.3191883
• 发表时间: 2022
• 期刊: IEEE Transactions on Information Theory
• 影响因子: 2.5
• 作者: Auddy, Arnab;Yuan, Ming
• 通讯作者: Yuan, Ming