CAREER: Statistical Inference in High Dimensions using Variational Approximations

职业：使用变分近似进行高维统计推断

• 批准号: 2239234
• 负责人: Subhabrata Sen
• 金额: $ 43.41万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2028-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2239234&HistoricalAwards=false
• 关键词: CAREER; Statistical; Inference; Dimensions; using

Modern data applications routinely involve massive datasets comprising a multitude of observations and features. To facilitate statistical learning in real time, there is an urgent need for principled and computationally efficient statistical methodology. Variational Inference methods have recently emerged as a popular choice in this context. The term "Variational Inference" refers to a general out-of-the-box strategy to develop statistical algorithms for a wide class of problems. For example, these algorithms are used as a sub-routine in text mining, generation of hyper-realistic artificial text and images, machine translation, etc. This approach is extremely attractive due to the computational efficiency of the proposed methods, and their superior practical performance. Despite these advantages, rigorous guarantees for these variational methods are still in a nascent state. This project will develop statistical guarantees for the validity of this approach in diverse settings. Subsequently, these new insights will be exploited to develop novel statistical methodology for modern data applications. The outcome of the proposed research will allow practitioners to deploy Variational Inference methods with confidence. In addition, the outcomes will add a new set of principled, computationally efficient methods to the statistician's toolkit. The PI will interweave his research and teaching throughout the research period and beyond. In particular, the PI will develop new undergraduate/graduate courses focusing on Variational Inference and mentor students (particularly those from under-represented backgrounds) with the aim of introducing them to opportunities in statistics and data science. The proposed research and educational activities will broaden participation in STEM generally, and encourage careers in statistics and data science.This project will study statistical inference based on variational approximations focusing on three concrete thrusts: (i) Statistical inference based on the Naive Mean Field (NMF) approximation for regression models, (ii) NMF approximation beyond regression and (iii) Advanced Mean Field approximations. Under theme (i), the PI will develop empirical Bayes methodology for the high-dimensional linear model, and compare Bayesian variable selection algorithms using the NMF approximation. Theme (ii) will focus on the NMF approximation for Hidden Markov Random Fields and Bayesian Neural Networks. Finally, theme (iii) will focus on certain alternative mean-field approximations. Physicists conjecture that if the number of datapoints and features are both large and comparable, the NMF approximation is no longer accurate; instead, the Thouless-Anderson-Palmer (TAP) approximation, an advanced mean-field approximation, should facilitate Bayes optimal inference. The proposed research will establish this conjecture in the context of high-dimensional linear regression under a proportional asymptotic regime. The theoretical foundations of the proposed methodology will rest on disparate ideas originating in non-linear large deviations (studied in probability and combinatorics), spin glasses (studied in probability and statistical physics) and graphical models. In turn, these ideas will be combined with classical statistical ideas (e.g. nonparametric maximum likelihood) to develop computationally efficient methods for high-dimensional inference. This cross-pollination of ideas will generate independent follow up research directions in each domain.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代数据应用程序通常涉及大量数据集，其中包括许多观察结果和功能。为了实时促进统计学习，迫切需要有原则和计算有效的统计方法。在这种情况下，变异推理方法最近成为一种流行选择。 “变分推断”一词是指为各种问题开发统计算法的一般策略。例如，这些算法用作文本挖掘，超现实的人造文本和图像，机器翻译等的子例程。由于所提出的方法的计算效率以及其出色的实践性能，这种方法非常有吸引力。尽管有这些优势，但这些变异方法的严格保证仍处于新生状态。该项目将在各种环境中为这种方法的有效性增长统计保证。随后，这些新见解将被利用，以开发现代数据应用的新型统计方法。拟议的研究的结果将使从业者能够自信地部署变异推理方法。此外，结果将为统计学家的工具包添加一组新的原则性，计算有效方法。 PI将在整个研究期间及以后交织他的研究和教学。特别是，PI将开发新的本科/研究生课程，重点是变异推理和指导学生（尤其是来自代表性不足的背景的学生），目的是将其介绍给统计和数据科学领域的机会。拟议的研究和教育活动将普遍扩大参与STEM的参与，并鼓励在统计和数据科学方面的职业。本项目将基于重点介绍三个混凝土推力的变异近似值来研究统计推断：（i）基于天真的平均值（NMF）回归模型的统计推断，（ii）NMF近似近似近似近似和（IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII次）。在主题（i）下，PI将开发高维线性模型的经验贝叶斯方法，并使用NMF近似比较贝叶斯变量选择算法。主题（II）将重点介绍隐藏的马尔可夫随机字段和贝叶斯神经网络的NMF近似。最后，主题（III）将集中在某些替代均值场近似值上。物理学家猜想，如果数据点和特征的数量均大且可比性，则NMF近似不再准确。取而代之的是，thouless-anderson-palmer（TAP）近似，即高级平均近似值，应促进贝叶斯的最佳推理。拟议的研究将在高维线性回归的背景下建立这种猜想。所提出的方法的理论基础将基于源自非线性大偏差（以概率和组合学的研究），自旋玻璃（以概率和统计物理学的研究）和图形模型进行的不同思想。反过来，这些想法将与经典的统计思想（例如非参数最大可能性）相结合，以开发用于高维推断的计算有效方法。这种想法的这种交叉授粉将在每个领域中产生独立的后续研究指示。该奖项反映了NSF的法定任务，并且使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估来获得支持。