Collaborative Research: Theoretical and Algorithmic Foundations of Variational Bayesian Inference

数据采集、处理和存储技术的惊人进步为现代统计学家提供了一个独特的机会来分析在许多科学调查以及社会和经济领域的研究中出现的具有前所未有丰富的大型和复杂的数据集。贝叶斯推理将先验知识和数据信息结合到后验分布中，为复杂的多水平数据集的概率建模和以有原则的方式执行相关的推理或预测任务提供了一种流行的范例。对于大多数实际问题，后验概率的计算需要数值近似；为此，基于抽样的方法，如马尔科夫链蒙特卡罗和确定性近似，都受到了广泛的关注。在基于最优化的确定性方法中，变分近似，通常也被称为变分推理，由于其对大数据集的可扩展性而非常受欢迎。通过这个项目，研究人员将探索常用变分程序的统计和算法特性，并在坚实的理论基础上开发新的方法和计算工具。这些结果旨在使从业者能够更好地理解变分推理可能成功的情况和存在潜在陷阱的情况。这项研究将通过在知名媒体发表文章和演讲的方式进行传播。此外，开发的方法的软件包将公开提供。研究人员致力于通过在各自的机构为学生提供建议并开发研究生和本科生主题课程来增强该提案的教学部分。受缓解贝叶斯计算中可伸缩性问题的日益增长的需求的推动，变分推理作为一种近似贝叶斯计算技术在过去20年中得到了极大的普及。尽管变分推理在大型复杂数据域中的经验证明是成功的，但对其统计性质的系统研究直到最近才开始。通过这个项目，研究人员将提出一些基础性问题，以解决在理解和解释变分近似在参数估计、统计推理和模型选择方面的巨大经验成功以及在新领域的应用方面的理论挑战。研究人员还将开发通用的充分条件来证明常用变分算法的收敛。理论发展将使用动力系统、功能优化和最优运输的工具，导致变分推理的统计和算法方面的统一处理。根据这一新理论，调查人员将对现有算法提出修改，使其具有更好的收敛行为。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Spectacular advances in data acquisition, processing and storage techniques offer modern-day statisticians a unique opportunity to analyze large and complex datasets of unprecedented richness which arise in many scientific investigations and in studies in the social and economic fields. Bayesian inference, which combines prior knowledge and data information into a posterior distribution, provides a popular paradigm for probabilistic modeling of complex multi-level datasets and for performing associated inferential or predictive tasks in a principled fashion. For most practical problems, computing the posterior probabilities require numerical approximations; to that end, sampling-based approaches such as Markov chain Monte Carlo and deterministic approximations have both received widespread attention. Among deterministic approaches based on optimization, variational approximations, also commonly referred to as variational inference, is highly popular due to its scalability to large datasets. Through this project, the investigators will explore statistical and algorithmic properties of popular variational procedures and develop new methodology and computational tools grounded on a strong theoretical foundation. The results are targeted to empower practitioners with a better understanding of situations where variational inference is likely to be successful and where potential pitfalls exist. The research will be disseminated through articles and talks at prominent outlets. Additionally, software packages for the methods developed will be made available publicly. The investigators are committed to enhancing the pedagogical component of the proposal through advising students and developing graduate and undergraduate topic courses at their respective institutions.Motivated by the increasing need to mitigate scalability issues in Bayesian computation, variational inference has tremendously grown in popularity over the last two decades as an approximate Bayesian computational technique. Despite the proven empirical successes of variational inference in large complex data domains, systematic investigations into its statistical properties have commenced only recently. Through this project, the investigators will pose a number of foundational questions to address theoretical challenges in understanding and explaining the great empirical success of variational approximations in parameter estimation, statistical inference, and model selection, coupled with applications in novel domains. The investigators will also develop general purpose sufficient conditions to certify convergence of popularly used variational algorithms. The theoretical development will employ tools from dynamical systems, functional optimization, and optimal transport, leading to a unified treatment of statistical and algorithmic aspects of variational inference. In light of this new theory, the investigators will propose modifications to existing algorithms with certifiably better convergence behaviors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.