RI: Small: Frontiers in Monte Carlo and Variational Inference

概率推理使人类能够从数据中获得洞察力并做出预测。在商业、政府和科学领域，使用数据回答问题的需求日益增长。通常，这些问题最好的答案是将它们表述为概率计算。随着数据集变得越来越大和越来越复杂，概率计算变得越来越困难，而且往往无法在合理的时间预算内准确执行。该项目将通过提供新的理论结果、算法和经验知识来促进科学和技术，这些知识涉及如何以一种在准确性和效率之间实现良好折衷的方式计算概率查询的近似答案。特别是，该项目将研究如何最好地结合两种不同策略的优势来计算概率。这项工作将提供实用、具有可调精度和可扩展到非常大的数据集的新技术。为了实现这些目标，该项目将结合两种不同的概率推理方法：变分推理(VI)和蒙特卡罗(MC)。MC算法具有通用性和渐近精确度，但可能不能在合理的时间内给出较好的答案，也不能在大规模数据集上得到满意的结果。相比之下，VI是一种通过将近似后验限制为可处理的家庭来快速获得“相当好的答案”的方法。这个项目将以一种原则性的方式结合起来，得出通用的、实用的、具有可调精度的、可扩展到非常大的数据集的算法。新的算法有望实现时间和精度的折衷，这是蒙特卡罗方法在一系列问题和时间预算中的主导。所提出的方法将1)通过设计基于蒙特卡罗估计器的近似族，将蒙特卡罗方法的优点融入变分推理中；2)通过使发散度和近似族适应下游蒙特卡罗估计器的需要，提高变分推理对下游任务的有用性。该项目将产生一个全面的评价基准以及一套实用的技术，以使该方法更加有效。生态领域的新应用将展示该项目在现实世界中的潜力。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Probabilistic inference allows humans to gain insight and make predictions from data. There is an ever-growing need in business, government, and science to answer questions using data. Often, these questions are best answered by phrasing them as probability calculations. As data sets grow larger and more complex, probability calculations are increasingly difficult and often cannot be performed exactly within reasonable time budgets. This project will promote science and technology by providing new theoretical results, algorithms, and empirical knowledge about how to compute approximate answers to probabilistic queries in a way that achieves good tradeoffs between accuracy and efficiency. In particular, the project will study how to best combine the strengths of two different strategies for calculating probabilities. This work will provide new techniques that are practical, have tunable accuracy, and scale to very large data sets.To meet these goals, this project will combine two different approaches to probabilistic inference: variational inference (VI), and Monte Carlo (MC). MC algorithms are general-purpose and are asymptotically exact, but may fail to give good answers in reasonable time or scale large data sets. In contrast, VI is a way to get a "pretty good answer, quickly" by restricting the approximate posterior to tractable family. This project will combine these in a principled way to derive algorithms that are general-purpose, practical, have tunable accuracy, and scale to very large data sets. The new algorithms are expected to achieve time-accuracy tradeoffs that dominate Monte Carlo methods for a wide range of problems and time budgets. The proposed methods will 1) incorporate strengths of Monte Carlo methods into variational inference by designing approximating families based on Monte Carlo estimators; and 2) improve the usefulness of variational inference for downstream tasks by adapting divergences and approximating families to the needs of a downstream Monte Carlo estimator. The project will result in a comprehensive evaluation benchmark as well as a set of practical techniques to make the method more effective. A novel application in ecology will demonstrate the project's real-world potential.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.