RI: Small: Frontiers in Monte Carlo and Variational Inference

概率推理允许人类从数据中获得洞察力并做出预测。企业、政府和科学界越来越需要使用数据来回答问题。通常，这些问题的最佳答案是将其表述为概率计算。随着数据集变得越来越大，越来越复杂，概率计算越来越困难，并且通常无法在合理的时间预算内精确执行。该项目将通过提供新的理论结果，算法和经验知识来促进科学和技术，这些理论结果，算法和经验知识是关于如何以一种在准确性和效率之间实现良好权衡的方式计算概率查询的近似答案。特别是，该项目将研究如何最好地结合联合收割机的优势，两种不同的战略计算概率。这项工作将提供新的技术，是实用的，有可调的精度，并扩展到非常大的数据sets.To满足这些目标，这个项目将结合联合收割机两种不同的方法来概率推理：变分推理（VI），和蒙特卡洛（MC）。MC算法是通用的，是渐近精确的，但可能无法在合理的时间或规模大的数据集给出很好的答案。相比之下，VI是一种通过将近似后验限制为易于处理的族来获得“相当好的答案，快速”的方法。这个项目将联合收割机以一种原则性的方式将这些结合起来，以获得通用的、实用的、具有可调精度的、可扩展到非常大的数据集的算法。新的算法预计将实现时间精度的权衡，占主导地位的蒙特卡罗方法的广泛的问题和时间预算。所提出的方法将1）通过基于Monte Carlo估计量设计近似族，将Monte Carlo方法的优势结合到变分推理中;以及2）通过使发散和近似族适应下游Monte Carlo估计量的需求，提高变分推理对下游任务的有用性。 该项目将产生一个全面的评估基准以及一套实用的技术，使该方法更加有效。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Probabilistic inference allows humans to gain insight and make predictions from data. There is an ever-growing need in business, government, and science to answer questions using data. Often, these questions are best answered by phrasing them as probability calculations. As data sets grow larger and more complex, probability calculations are increasingly difficult and often cannot be performed exactly within reasonable time budgets. This project will promote science and technology by providing new theoretical results, algorithms, and empirical knowledge about how to compute approximate answers to probabilistic queries in a way that achieves good tradeoffs between accuracy and efficiency. In particular, the project will study how to best combine the strengths of two different strategies for calculating probabilities. This work will provide new techniques that are practical, have tunable accuracy, and scale to very large data sets.To meet these goals, this project will combine two different approaches to probabilistic inference: variational inference (VI), and Monte Carlo (MC). MC algorithms are general-purpose and are asymptotically exact, but may fail to give good answers in reasonable time or scale large data sets. In contrast, VI is a way to get a "pretty good answer, quickly" by restricting the approximate posterior to tractable family. This project will combine these in a principled way to derive algorithms that are general-purpose, practical, have tunable accuracy, and scale to very large data sets. The new algorithms are expected to achieve time-accuracy tradeoffs that dominate Monte Carlo methods for a wide range of problems and time budgets. The proposed methods will 1) incorporate strengths of Monte Carlo methods into variational inference by designing approximating families based on Monte Carlo estimators; and 2) improve the usefulness of variational inference for downstream tasks by adapting divergences and approximating families to the needs of a downstream Monte Carlo estimator. The project will result in a comprehensive evaluation benchmark as well as a set of practical techniques to make the method more effective. A novel application in ecology will demonstrate the project's real-world potential.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.