CAREER: Automatic Variational Inference

职业：自动变分推理

• 批准号: 2045900
• 负责人: Justin Domke
• 金额: $ 55.08万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2045900&HistoricalAwards=false
• 关键词: CAREER; Automatic; Variational; Inference

Hundreds of thousands of people across science, government, and business use automatic probabilistic inference tools. In these tools, people carefully state their assumptions, and then combine them with observed data. However, these automatic tools only work at a relatively modest scale, and ever-growing datasets require more powerful methods. Recent years have seen the development of a novel strategy for inference that has been able to address data orders of magnitude larger. However, great care and expertise is needed to wield these methods successfully, putting them out of reach for most potential users. This project seeks to promote the progress of science by making these large-scale techniques more automatic, putting them within reach of the vast majority of users not able to invest huge amounts of effort in manual algorithmic engineering.This project advances methodology for automatic and general-purpose variational inference, with the goal of answering two questions. The first question is when does variational inference work. This is paramount, since no method can succeed on all problems. We take three directions, namely new diagnostic error measures, improved scalability for diagnostics, and an empirical evaluation on a corpus of real non-expert models gathered from an integrated course. The second question is how to automate algorithmic design choices. Variational inference algorithms require many delicate design choices, currently made manually. The core idea to automate these decisions is to maintain statistics so the effect of any set of choices on optimization speed can be predicted. This project will contribute 1) a corpus and evaluation of automatic inference on non-expert models, 2) improved diagnostic performance measures, and 3) methods to automatically make variational inference choices, guided by convergence rates.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学界、政府和企业界的数十万人使用自动概率推理工具。在这些工具中，人们仔细陈述他们的假设，然后将它们与观察到的数据结合起来。然而，这些自动化工具只能在相对较小的规模下工作，而且不断增长的数据集需要更强大的方法。近年来，一种新的推理策略得到了发展，它能够处理更大数量级的数据。然而，成功使用这些方法需要极大的谨慎和专业知识，使大多数潜在用户无法使用它们。该项目旨在通过使这些大规模技术更加自动化来促进科学进步，使绝大多数用户能够接触到它们，而这些用户无法在手动算法工程中投入大量精力。该项目推进了自动和通用变分推理的方法学，目的是回答两个问题。第一个问题是，变分推理什么时候起作用。这是最重要的，因为没有一种方法可以在所有问题上都成功。我们采取了三个方向，即新的诊断误差度量，提高了诊断的可扩展性，以及对从综合课程收集的真实非专家模型语料库进行的实证评估。第二个问题是如何使算法设计选择自动化。变分推理算法需要许多精细的设计选择，目前是手动完成的。自动化这些决策的核心思想是维护统计数据，以便可以预测任何一组选择对优化速度的影响。该项目将有助于1)对非专家模型的自动推理的语料库和评估，2)改进的诊断性能测量，以及3)在收敛速度的指导下自动做出变分推理选择的方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。