CAREER: Automatic Variational Inference

科学界、政府和商业界的数十万人使用自动概率推理工具。在这些工具中，人们仔细地陈述他们的假设，然后将它们与观察到的数据联合收割机结合起来。然而，这些自动工具只能在相对较小的规模上工作，并且不断增长的数据集需要更强大的方法。近年来已经看到了一种新的推理策略的发展，该策略已经能够处理更大数量级的数据。然而，要成功地运用这些方法，需要非常谨慎和专业知识，使大多数潜在用户无法接触到这些方法。该项目旨在通过使这些大规模技术更加自动化，使绝大多数无法投入大量精力进行人工算法工程的用户能够获得这些技术，从而促进科学的进步。该项目推进了自动和通用变分推理的方法，目标是回答两个问题。第一个问题是变分推理何时起作用。这是至关重要的，因为没有一种方法可以在所有问题上都取得成功。我们采取三个方向，即新的诊断错误的措施，提高诊断的可扩展性，并从综合课程收集的真实的非专家模型的语料库的实证评估。第二个问题是如何自动化算法设计选择。变分推理算法需要许多微妙的设计选择，目前手动。自动化这些决策的核心思想是维护统计数据，以便可以预测任何一组选择对优化速度的影响。该项目将贡献1）语料库和非专家模型的自动推理评估，2）改进的诊断性能措施，和3）方法，以自动进行变分推理选择，收敛率为指导。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Hundreds of thousands of people across science, government, and business use automatic probabilistic inference tools. In these tools, people carefully state their assumptions, and then combine them with observed data. However, these automatic tools only work at a relatively modest scale, and ever-growing datasets require more powerful methods. Recent years have seen the development of a novel strategy for inference that has been able to address data orders of magnitude larger. However, great care and expertise is needed to wield these methods successfully, putting them out of reach for most potential users. This project seeks to promote the progress of science by making these large-scale techniques more automatic, putting them within reach of the vast majority of users not able to invest huge amounts of effort in manual algorithmic engineering.This project advances methodology for automatic and general-purpose variational inference, with the goal of answering two questions. The first question is when does variational inference work. This is paramount, since no method can succeed on all problems. We take three directions, namely new diagnostic error measures, improved scalability for diagnostics, and an empirical evaluation on a corpus of real non-expert models gathered from an integrated course. The second question is how to automate algorithmic design choices. Variational inference algorithms require many delicate design choices, currently made manually. The core idea to automate these decisions is to maintain statistics so the effect of any set of choices on optimization speed can be predicted. This project will contribute 1) a corpus and evaluation of automatic inference on non-expert models, 2) improved diagnostic performance measures, and 3) methods to automatically make variational inference choices, guided by convergence rates.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.