CISE: RI: Small: Amortized Inference for Large-Scale Graphical Models

概率图模型提供了一个数学框架来描述实体之间的空间和时间关系。从一些可观察的测量结果中，可以推断出其他相关的隐藏因素。然而，当应用于大数据集时，标准的推理方法非常耗时，需要大量的迭代更新来推断每个变量。为了避免这些更新，摊销推理使用神经网络直接计算每个变量的近似解，这要快得多。分期推理适用于更简单的模型，但不适用于捕捉数据之间更多相关性的模型。这个项目的目标是将摊销推广到数据之间具有依赖本地连接的模型。该项目的研究结果可能适用于许多有用的大规模应用，例如北美鸟类目击的空间建模以及美国社区，州和区域范围内阿片类药物过量的时空建模，以提供更有效的公共卫生干预措施。这项研究将进一步支持基于项目的概率图形模型大学课程的开发，以及开源软件包，使该项目的新推理方法可用于非该项目的技术贡献将带来摊销推理的效率，从对数据做出强条件独立假设的模型到更广泛的概率图形模型，数据之间的真实结构化依赖关系。在三种常见的推理算法中-结构化变分推理（VI），循环信念传播（LBP）和期望传播（EP）-将应用两个关键的技术创新：（1）分解优化目标以允许可缩放的逐邻域处理，以及（2）识别可重用的邻域子结构，该邻域子结构可以被馈送到摊销神经网络以产生近似的局部后验分布。这些创新是具有挑战性的，因为大多数直接的尝试会遇到一个棘手的熵项（VI）或不保持随机变量（LBP或EP）的局部边缘分布之间的一致性。这些挑战可以通过开发改进的边界和参数化来实现一致性，从而实现摊销推理，即使应用于包含数百万个变量的大型图形模型，参数的数量也保持固定且负担得起。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Probabilistic graphical models offer a mathematical framework to describe spatial and temporal relationships between entities. From some observable measurements, inferences can be made about other connected hidden factors. When applied to big datasets, however, standard inference methods are prohibitively time-consuming, requiring a large number of iterative updates to infer each variable. To avoid these updates, amortized inference uses a neural network to directly compute an approximate solution for every variable, which is much faster. Amortized inference works well for simpler models but not for models that capture more correlations between data. This project's goal is to generalize amortization to models that have dependent local connectivity between data. Findings from this project may be applicable to many useful large-scale applications, such as spatial modeling of bird sightings across North America and spatiotemporal modeling of opioid overdoses across neighborhood, state, and regional scales in the U.S. to inform more effective public health interventions. This research will further support the development of a project-based college-level course on probabilistic graphical models, as well as open-source software packages that make the project's new inference methods available to non-experts.The project's technical contribution will bring the efficiencies of amortized inference from models that make strong conditional independence assumptions about data to a wider class of probabilistic graphical models that capture more realistic structured dependencies between data. Across three common inference algorithms -- structured Variational Inference (VI), Loopy Belief Propagation (LBP), and Expectation Propagation (EP) -- two key technical innovations will be applied: (1) decomposition of the optimization objective to allow scalable neighborhood-by-neighborhood processing, and (2) identification of reusable neighborhood substructure that can be fed into an amortizing neural network to produce approximate local posterior distributions. These innovations are challenging because most straightforward attempts would encounter an intractable entropy term (VI) or not maintain consistency between local marginal distributions of random variables (LBP or EP). These challenges may be met by developing improved bounds and parameterizations that enforce consistency, leading to amortized inference in which the number of parameters remains fixed and affordable even when applied to large graphical models that include millions of variables.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.