CISE: RI: Small: Amortized Inference for Large-Scale Graphical Models

概率图形模型提供了一个数学框架来描述实体之间的空间和时间关系。从一些可观察到的测量中，可以推断出其他相关的隐藏因素。然而，当应用于大数据集时，标准推理方法非常耗时，需要大量迭代更新才能推断出每个变量。为了避免这些更新，分期推理使用神经网络来直接计算每个变量的近似解，这要快得多。摊销推理适用于更简单的模型，但不适用于捕获数据之间更多相关性的模型。该项目的目标是将摊销推广到数据之间具有依赖的本地连接性的模型。该项目的发现可能适用于许多有用的大规模应用，例如北美各地鸟类目击的空间模型和美国邻里、州和区域尺度上的阿片类药物过量的时空模型，以便为更有效的公共卫生干预提供信息。这项研究将进一步支持基于项目的大学级概率图形模型课程的开发，以及使该项目的新推理方法可供非专家使用的开源软件包。该项目的技术贡献将把对数据做出强烈条件独立性假设的模型的分期推理的效率带到更广泛的概率图形模型类别，以捕捉数据之间更现实的结构化依赖关系。在三种常见的推理算法--结构化变分推理(VI)、循环信念传播(LBP)和期望传播(EP)--中，将应用两个关键技术创新：(1)分解优化目标以允许逐个邻域进行可扩展处理；(2)识别可重复使用的邻域子结构，该子结构可以馈送到摊销神经网络以产生近似的局部后验分布。这些创新是具有挑战性的，因为大多数直接的尝试都会遇到难以处理的熵项(VI)，或者不能保持随机变量(LBP或EP)的局部边缘分布之间的一致性。这些挑战可以通过开发改进的界限和参数来加强一致性，导致摊销推理，其中参数的数量保持固定和负担得起，即使应用于包括数百万个变量的大型图形模型。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Probabilistic graphical models offer a mathematical framework to describe spatial and temporal relationships between entities. From some observable measurements, inferences can be made about other connected hidden factors. When applied to big datasets, however, standard inference methods are prohibitively time-consuming, requiring a large number of iterative updates to infer each variable. To avoid these updates, amortized inference uses a neural network to directly compute an approximate solution for every variable, which is much faster. Amortized inference works well for simpler models but not for models that capture more correlations between data. This project's goal is to generalize amortization to models that have dependent local connectivity between data. Findings from this project may be applicable to many useful large-scale applications, such as spatial modeling of bird sightings across North America and spatiotemporal modeling of opioid overdoses across neighborhood, state, and regional scales in the U.S. to inform more effective public health interventions. This research will further support the development of a project-based college-level course on probabilistic graphical models, as well as open-source software packages that make the project's new inference methods available to non-experts.The project's technical contribution will bring the efficiencies of amortized inference from models that make strong conditional independence assumptions about data to a wider class of probabilistic graphical models that capture more realistic structured dependencies between data. Across three common inference algorithms -- structured Variational Inference (VI), Loopy Belief Propagation (LBP), and Expectation Propagation (EP) -- two key technical innovations will be applied: (1) decomposition of the optimization objective to allow scalable neighborhood-by-neighborhood processing, and (2) identification of reusable neighborhood substructure that can be fed into an amortizing neural network to produce approximate local posterior distributions. These innovations are challenging because most straightforward attempts would encounter an intractable entropy term (VI) or not maintain consistency between local marginal distributions of random variables (LBP or EP). These challenges may be met by developing improved bounds and parameterizations that enforce consistency, leading to amortized inference in which the number of parameters remains fixed and affordable even when applied to large graphical models that include millions of variables.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.