CISE: RI: Small: Amortized Inference for Large-Scale Graphical Models

CISE：RI：小型：大规模图形模型的摊销推理

• 批准号: 1908617
• 负责人: Liping Liu
• 金额: $ 39.99万
• 依托单位: Tufts University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1908617&HistoricalAwards=false
• 关键词: CISE; RI; Small; Amortized; Inference

Probabilistic graphical models offer a mathematical framework to describe spatial and temporal relationships between entities. From some observable measurements, inferences can be made about other connected hidden factors. When applied to big datasets, however, standard inference methods are prohibitively time-consuming, requiring a large number of iterative updates to infer each variable. To avoid these updates, amortized inference uses a neural network to directly compute an approximate solution for every variable, which is much faster. Amortized inference works well for simpler models but not for models that capture more correlations between data. This project's goal is to generalize amortization to models that have dependent local connectivity between data. Findings from this project may be applicable to many useful large-scale applications, such as spatial modeling of bird sightings across North America and spatiotemporal modeling of opioid overdoses across neighborhood, state, and regional scales in the U.S. to inform more effective public health interventions. This research will further support the development of a project-based college-level course on probabilistic graphical models, as well as open-source software packages that make the project's new inference methods available to non-experts.The project's technical contribution will bring the efficiencies of amortized inference from models that make strong conditional independence assumptions about data to a wider class of probabilistic graphical models that capture more realistic structured dependencies between data. Across three common inference algorithms -- structured Variational Inference (VI), Loopy Belief Propagation (LBP), and Expectation Propagation (EP) -- two key technical innovations will be applied: (1) decomposition of the optimization objective to allow scalable neighborhood-by-neighborhood processing, and (2) identification of reusable neighborhood substructure that can be fed into an amortizing neural network to produce approximate local posterior distributions. These innovations are challenging because most straightforward attempts would encounter an intractable entropy term (VI) or not maintain consistency between local marginal distributions of random variables (LBP or EP). These challenges may be met by developing improved bounds and parameterizations that enforce consistency, leading to amortized inference in which the number of parameters remains fixed and affordable even when applied to large graphical models that include millions of variables.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

概率图形模型提供了一个数学框架来描述实体之间的空间和时间关系。从一些可观察到的测量，可以推断出其他相关的隐藏因素。然而，当应用于大数据集时，标准推理方法非常耗时，需要大量的迭代更新来推断每个变量。为了避免这些更新，平摊推理使用神经网络直接计算每个变量的近似解，这要快得多。平摊推理适用于更简单的模型，但不适用于捕获数据之间更多相关性的模型。这个项目的目标是将摊销推广到数据之间具有依赖的本地连接的模型。该项目的研究结果可能适用于许多有用的大规模应用，例如北美鸟类目击的空间建模以及美国社区，州和区域尺度上阿片类药物过量的时空建模，以提供更有效的公共卫生干预措施。这项研究将进一步支持基于项目的概率图形模型大学水平课程的开发，以及开源软件包，使项目的新推理方法可供非专家使用。该项目的技术贡献将把对数据进行强条件独立假设的模型的平摊推理的效率提高到更广泛的概率图形模型，这些模型可以捕获数据之间更现实的结构化依赖关系。在三种常见的推理算法中——结构化变分推理（VI）、环形信念传播（LBP）和期望传播（EP）——将应用两个关键的技术创新：(1)分解优化目标，以允许可扩展的邻域处理；(2)识别可重复使用的邻域子结构，这些子结构可以输入到平摊神经网络中，以产生近似的局部后验分布。这些创新具有挑战性，因为最直接的尝试会遇到棘手的熵项（VI），或者无法保持随机变量的局部边际分布（LBP或EP）之间的一致性。这些挑战可以通过开发改进的边界和参数化来解决，这些边界和参数化可以增强一致性，从而导致平摊推理，在平摊推理中，即使应用于包含数百万变量的大型图形模型，参数的数量也可以保持固定和负担得起。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Easy Variational Inference for Categorical Models via an Independent Binary Approximation

通过独立二元近似对分类模型进行简单的变分推理

• 发表时间: 2022
• 期刊: International Conference on Machine Learning
• 作者: Michael T. Wojnowicz;Shuchin Aeron;Eric L. Miller;Michael C. Hughes
• 通讯作者: Michael C. Hughes

Interpretable Node Representation with Attribute Decoding

• DOI: 10.48550/arxiv.2212.01682
• 发表时间: 2022-12
• 期刊: Trans. Mach. Learn. Res.
• 作者: Xiaohui Chen;Xi Chen;Liping Liu

PatchGT: Transformer over Non-trainable Clusters for Learning Graph Representations

• DOI: 10.48550/arxiv.2211.14425
• 发表时间: 2022-11
• 作者: Han Gao;Xuhong Han;Jiaoyang Huang;Jian-Xun Wang;Liping Liu

Stochastic Iterative Graph Matching

• 发表时间: 2021-06
• 期刊: ArXiv
• 作者: Linfeng Liu;M. Hughes;S. Hassoun;Liping Liu

Predicting Spatiotemporal Counts of Opioid-related Fatal Overdoses via Zero-Inflated Gaussian Processes

通过零膨胀高斯过程预测阿片类药物相关致命过量的时空计数

• 发表时间: 2022
• 期刊: and Decision-making Systems.
• 作者: Kyle Heuton;Shikhar Shrestha;Thomas J. Stopka;Jennifer Pustz;Li-Ping Liu;Michael C. Hughes
• 通讯作者: Michael C. Hughes