III: Small: Novel Representations for Inference in Graphical Models

概率推理是人类从数据中获得洞察力并做出预测的重要工具。分析师为观测数据建立概率模型，然后进行推理以拟合参数、评估模型有效性并做出预测。随着科学和社会几乎所有领域的数据在规模和复杂性上的增长，丰富的概率模型和快速准确的推理算法对于数据分析过程至关重要。概率图形模型通过利用图结构来设计丰富的模型并启用用于推理的快速消息传递算法来支持这些目标。但是，准确的消息传递仅适用于有限的模型类。这个项目的目标是通过开发概率分布和消息的新表示来显著扩展消息传递算法应用的模型类别。这项研究有望显著改进在实践中遇到的各种模型的现有推理技术，特别是在时间序列建模和种群生态学领域。该项目网站将用于传播研究原型。本文提出了图形模型中因子的两种新的表示方法。第一种表示是基于带计数变量的模型的概率母函数。消息传递目前不适用于这些模型，因为变量具有可计数的无限支持，而因子不能在有限的空间和时间内使用标准表示法进行操作。然而，在许多情况下，概率母函数可以精确而紧凑地表示可计数的无限因素。所提出的工作的目标是开发易于算法操作并且能够表示图形模型中常见出现的因素的生成函数的表达类，然后开发精确和近似的算法来直接在生成函数空间中执行消息传递。第二种表示是基于分段指数函数，它提供了连续密度的近似表示。分段指数函数是紧凑的，在计算上容易处理，并且提供了可调的逼近水平。它们适用于具有连续但非高斯变量的模型。这项工作的目标是开发技术来构造准确的分段指数近似，并在消息传递算法中传播它们。这项工作开发的算法将被测试和应用，以改进现有模型中的推理，并为种群生态学中的模型提供新的能力。

Probabilistic inference is an important tool that allows humans to gain insight and make predictions from data. An analyst posits a probabilistic model for observed data, and then conducts inference to fit parameters, assess model validity, and make predictions. With data growing in size and complexity across nearly all domains of science and society, rich probabilistic models paired with fast and accurate inference algorithms are essential to the data analytic process. Probabilistic graphical models support these goals by leveraging graph structure to design rich models and enable fast message-passing algorithms for inference. However, exact message passing applies only to limited model classes. The goal of this project is to significantly expand the class of models to which message-passing algorithms apply by developing novel representations of probability distributions and messages. The research is expected to significantly improve the available inference techniques for a wide range of models that are encountered in practice, especially in the fields of time-series modeling and population ecology. The project website will be used to disseminate research prototypes. This work proposes two novel representations for factors in graphical models. The first proposed representation is based on probability generating functions for models with count variables. Message passing does not currently apply to these models because the variables have countably infinite support and factors cannot be manipulated in finite space and time using standard representations. However, in many cases, probability generating functions can represent countably infinite factors exactly and compactly. The goal of the proposed work is develop expressive classes of generating functions that are amenable to algorithmic manipulation and can represent commonly appearing factors in graphical models, and then to develop exact and approximate algorithms to perform message passing directly in the space of generating functions. The second proposed representation is based on piecewise-exponential functions, which provide approximate representations of continuous densities. Piecewise-exponential functions are compact, computationally tractable, and provide a tunable level of approximation. They apply to models with continuous but non-Gaussian variables. The goal of the proposed work is to develop techniques to construct accurate piecewise exponential approximations and propagate them within message-passing algorithms. The algorithms developed by this work will be tested and applied to improve inference in existing models and provide new capabilities for models in population ecology.