III: Small: Novel Representations for Inference in Graphical Models

概率推理是人类从数据中获得洞察力和做出预测的重要工具。分析人员对观察到的数据假设一个概率模型，然后进行推理来拟合参数，评估模型的有效性，并做出预测。随着几乎所有科学和社会领域的数据规模和复杂性的增长，丰富的概率模型与快速准确的推理算法相结合，对于数据分析过程至关重要。概率图模型通过利用图结构来设计丰富的模型，并支持用于推理的快速消息传递算法，从而支持这些目标。然而，精确的消息传递仅适用于有限的模型类。这个项目的目标是通过开发概率分布和消息的新表示来显著扩展消息传递算法应用的模型类别。该研究有望显著改善实践中遇到的各种模型的可用推断技术，特别是在时间序列建模和种群生态学领域。该项目网站将用于传播研究原型。这项工作提出了两种新的表示图形模型中的因素。第一个提出的表示是基于具有计数变量的模型的概率生成函数。消息传递目前并不适用于这些模型，因为变量具有无限的支持，并且不能使用标准表示在有限的空间和时间内操纵因子。然而，在许多情况下，概率生成函数可以精确而紧凑地表示可数无限因子。所提出的工作的目标是开发具有表达性的生成函数类，这些生成函数类可以适应算法操作，并且可以表示图形模型中常见的因素，然后开发精确和近似的算法来直接在生成函数空间中执行消息传递。第二种建议的表示是基于分段指数函数，它提供了连续密度的近似表示。分段指数函数是紧凑的，计算上易于处理，并提供一个可调的近似水平。它们适用于具有连续但非高斯变量的模型。所提出的工作的目标是开发构建精确的分段指数近似的技术，并在消息传递算法中传播它们。这项工作开发的算法将被测试和应用于改进现有模型中的推理，并为种群生态学模型提供新的功能。

Probabilistic inference is an important tool that allows humans to gain insight and make predictions from data. An analyst posits a probabilistic model for observed data, and then conducts inference to fit parameters, assess model validity, and make predictions. With data growing in size and complexity across nearly all domains of science and society, rich probabilistic models paired with fast and accurate inference algorithms are essential to the data analytic process. Probabilistic graphical models support these goals by leveraging graph structure to design rich models and enable fast message-passing algorithms for inference. However, exact message passing applies only to limited model classes. The goal of this project is to significantly expand the class of models to which message-passing algorithms apply by developing novel representations of probability distributions and messages. The research is expected to significantly improve the available inference techniques for a wide range of models that are encountered in practice, especially in the fields of time-series modeling and population ecology. The project website will be used to disseminate research prototypes. This work proposes two novel representations for factors in graphical models. The first proposed representation is based on probability generating functions for models with count variables. Message passing does not currently apply to these models because the variables have countably infinite support and factors cannot be manipulated in finite space and time using standard representations. However, in many cases, probability generating functions can represent countably infinite factors exactly and compactly. The goal of the proposed work is develop expressive classes of generating functions that are amenable to algorithmic manipulation and can represent commonly appearing factors in graphical models, and then to develop exact and approximate algorithms to perform message passing directly in the space of generating functions. The second proposed representation is based on piecewise-exponential functions, which provide approximate representations of continuous densities. Piecewise-exponential functions are compact, computationally tractable, and provide a tunable level of approximation. They apply to models with continuous but non-Gaussian variables. The goal of the proposed work is to develop techniques to construct accurate piecewise exponential approximations and propagate them within message-passing algorithms. The algorithms developed by this work will be tested and applied to improve inference in existing models and provide new capabilities for models in population ecology.