Graphical Models, Exponential Families, and Variational Inference
Graphical Models, Exponential Families, and Variational Inference
复制标题
DOI：
10.1561/2200000001
复制
发表时间：
2008-01-01
影响因子：
32.8
通讯作者：
Jordan, Michael I.
中科院分区：
文献类型：
--
作者：
Wainwright, Martin J.;Jordan, Michael I.
概率图模型的形式主义为捕获随机变量之间的复杂依赖关系和构建大规模多元统计模型提供了统一的框架。图模型已经成为许多统计、计算和数学领域的研究热点，包括生物信息学、通信理论、统计物理、组合优化、信号和图像处理、信息检索和统计机器学习。在特定情况下出现的许多问题，包括计算边际和概率分布模式的关键问题，最好在一般情况下进行研究。工作与指数族表示，并利用共轭的对偶累积函数和指数族的熵，我们开发一般变分表示的问题计算似然，边际概率和最可能的配置。我们描述了各种各样的算法-其中包括总和产品，集群变分方法，期望传播，平均场方法，最大产品和线性规划松弛，以及圆锥规划松弛-都可以理解这些变分表示的精确或近似形式。变分方法提供了一个互补的替代马尔可夫链蒙特卡罗作为一个一般来源的近似方法在大规模的统计模型的推断。
The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances including the key problems of computing marginals and modes of probability distributions - are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, we develop general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. We describe how a wide variety of algorithms - among them sum-product, cluster variational methods, expectation-propagation, mean field methods, max-product and linear programming relaxation, as well as conic programming relaxations - can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.