Collapsed Variational Inference for Sum-Product Networks
Collapsed Variational Inference for Sum-Product Networks
复制标题
和积网络的折叠变分推理
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Brandon Amos
中科院分区：
文献类型：
--
作者：
H. Zhao;T. Adel;Geoffrey J. Gordon;Brandon Amos
和积网络（spn）是一种概率推理机，它允许在线性时间内对网络的大小进行精确推理。现有的spn参数学习方法主要基于极大似然原理，与贝叶斯方法相比，存在过拟合的问题。spn的精确贝叶斯后验推理在计算上是难以处理的。即使是近似技术，如标准变分推理和spn的后验抽样，由于每个实例有大量的局部潜在变量，即使对于中等规模的网络，在计算上也是不可行的。在这项工作中，我们为spn提出了一种新的确定性崩溃变分推理算法，该算法计算效率高，易于实现，同时允许我们将先验信息纳入优化公式。大量的实验表明，与基于最大似然的方法相比，准确率有了显著提高。
Sum-Product Networks (SPNs) are probabilistic inference machines that admit exact inference in linear time in the size of the network. Existing parameter learning approaches for SPNs are largely based on the maximum likelihood principle and are subject to overfitting compared to more Bayesian approaches. Exact Bayesian posterior inference for SPNs is computationally intractable. Even approximation techniques such as standard variational inference and posterior sampling for SPNs are computationally infeasible even for networks of moderate size due to the large number of local latent variables per instance. In this work, we propose a novel deterministic collapsed variational inference algorithm for SPNs that is computationally efficient, easy to implement and at the same time allows us to incorporate prior information into the optimization formulation. Extensive experiments show a significant improvement in accuracy compared with a maximum likelihood based approach.