Variational Training for Large-Scale Noisy-OR Bayesian Networks
Variational Training for Large-Scale Noisy-OR Bayesian Networks
复制标题
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Geng Ji;Dehua Cheng;Huazhong Ning;Changhe Yuan;Hanning Zhou;Liang Xiong;Erik B. Sudderth
中科院分区：
文献类型：
--
作者：
Geng Ji;Dehua Cheng;Huazhong Ning;Changhe Yuan;Hanning Zhou;Liang Xiong;Erik B. Sudderth
我们提出了一个随机变分推理算法训练大规模贝叶斯网络，噪声或条件分布被用来捕捉高阶关系。一个应用是学习文本数据的分层主题模型。虽然以前的工作主要集中在医疗诊断等应用中流行的双层网络上，但我们为深度网络开发了可扩展的算法，这些算法可以捕获多级层次的交互。我们的关键创新是一个家庭的约束变分界限，只显式优化后验概率的子图的主题最相关的稀疏观测在一个给定的文件。这些约束边界具有相当的精度，但大大降低了计算成本。使用基于变分边界的随机梯度更新，我们学习噪声或贝叶斯网络的数量级比以前的Monte Carlo学习算法更快，并为理解大规模二进制数据提供了一种新的工具。
We propose a stochastic variational inference algorithm for training large-scale Bayesian networks, where noisy-OR conditional distributions are used to capture higher-order relationships. One application is to the learning of hierarchical topic models for text data. While previous work has focused on two-layer networks popular in applications like medical diagnosis, we develop scalable algorithms for deep networks that capture a multi-level hierarchy of interactions. Our key innovation is a family of constrained variational bounds that only explicitly optimize posterior probabilities for the sub-graph of topics most related to the sparse observations in a given document. These constrained bounds have comparable accuracy but dramatically reduced computational cost. Using stochastic gradient updates based on our variational bounds, we learn noisy-OR Bayesian networks orders of magnitude faster than was possible with prior Monte Carlo learning algorithms, and provide a new tool for understanding large-scale binary data.