Automatic structured variational inference
Automatic structured variational inference
复制标题
DOI：
--
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
L. Ambrogioni;M. Hinne;M. Gerven
中科院分区：
文献类型：
--
作者：
L. Ambrogioni;M. Hinne;M. Gerven
概率编程的目的是使任意概率模型(程序)中概率推理的各个方面自动化，以便用户可以将注意力集中在建模上，而不是处理特殊的推理方法。基于梯度的自动微分随机变分推理结合了高性能和高计算效率，为(可微)概率规划提供了一个有吸引力的默认方法。然而，任何(参数)变分方法的性能取决于适当的变分族的选择。在这里，我们介绍了一种受共轭模型闭合形式更新的启发而构造结构变分族的全自动方法。这些伪共轭族结合了输入概率程序的前向传递，可以捕获复杂的统计相关性。伪共轭族具有与输入概率程序相同的空间和时间复杂性，因此在非常大的一类模型中是容易处理的。我们在包括深度学习组件在内的一系列高维推理问题上验证了我们的自动变分方法。
The aim of probabilistic programming is to automatize every aspect of probabilistic inference in arbitrary probabilistic models (programs) so that the user can focus her attention on modeling, without dealing with ad-hoc inference methods. Gradient based automatic differentiation stochastic variational inference offers an attractive option as the default method for (differentiable) probabilistic programming as it combines high performance with high computational efficiency. However, the performance of any (parametric) variational approach depends on the choice of an appropriate variational family. Here, we introduced a fully automatic method for constructing structured variational families inspired to the closed-form update in conjugate models. These pseudo-conjugate families incorporate the forward pass of the input probabilistic program and can capture complex statistical dependencies. Pseudo-conjugate families have the same space and time complexity of the input probabilistic program and are therefore tractable in a very large class of models. We validate our automatic variational method on a wide range of high dimensional inference problems including deep learning components.