An Instability in Variational Inference for Topic Models
An Instability in Variational Inference for Topic Models
复制标题
DOI：
--
复制
发表时间：
2018-02
期刊：
影响因子：
--
通讯作者：
B. Ghorbani;H. Javadi;A. Montanari
中科院分区：
文献类型：
--
作者：
B. Ghorbani;H. Javadi;A. Montanari
主题模型是贝叶斯模型，经常用于捕获某些文档或图像语料库的潜在结构。这样一个语料库中的每个数据元素（例如科学文章集合中的每个项目）被视为与“主题”或“组成部分”相对应的少量向量的凸组合。假设权重具有Dirichlet先验分布。接近后验的标准方法是使用变分推理算法，特别是平均场近似。我们表明，这种方法遭受的不稳定性，可以产生误导性的结论。也就是说，对于某些制度的模型参数，变分推理输出一个非平凡的分解成主题。然而，对于相同的参数值，数据不包含关于真实分解的实际信息，因此算法的输出与真实的主题分解不相关。在其他后果中，估计的后验均值是明显错误的，估计的贝叶斯可信区域没有达到名义覆盖。我们将讨论如何补救这种不稳定性更准确的平均场近似。
Topic models are Bayesian models that are frequently used to capture the latent structure of certain corpora of documents or images. Each data element in such a corpus (for instance each item in a collection of scientific articles) is regarded as a convex combination of a small number of vectors corresponding to `topics' or `components'. The weights are assumed to have a Dirichlet prior distribution. The standard approach towards approximating the posterior is to use variational inference algorithms, and in particular a mean field approximation. We show that this approach suffers from an instability that can produce misleading conclusions. Namely, for certain regimes of the model parameters, variational inference outputs a non-trivial decomposition into topics. However --for the same parameter values-- the data contain no actual information about the true decomposition, and hence the output of the algorithm is uncorrelated with the true topic decomposition. Among other consequences, the estimated posterior mean is significantly wrong, and estimated Bayesian credible regions do not achieve the nominal coverage. We discuss how this instability is remedied by more accurate mean field approximations.