Mean Field for the Stochastic Blockmodel: Optimization Landscape and Convergence Issues
Mean Field for the Stochastic Blockmodel: Optimization Landscape and Convergence Issues
复制标题
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Soumendu Sundar Mukherjee;Purnamrita Sarkar;Y. X. R. Wang;Bowei Yan
中科院分区：
文献类型：
--
作者：
Soumendu Sundar Mukherjee;Purnamrita Sarkar;Y. X. R. Wang;Bowei Yan
变分近似近年来在大规模贝叶斯推理中得到了广泛的应用，其中最简单的一种是施加平均场假设来近似复杂的潜在结构。尽管平均场具有计算可扩展性，但对其损失函数曲面和优化损失的迭代更新收敛行为的理论研究还远远不够。本文主要研究一类简单两类随机块模型（SBM）的群体检测问题。利用批坐标上升（batch coordinate ascent， BCAVI）进行更新，我们给出了所有临界点的完整表征，并展示了不同初始化的收敛行为。当参数已知时，我们证明了很大比例的随机初始化将收敛于基本真理。另一方面，当需要对参数本身进行估计时，随机初始化将收敛到无信息的局部最优。
Variational approximation has been widely used in large-scale Bayesian inference recently, the simplest kind of which involves imposing a mean field assumption to approximate complicated latent structures. Despite the computational scalability of mean field, theoretical studies of its loss function surface and the convergence behavior of iterative updates for optimizing the loss are far from complete. In this paper, we focus on the problem of community detection for a simple two-class Stochastic Blockmodel (SBM). Using batch co-ordinate ascent (BCAVI) for updates, we give a complete characterization of all the critical points and show different convergence behaviors with respect to initializations. When the parameters are known, we show a significant proportion of random initializations will converge to ground truth. On the other hand, when the parameters themselves need to be estimated, a random initialization will converge to an uninformative local optimum.