Small-Variance Asymptotics for Bayesian Nonparametric Models with Constraints
Small-Variance Asymptotics for Bayesian Nonparametric Models with Constraints
复制标题
带约束的贝叶斯非参数模型的小方差渐近
DOI：
10.1007/978-3-319-18032-8_8
复制
发表时间：
2015
影响因子：
--
通讯作者：
S. Venkatesh
中科院分区：
文献类型：
--
作者：
Cheng Li;Santu Rana;Dinh Q. Phung;S. Venkatesh
当采用贝叶斯非参数模型（BNP）时，用户通常具有额外的知识，例如，对于聚类，可能存在一些数据实例应该在同一聚类（必须链接约束）或不同聚类（不能链接约束）中的先验知识，并且类似地，对于主题建模，由于底层语义，一些词应该被分组在一起或分开。这可以通过基于这种约束施加适当的采样概率来实现。然而，通过Gibbs采样的BNP模型的传统推断技术是耗时的，并且对于大数据是不可扩展的。变分近似法更快，但很多时候它们不能提供好的解决方案。针对这一点，我们提出了一个小方差渐近分析的MAP估计BNP模型的约束。推导了带约束的Dirichlet过程混合模型的目标函数，并设计了一种简单有效的K-means算法。我们进一步扩展的小方差分析层次BNP模型的约束，并设计了一个类似的简单的目标函数。在合成数据集和真实的数据集上的实验证明了算法的有效性。
The users often have additional knowledge when Bayesian nonparametric models (BNP) are employed, e.g. for clustering there may be prior knowledge that some of the data instances should be in the same cluster (must-link constraint) or in different clusters (cannot-link constraint), and similarly for topic modeling some words should be grouped together or separately because of an underlying semantic. This can be achieved by imposing appropriate sampling probabilities based on such constraints. However, the traditional inference technique of BNP models via Gibbs sampling is time consuming and is not scalable for large data. Variational approximations are faster but many times they do not offer good solutions. Addressing this we present a small-variance asymptotic analysis of the MAP estimates of BNP models with constraints. We derive the objective function for Dirichlet process mixture model with constraints and devise a simple and efficient K-means type algorithm. We further extend the small-variance analysis to hierarchical BNP models with constraints and devise a similar simple objective function. Experiments on synthetic and real data sets demonstrate the efficiency and effectiveness of our algorithms.