Multi-directional search from the primitive initial point for Gaussian mixture estimation using variational Bayes method
Multi-directional search from the primitive initial point for Gaussian mixture estimation using variational Bayes method
复制标题
DOI：
10.1016/j.neunet.2009.08.003
复制
发表时间：
2010-04
期刊：
影响因子：
--
通讯作者：
Yuta Ishikawa;I. Takeuchi;R. Nakano
中科院分区：
文献类型：
--
作者：
Yuta Ishikawa;I. Takeuchi;R. Nakano
高斯混合模型（GMM）由于能够近似各种形式的概率分布而被广泛应用。本文利用变分贝叶斯方法研究了广义矩估计问题。在这种方法中，人们只能找到局部最优，因为问题的自由能函数是多峰的。为了找到更好的解决方案，最近将确定性退火应用于VB方法（DAVB方法）。在本文中，我们提供了一种替代方法的DAVB方法的GMM估计问题。我们提出了一种从原始初始点（PIP）开始的多方向搜索方法，该原始初始点被定义为DAVB方法在最高温度下的解。对原始（未退火）自由能函数的曲率信息的研究表明，PIP是一个鞍点。利用Hessian矩阵的特征分析，提出了一种有效的多方向搜索策略。利用真实的数据集进行的数值实验表明了该方法的有效性。
Gaussian mixture model (GMM) is widely used in many applications because it can approximate various forms of probability distributions. In this paper, we are concerned with GMM estimation problem using the variational Bayes (VB) method. In this approach, one can only find local optima because the free energy function of the problem is multimodal. In order to find better solutions, deterministic annealing was recently adapted to the VB method (DAVB method). In this paper, we offer an alternative approach to the DAVB method for GMM estimation problem. We propose a multi-directional search method from the primitive initial point (PIP), which is defined as the solution of the DAVB method at the highest temperature. Investigation on the curvature information of the original (not annealed) free energy function reveals that the PIP is a saddle point. An efficient multi-directional search strategy from the neighborhoods of the PIP is proposed using the eigen-analysis of the Hessian matrix. Numerical experiments using real data sets demonstrate the effectiveness of our method.