Singularities in mixture models and upper bounds of stochastic complexity
Singularities in mixture models and upper bounds of stochastic complexity
复制标题
DOI：
10.1016/s0893-6080(03)00005-4
复制
发表时间：
2003-09-01
期刊：
影响因子：
7.8
通讯作者：
Watanabe, S
中科院分区：
文献类型：
--
作者：
Yamazaki, K;Watanabe, S
学习机是几种分布的混合，例如高斯混合或专家混合，具有广泛的应用。然而，这样的机器是一个不可识别的统计模型，在参数空间中有很多奇异点，因此它的泛化性能是未知的。最近，一种代数几何方法已经开发出来，使我们能够对待这样的学习机数学。基于此方法，本文严格证明了混合学习机具有比常规统计模型更小的贝叶斯随机复杂度。由于学习机的泛化误差等于随机复杂度的增加，本文的结果表明，如果在统计推断中应用贝叶斯估计，混合模型可以获得比常规统计模型更精确的预测。(C)2003爱思唯尔科技有限公司版权所有。
A learning machine which is a mixture of several distributions, for example, a gaussian mixture or a mixture of experts, has a wide range of applications. However, such a machine is a non-identifiable statistical model with a lot of singularities in the parameter space, hence its generalization property is left unknown. Recently an algebraic geometrical method has been developed which enables us to treat such learning machines mathematically. Based on this method, this paper rigorously proves that a mixture learning machine has the smaller Bayesian stochastic complexity than regular statistical models. Since the generalization error of a learning machine is equal to the increase of the stochastic complexity, the result of this paper shows that the mixture model can attain the more precise prediction than regular statistical models if Bayesian estimation is applied in statistical inference. (C) 2003 Elsevier Science Ltd. All rights reserved.