Stochastic Complexity of Bayesian Networks
Stochastic Complexity of Bayesian Networks
复制标题
贝叶斯网络的随机复杂性
DOI：
10.1109/tnn.2004.841792
复制
发表时间：
2002
期刊：
影响因子：
--
通讯作者：
Sumio Watanabe
中科院分区：
文献类型：
--
作者：
Keisuke Yamazaki;Sumio Watanabe
例如，贝叶斯网络现在被用于许多领域。系统诊断数据挖掘、聚类等。尽管它们的应用范围很广，但由于模型的不可识别性和非规则性，其统计特性尚未得到澄清。在贝叶斯网络中。较小模型的参数集是在大模型的参数空间中具有奇点的解析集。由于这些奇异性，Fisher信息矩阵不是正定的。换句话说，学习的数学基础还没有建立起来。然而，近年来，我们已经开发了一种方法来分析非正则模型，使用代数几何。该方法揭示了模型的奇异性与其统计性质之间的关系。本文将该方法应用于具有潜在变量的贝叶斯网络，阐明了随机复杂性的阶数。我们的结果表明，它们的上界小于参数空间的维数。这意味着贝叶斯泛化误差也远小于常规模型，并且施瓦茨的模型选择准则BIC需要针对贝叶斯网络进行改进。
Bayesian networks arc now used in enormous fields, for example. system diagnosis. data mining, clusterings etc. In spite of wide range of their applications, the statistical properties have not yet bcen clarified because the models are nonidentifiable and non-regular. In a Bayesian network. the set of parameters for a smaller model is an analytic set with singularities in the parameter space of a large model. Because of these singularities, the Fisher information matrices are not positive definite. In other words, the mathematical foundation for learning has not been constructed. In recent years, however, we have developed a method to analyze nonregular models by using algebraic geometry. This method revealed the relation between model's singularities and its statistical properties. In this paper, applying this method to Bayesian networks with latent variables, we clarify the orders of the stochastic complexities. Our result shows that their upper bound is smaller than thc dimension of the parameter space. This means that the Bayesian generalization error is also far smaller than that of a regular model, and that Schwarz's model selection criterion BIC needs to be improved for Bayesian networks.