Establishment of Algebraic Geometrical Methods in Statistical Learning Theory

许多学习机，如人工神经网络、正态混合、隐马尔可夫模型、Boltzmann机器、贝叶斯网络、降阶回归等都是不可识别的奇异学习机。它们具有退化的Fisher信息矩阵，导致传统的统计学习理论不成立。为了建立新的学习理论，需要用代数几何的方法来处理似然函数。在本研究中，我们得到了以下结果。(1)基于去奇异化问题，给出了似然函数的基本形式。(2)基于学习理论中的Zeta函数，推导了边缘似然的渐近形式。(3)得到了变分贝叶斯近似或平均场近似的渐近行为。(4)得到了经验贝叶斯逼近的渐近行为。这些都是本研究最初建立的数学基础。

A lot of learning machines such as artificial neural networks, normal mixtures, hidden Markov models, Boltzmann machines, Bayesian networks, reduced rank regressions are nonidentifiable and singular learning machines. They have degenerate Fisher Information matrices resulting that the conventional statistical learning theory does not hold. In order to establish the new learning theory, we need algebraic geometrical method to treat the likelihood functions. In this research, we have the following results. (1) The fundamental form of the likelihood function were introduced based on desingularization problem. (2) The asymptotic form of the marginal likelihood were derived based on the zeta function of learning theory. (3) The asymptotic behavior of the variational Bayes or the mean field approximation were obtained. (4) The asymptotic behavior of the empirical Bayes appriximation were obtained. These are the mathematical foundation originally established in this research.