Establishment of Algebraic Geometrical Methods in Statistical Learning Theory

统计学习理论中代数几何方法的建立

• 批准号: 15500130
• 负责人: WATANABE Sumio
• 金额: $ 1.86万
• 依托单位: Tokyo Institute of Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 2003
• 资助国家: 日本
• 起止时间: 2003 至 2006
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-15500130/
• 关键词: Algebraic Geometry; Statistical Learning Theory; Zeta function; Learning System; Learning Machine; Learning Theory; Algebraic Analysis; Information System; 特異モデル; 確率的複雑さ; ベイズ法; 変分ベイズ法; ゼータ函数; 縮小ランク回帰; 文脈自由文法; 汎化誤差; カルバック情報量; ブローアップ; 特定不能

A lot of learning machines such as artificial neural networks, normal mixtures, hidden Markov models, Boltzmann machines, Bayesian networks, reduced rank regressions are nonidentifiable and singular learning machines. They have degenerate Fisher Information matrices resulting that the conventional statistical learning theory does not hold. In order to establish the new learning theory, we need algebraic geometrical method to treat the likelihood functions. In this research, we have the following results. (1) The fundamental form of the likelihood function were introduced based on desingularization problem. (2) The asymptotic form of the marginal likelihood were derived based on the zeta function of learning theory. (3) The asymptotic behavior of the variational Bayes or the mean field approximation were obtained. (4) The asymptotic behavior of the empirical Bayes appriximation were obtained. These are the mathematical foundation originally established in this research.

许多学习机器，如人工神经网络，正常的混合，隐马尔可夫模型，玻尔兹曼机，贝叶斯网络，降秩回归是不可识别的和奇异的学习机器。它们具有退化的Fisher信息矩阵，导致传统的统计学习理论不成立。为了建立新的学习理论，需要用代数几何方法来处理似然函数。在本研究中，我们有以下结果。(1)基于去奇异化问题，给出了似然函数的基本形式。(2)基于学习理论中的zeta函数，导出了边缘似然的渐近形式。(3)得到了变分Bayes或平均场近似的渐近性态。(4)得到了经验贝叶斯近似的渐近性态。这些是本研究最初建立的数学基础。

项目成果

Stochastic complexity of hidden markov model in Variational Bayes learning.

变分贝叶斯学习中隐马尔可夫模型的随机复杂性。

• 发表时间: 2006
• 期刊: IEICE Transactions Vol.J89-D-II
• 作者: Tikara Hosino;Kazuho Watanabe;Sumio Watanabe
• 通讯作者: Sumio Watanabe

渡辺澄夫: "特異モデルとベイズ学習"日本神経回路学会誌. 10・4. 211-219 (2003)

Sumio Watanabe：“奇异模型和贝叶斯学习”日本神经网络学会杂志 10・4（2003 年）。

Algebraic geometry and stochastic complexity of hidden Markov models

• DOI: 10.1016/j.neucom.2005.02.014
• 发表时间: 2005-12-01
• 期刊: NEUROCOMPUTING
• 影响因子: 6
• 作者: Yamazaki, K;Watanabe, S
• 通讯作者: Watanabe, S

代数幾何と学習理論

代数几何与学习理论

• 发表时间: 2006
• 作者: 山崎啓介;渡辺澄夫;渡辺澄夫;渡辺澄夫
• 通讯作者: 渡辺澄夫

Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation

• 发表时间: 2006-12
• 期刊: J. Mach. Learn. Res.
• 作者: Kazuho Watanabe;Sumio Watanabe