Learning of Bayesian Neural Networks and Their Applications to Hidden Markov Chain

贝叶斯神经网络的学习及其在隐马尔可夫链中的应用

• 批准号: 17500153
• 负责人: ITO Yoshifusa
• 金额: $ 2.18万
• 依托单位: Aichi Gakuin University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 2005
• 资助国家: 日本
• 起止时间: 2005 至 2007
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-17500153/
• 关键词: Neural network; Hidden Markov chain; Bayesian decision; ベイズ判別関数; 三層神経回路網; 学習; ベイズ神経回路網; 判別関数; 関数近似; 多項式

The goal of this research was to develop a sophisticated neural network which can learn the Bayesian discriminant function and, to use it to estimate the hidden Markov chain. The results we have obtained during the period of the research supported by the Grant-in-Aid for Scientific Research c can be summarized into three points.1. The three layer neural network, which may learn a Bayesian discriminant function, had been proposed before we started the present work. However, it had difficulty in learning. As it is a general belief that a neural network having fewer units can learn better, we first tried to decrease the hidden units. We have theoretically proved that the small number of the hidden units of our network is actually the minimum.2.In the rase where the probability distributions are simple, this network, having the minimum number of hidden units, can be used for estimating the hidden Markov chain. When this network is equipped with parameter units, it can learn simultaneously several Bayesian discriminant functions respectively corresponding to the several states of the hidden Markov chain.3.However, the network cannot learn the dicriminant functions in general cases. The reason is that learning with dichotomous teacher signals is difficult. So we constructed a new type of neural network, where the degree of freedom of the hidden units is limited. Though this inevitably causes an increment of hidden units, the network performs better The theory is stated in a paper which is now in printing, and the simulation results have been presented at a domestic and several international conferences.Thus, when the probability distributions are simple the network can estimate the hidden Markov chains. Even in general cases, the recent results are promising.

本研究的目标是开发一个复杂的神经网络，可以学习贝叶斯判别函数，并使用它来估计隐马尔可夫链。我们在科学研究补助金资助的研究期间所取得的成果可以概括为三点。三层神经网络，它可以学习贝叶斯判别函数，已经提出了之前，我们开始本工作。然而，它在学习上有困难。由于人们普遍认为具有较少单元的神经网络可以更好地学习，因此我们首先尝试减少隐藏单元。从理论上证明了该网络的隐单元数是最小的。2.在概率分布简单的情况下，具有最小隐单元数的网络可用于隐马尔可夫链的估计。当该网络具有参数单元时，它可以同时学习分别对应于隐马尔可夫链多个状态的多个贝叶斯判别函数。3.但在一般情况下，该网络不能学习判别函数。原因是，学习与二分法教师信号是困难的。因此，我们构造了一种新型的神经网络，其中隐层单元的自由度是有限的。虽然这不可避免地导致隐单元的增加，但网络性能更好。理论在一篇论文中阐述，现在正在印刷，并且模拟结果已在国内和几个国际会议上发表。因此，当概率分布是简单的网络可以估计隐马尔可夫链。即使在一般情况下，最近的结果也是有希望的。

项目成果

Learning of neural networks with dichotomic random teacher signals

使用二分随机教师信号学习神经网络

• 发表时间: 2007
• 作者: Yoshifusa Ito;Cidambi Srinivasan;Hiroyuki Izumi
• 通讯作者: Hiroyuki Izumi

Learning of Bayesian discriminant functions by $a$ layered neural network

通过$a$分层神经网络学习贝叶斯判别函数

• 发表时间: 2007
• 作者: Yoshifusa Ito;Cidambi Srinivasan;Hiroyuki Izumi
• 通讯作者: Hiroyuki Izumi

Bayesian decision theory on three-layer neural networks

三层神经网络的贝叶斯决策理论

• 发表时间: 2005
• 期刊: Neurocomputing 63
• 作者: Yoshifusa Ito;Cidambi Srinivasan
• 通讯作者: Cidambi Srinivasan

2値乱数による神経回路網の学習とベイズ判別関数学習への応用

使用二进制随机数学习神经网络及其在贝叶斯判别函数学习中的应用

• 发表时间: 2007
• 作者: 伊藤嘉房;キダンビ スリニヴァサン;泉寛幸
• 通讯作者: 泉寛幸

Simultaneous approximation of polynomials and derivatives and their applications to neural networks

多项式和导数的同时逼近及其在神经网络中的应用

• 发表时间: 2008
• 期刊: Neural Computation 20(ゲラ刷りによる)
• 作者: WATANABE Takesato;YAMAGUCHI Koji;KUDO Kazuo;KAWASAKI Yoshinori;NOHARA Hitoshi;Yoshifusa Ito,
• 通讯作者: Yoshifusa Ito,