Learning of Bayesian Neural Networks and Their Applications to Hidden Markov Chain

这项研究的目的是开发一种复杂的神经网络，它可以学习贝叶斯判别函数，并用它来估计隐藏的马尔可夫链。我们在科学研究助学金c资助的研究期间所取得的成果可以概括为三点。三层神经网络可以学习贝叶斯判别函数，在我们开始本工作之前就已经提出了。然而，它在学习上有困难。由于人们普遍认为，具有较少单元的神经网络可以学习得更好，因此我们首先尝试减少隐藏单元。2.在概率分布简单的情况下，隐单元数最少的网络可以用来估计隐马尔可夫链。当该网络具有参数单元时，可以同时学习隐马尔可夫链的几种状态对应的多个贝叶斯判别函数。3.一般情况下，该网络不能学习判别函数。原因是，用两种不同的教师信号进行学习是困难的。因此，我们构造了一种新型的神经网络，其中隐含单元的自由度是有限的。虽然这不可避免地会导致隐藏单元的增加，但网络的性能更好。该理论已在一篇现已出版的论文中阐述，仿真结果已在国内和几个国际会议上公布。因此，当概率分布简单时，网络可以估计隐藏马尔可夫链。即使在一般情况下，最近的结果也是有希望的。

The goal of this research was to develop a sophisticated neural network which can learn the Bayesian discriminant function and, to use it to estimate the hidden Markov chain. The results we have obtained during the period of the research supported by the Grant-in-Aid for Scientific Research c can be summarized into three points.1. The three layer neural network, which may learn a Bayesian discriminant function, had been proposed before we started the present work. However, it had difficulty in learning. As it is a general belief that a neural network having fewer units can learn better, we first tried to decrease the hidden units. We have theoretically proved that the small number of the hidden units of our network is actually the minimum.2.In the rase where the probability distributions are simple, this network, having the minimum number of hidden units, can be used for estimating the hidden Markov chain. When this network is equipped with parameter units, it can learn simultaneously several Bayesian discriminant functions respectively corresponding to the several states of the hidden Markov chain.3.However, the network cannot learn the dicriminant functions in general cases. The reason is that learning with dichotomous teacher signals is difficult. So we constructed a new type of neural network, where the degree of freedom of the hidden units is limited. Though this inevitably causes an increment of hidden units, the network performs better The theory is stated in a paper which is now in printing, and the simulation results have been presented at a domestic and several international conferences.Thus, when the probability distributions are simple the network can estimate the hidden Markov chains. Even in general cases, the recent results are promising.