Neural networks with excellent abilities in learning and recognition problems for complex patterns

误差反向传播是一种众所周知的、功能强大的前馈多层神经网络监督学习算法，但它引入的问题是其学习过程经常陷入元稳定状态。已有研究表明，以Kullback散度代替传统的二次误差作为误差度量的误差反向传播算法具有良好的学习倾向，有望适用于各种问题，即使不知道其最优网络大小。针对手写数字字符的识别问题，对该算法进行了细致的研究，讨论了该算法的可扩展性、灵活性、损伤容忍度以及对非学习数据的识别能力。对三层前馈网络进行了数值研究；一个输入层，一个隐藏的中间层，有不同数量的神经元，最多8129个，还有一个输出层。数值研究结果表明，该学习算法具有较高的泛化能力，具有较强的实际应用价值。这些网络能力也会通过扩大网络规模而得到改善。如果我们准备一个足够大的网络，经过几个学习步骤，我们可以很容易地获得99%以上的识别率，未学习字符的识别率超过90%。这些研究成果已在《仙台工业大学研究报告》上发表了几篇论文，并在东北地区电气与信息工程师学会联合会议等各种会议上发表。

Error back propagation is a well-known and powerful supervised learning algorithm for feed-forward multi-layered neural networks, but it is known to introduce the problem of its learning process being often trapped in a meta-stable state. It has been reported that the error back propagation algorithm with Kullback divergence instead of conventional quadratic error as an error measure demonstrates an excellent learning tendency and is expected to be applicable to various problems, even without the foreknowledge of their optimal network size.We have studied this algorithm carefully based on the identification problem of handwritten numerical characters and discussed its abilities such as scalability, flexibility, damage tolerance, and recognition of unlearned data. The numerical studies have been done on the three-layered feed-forward network ; an input layer, a hidden middle layer with various numbers of neurons up to 8129, and an output layer. Results of numerical studies indicate that this learning algorithm can have high generalization ability and be sufficiently powerful for practical use. Those network abilities are also improved by increasing its size. If we prepare an enough size of network, we could easily obtain the identification rate of higher than 99% after a few learning steps and recognition rate for unlearned characters over 90%.Those research results have been published on several papers in Research Reports on Sendai National College of technology, and also presented the papers at various conferences such as Tohoku-section Joint Convention of Institutes of Electrical And Information Engineers.