Neural networks with excellent abilities in learning and recognition problems for complex patterns

点击翻译按钮获取中文摘要

Error back propagation is a well-known and powerful supervised learning algorithm for feed-forward multi-layered neural networks, but it is known to introduce the problem of its learning process being often trapped in a meta-stable state. It has been reported that the error back propagation algorithm with Kullback divergence instead of conventional quadratic error as an error measure demonstrates an excellent learning tendency and is expected to be applicable to various problems, even without the foreknowledge of their optimal network size.We have studied this algorithm carefully based on the identification problem of handwritten numerical characters and discussed its abilities such as scalability, flexibility, damage tolerance, and recognition of unlearned data. The numerical studies have been done on the three-layered feed-forward network ; an input layer, a hidden middle layer with various numbers of neurons up to 8129, and an output layer. Results of numerical studies indicate that this learning algorithm can have high generalization ability and be sufficiently powerful for practical use. Those network abilities are also improved by increasing its size. If we prepare an enough size of network, we could easily obtain the identification rate of higher than 99% after a few learning steps and recognition rate for unlearned characters over 90%.Those research results have been published on several papers in Research Reports on Sendai National College of technology, and also presented the papers at various conferences such as Tohoku-section Joint Convention of Institutes of Electrical And Information Engineers.