权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Study about a construction of optimally generalizing neural networks

最优泛化神经网络的构建研究

基本信息

批准号：
06452399
负责人：
OGAWA Hidemitsu
金额：
$ 3.84万
依托单位：
TOKYO INSTITUTE OF TECHNOLOGY
依托单位国家：
日本
项目类别：
Grant-in-Aid for General Scientific Research (B)
财政年份：
1994
资助国家：
日本
起止时间：
1994 至 1995
项目状态：
已结题

项目摘要

In this sutdy, we first formalized the problem of training a neural network as one of an inverse problem in function approximation. Next, we provided necessary and sufficient conditions for optimal generalization capability in terms of the number of hidden units, the basis functions, and the weights connected to each hidden unit. Furthermore, we gave a methodology to construct neural networks with optimal generalization capability.From this methodology, we can see that are an infinite numbers of neural networks with the same generalization capability. From among this infinite number, we specified the ones which are most robust with respect to some kinds of faults which may occur in actual usage. Concretely, we gave methods to decide weights in neural networks which optimally suppress the influences of each of the following faults : a proportional errors in the weights, a connection fault, and a stuck-at gamma fault. Moreover, we gave the methods to decide not only weights but also basis functions for the hidden units which optimally suppress the above three faults.Next we constructed a method for incremental learning in which only the current network and one new datum are used to obtain a new network at each step, while maintaining the property that the neural network is optimally generalizing with respect to all of the data learned so far. We think our results have the potential for being applied to the problem of active learning.Moreover, we gave a solution to the problem of over-learning which occurs in training of neural networks using the error-backpropagation algorithm, i.e., we introduced the concept of admissibility defined by relation-ship between two learning criteria. According to the admissibility, we devised methods for choosing training data to prevent over-leaning.

本文首先将神经网络的训练问题形式化为函数逼近中的一个反问题。接下来，我们提供了最佳泛化能力的必要条件和充分条件，隐含单元的数量，基函数和连接到每个隐含单元的权重。此外，我们还给出了一种构造具有最优泛化能力的神经网络的方法，从这个方法中我们可以看出，有无限多个神经网络具有相同的泛化能力。从这个无限的数字中，我们指定了那些最强大的相对于某些类型的故障，可能会发生在实际使用中。具体地说，我们给出的方法来决定神经网络的权重，最佳地抑制以下故障的影响：一个比例误差的权重，连接故障，和一个固定的伽玛故障。此外，我们给出了确定隐层单元的权值和基函数的方法，从而最优地抑制了上述三种故障。接下来，我们构造了一种增量学习方法，在该方法中，每一步只使用当前网络和一个新数据来获得一个新网络，同时保持神经网络对迄今为止所学习的所有数据进行最优泛化的性质。我们认为我们的结果具有应用于主动学习问题的潜力。此外，我们给出了使用误差反向传播算法训练神经网络时出现的过学习问题的解决方案，即，我们引入了由两个学习准则之间的关系定义的容许性的概念。根据可接受性，我们设计了选择训练数据的方法，以防止过度学习。