Active learning for optimally generalizing neural networks

从非线性函数逼近的角度讨论了主动学习问题。使用固定数量的训练集可实现的泛化能力水平高度依赖于所使用的数据的质量。同样有趣的是，许多自然学习系统（如人类）并不是简单的被动学习，而是利用至少某种形式的主动学习来检查问题域。主动学习是指任何形式的学习，其中学习程序对它训练的输入有一定的控制。主动学习的关键问题是“最佳数据选择”和“增量学习”。对于第一个问题，我们给出了一种设计训练数据的方法，该训练数据提供关于Wiener学习准则的最佳泛化能力;对于第二个问题，我们设计了计算效率高的方法，当有新的训练数据可用时，增量地计算学习算子和学习函数。 ...更多信息 标签。注意，该方法提供了与使用整个训练数据的批量学习相同的泛化能力。这些增量式的结果被成功地应用于一个2-自由度（DOF）机器人手臂的感觉运动映射的学习。我们还提出了一种新的主动学习，这是与两个学习标准。误差反向传播算法常用于前馈神经网络的学习。然而，据说训练误差的减小并不意味着泛化误差的减小，而是可能导致泛化能力降低。这种现象被称为过度学习。为了克服这个问题，我们引入了容许性的概念，它保证了一个学习标准成为另一个标准的替代。此外，我们还阐明了两个学习准则之间的可容许性由所使用的训练数据控制。在这项研究中，我们设计了一种方法来实现的容许性时，增加和/或删除训练数据的容许性不成立的一组给定的训练数据。少

The problem of active learning was discussed from the point of view of nonlinear function approximation. The level of generalization ability achievable with a fixed number of training set is highly dependent on the quality of the data used. It is also interesting to note that many natural learning systems like humans are not simply passive, but make use of at least some form of active learning to examine the problem domain. By active learning, we mean any form of learning in which the learning program has some control over the inputs over which it trains.The key problems in active learning are 'optimal data selection' and 'incremental learning'. For the first problem, we gave a method for designing the training data which provide the optimal generalization ability with respect to the Wiener learning criterion.For the second problem, we devised computationally efficient means of incrementally computing the learning operator and the learned function when a new training datum is made avai … More lable. Note that the method provides the same generalization ability as the batch learning with the entire training data. These incremental results were successfully applied to the learning of sensorimotor maps of a 2-Degree Of Freedom (DOF) robot arm.We also proposed a new type of active learning, which is related to two learning criteria. The error-backpropagation algorithm is often used for learning in feedforward neural networks. However, it is said that a decrease of the training error does not imply a decrease of generalization error, but may lead to lower generalization ability. Such a phenomenon is called the over-learning. In order to overcome the problem, we have introduced the concept of admissibility, which guarantees for a learning criterion to become a substitution of another criterion. Furthermore, we have clarified that the admissibility between two learning criteria is controled by the training data used. In this study, we devised a method to realize admissibility by adding and/or deleting training data when admissibility does not hold for a given set of training data. Less