Generalization Capability of Memorization Leaning

监督学习的目的是能够正确回答不一定包括在训练示例中的查询，即，以获得更高水平的泛化能力。然而，大多数学习方法，如误差反向传播是所谓的记忆学习，其目的是减少错误的训练样本。因此，在理论上无法保证最佳的泛化能力，这就产生了如下问题：首先，要弄清为什么记忆学习不需要泛化能力，但却可以获得更高水平的泛化能力。第二个问题是明确记忆学习有效的适用范围。第三，进一步拓展适用范围的方法。对于第一个问题，通过引入可容许性的概念，给出了清晰的解释。第二 ...更多信息 问题，我们引入了一族投影学习的概念，它允许我们在理论上同时讨论无限多种学习方法。利用可容许性和投射学习族的概念，阐明了狭义记忆学习对投射学习族的适用范围。对于第三个问题，我们提出了大量的解决方案：我们将记忆学习的概念从死记硬背扩展到纠错记忆学习，进一步扩大了适用范围。从主动学习的角度出发，给出了最大限度地提高泛化能力的训练样本的设计方法。此外，从模型选择的角度来看，我们提出了子空间信息准则（SIC），这是一个模型选择标准，其有效性理论上保证了有限数量的训练样本。以SIC为例，给出了最优正则化参数的设计方法。少

The purpose of supervised learning is to be able to answer correctly to queries that are not necessarily included in the training examples, i.e., to acquire a higher level of the generalization capability. However, most of the learning methods such as the error back-propagation are the so-called memorization learning, which is aimed at reducing the error only for training examples. Therefore, there is no theoretical guarantee for optimal generalization.This gives rise to the following problems: First is to clarify the reason why a higher level of the generalization capability can be acquired by the memorization learning despite the fact that it does not require the generalization capability. The second problem is to clarify the range of applicability that: the memorization learning works effectively. Third is to develop methods for further expanding the range of applicability.For the first problem, we gave a lucid explanation by introducing the concept of admissibility. For the second … More problem, we introduced the concept of a family of projection learnings which allows us to theoretically discuss an infinitely many kinds of learning methods simultaneously. Utilizing the concepts of admissibility and a family of projection learning, we clarified the range of applicability of the memorization learning in the narrow sense with respect to a family of projection learnings. For the third problem, we showed that there are a large number of solutions : We extended the concept of the memorization learning from the rote memorization learning to the error corrected memorization learning, which further enlarges the range of applicability. From the view point of active learning, we gave design methods of training examples that maximally enhance the generalization capability. Furthermore, from the standpoint of model selection, we proposed the subspace information criterion (SIC) , which is a model selection criterion with its effectiveness theoretically guaranteed for a finite number of training examples. Based on SIC, we gave, for example, a design method of the optimal regularization parameter. Less