权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Neural network learning with regulatizers and generalization ability

具有调节器和泛化能力的神经网络学习

基本信息

批准号：
09680371
负责人：
ISHIKAWA Masumi
金额：
$ 1.41万
依托单位：
Kyushu Institute of Technology
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (C)
财政年份：
1997
资助国家：
日本
起止时间：
1997 至 1998
项目状态：
已结题

项目摘要

The relation between neural network learning with regularizes and generalization ability is clarified both theoretically and empirically. In the present theoretical study, a Laplacian regularize, a Gaussian regularize and their combinations are considered.In the first stage, various empirical procedures in a structural learning with forgetting proposed by the authors are theoretically clarified. For example, a structural learning with selective forgetting has something to do with the line process in vision under the assumption that a scene is almost smooth except a small number of discontinuous points.In the second stage, a estimation of mean value using regularization is theoretically studied. It is the simplest case of multiple regression models. It is demonstrated that the proposed regularization method is effectively applied to real data.In the third stage, excessive simplification in a formulation of generalization errors in multiple regression models is rectified. So for it has been assumed that input variables are mutually independent, and true model parameters and a noise variance are known a priori. We modified formulations so as to allow correlations between input variables. We propose a novel procedure for theoretically evaluating regularizes based on data. Firstly, we estimate model parameters and a noise variance from data. Secondly, assuming that these estimates are true, we calculate the optimal regularization parameters and model parameters by the previously proposed method. It provides, hopefully, their better estimates. Thirdly, assuming that the resulting estimates are true, we again calculate the optimal regularization parameters and model parameters. This procedure can be repeated iteratively. This iterative estimation is a key idea of the present study. Applications of the proposed method to real data demonstrates that better estimates with smaller generalization errors are obtained successfully.

从理论和实证两方面阐明了带正则化的神经网络学习与泛化能力之间的关系。本文的理论研究主要包括Laplacian正则化、Gaussian正则化以及它们的组合。在第一阶段中，从理论上阐明了作者提出的带遗忘的结构学习中的各种经验过程。例如，在场景除了少数不连续点之外几乎平滑的假设下，具有选择性遗忘的结构学习与视觉中的线过程有关。在第二阶段，从理论上研究了使用正则化的均值估计。这是多元回归模型中最简单的情况。结果表明，所提出的正则化方法能有效地应用于真实的数据。第三阶段，纠正了多元回归模型泛化误差公式中的过度简化。因此，假设输入变量是相互独立的，并且真实的模型参数和噪声方差是先验已知的。我们修改配方，以便允许输入变量之间的相关性。我们提出了一个新的程序，从理论上评估正则化的数据。首先，我们从数据中估计模型参数和噪声方差。其次，假设这些估计是真的，我们计算的最佳正则化参数和模型参数由以前提出的方法。希望它能提供更好的估计。第三，假设得到的估计是真的，我们再次计算最优正则化参数和模型参数。该过程可以迭代地重复。这种迭代估计是本研究的一个关键思想。对真实的数据的应用表明，该方法能获得较好的估计和较小的推广误差。