权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Aresearch of statistical properties of singular models such as a multi-layer perceptron

多层感知器等奇异模型统计特性的研究

基本信息

批准号：
18500171
负责人：
HAGIWARA Katsuyuki
金额：
$ 1.6万
依托单位：
Mie University
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (C)
财政年份：
2006
资助国家：
日本
起止时间：
2006 至 2007
项目状态：
已结题

项目摘要

The multi-layer perceptron is known to be a singular model in which the Fisher information matrix can be singular in some cases. The singularity comes from parametric basis functions by which the shapes of basis functions vary. To reveal statistical properties of a singular model, in this research, we focus on the variable basis functions. We derived a upper bound of the degree of over-fitting under a restriction on basis function outputs. For example, the restriction can be achieved by restricting an input weight space in a multi-layer perceptron. By applying this result, we showed that, for a regression using one Gaussian unit, a very small value is frequently obtained for a width parameter in training. On the other hand, we derived the expectations of the training error and generalization error of learning machine in which basis functions which are chosen from a finite set of orthogonal functions. This clarifies that AIC type model selection criteria for machines with variable basis functions need an information of a target function. We solve this problem by applying a shrinkage method. From a viewpoint of variable basis functions, furthermore, we proposed a shrinkage method for a nonparametric regression. The method produces a machine with low generalization error in less computational time. The results obtained in this research helps for clarifying statistical properties of a machine with variable basis functions, thus, a singular model such as multi-layer perceptrons.

已知多层感知器是一个奇异模型，其中Fisher信息矩阵在某些情况下可以是奇异的。奇异性来源于基函数的形状变化的参数基函数。为了揭示奇异模型的统计性质，本研究主要关注变量基函数。在基函数输出的限制下，导出了过拟合度的上界。例如，可以通过限制多层感知器中的输入权空间来实现限制。通过应用这一结果，我们表明，对于使用一个高斯单元的回归，训练中宽度参数经常得到一个非常小的值。另一方面，我们推导了从有限正交函数集合中选择基函数的学习机的训练误差和泛化误差的期望。这阐明了变基函数机器的AIC型模型选择准则需要目标函数的信息。我们用收缩法解决了这个问题。进一步，从变基函数的角度出发，提出了一种非参数回归的收缩方法。该方法在较短的计算时间内得到了一个泛化误差小的机器。本研究的结果有助于澄清具有可变基函数的机器的统计性质，因此，多层感知器等奇异模型。