Algebraic Analysis for Singular Statistical Estimation
Algebraic Analysis for Singular Statistical Estimation
复制标题
奇异统计估计的代数分析
DOI：
10.1007/3-540-46769-6_4
复制
发表时间：
1999
影响因子：
4.4
通讯作者：
Sumio Watanabe
中科院分区：
文献类型：
--
作者：
Sumio Watanabe
本文阐明了一个非正则参数模型，如神经网络的学习效率，其真正的参数集是一个具有奇点的解析簇。利用Sato的b-函数严格证明了自由能或贝叶斯随机复杂度渐近等于λ1 log n -（m1 - 1）log log n+常数，其中λ1为有理数，m1为自然数，n为训练样本数.同时给出了一种基于奇异性分解的λ1和m1的计算方法。在规则模型中，2λ1等于参数的数量，m1 = 1，而在非规则模型（如神经网络）中，2λ1小于参数的数量，m1 ≥ 1。
This paper clarifies learning efficiency of a non-regular parametric model such as a neural network whose true parameter set is an analytic variety with singular points. By using Sato's b-function we rigorously prove that the free energy or the Bayesian stochastic complexity is asymptotically equal to λ1 log n - (m1 - 1) log log n+constant, where λ1 is a rational number, m1 is a natural number, and n is the number of training samples. Also we show an algorithm to calculate λ1 and m1 based on the resolution of singularity. In regular models, 2λ1 is equal to the number of parameters and m1 = 1, whereas in non-regular models such as neural networks, 2λ1 is smaller than the number of parameters and m1 ≥ 1.