Geometrical Singularities in the Neuromanifold of Multilayer Perceptrons
Geometrical Singularities in the Neuromanifold of Multilayer Perceptrons
复制标题
多层感知器神经流形中的几何奇点
DOI：
10.7551/mitpress/1120.003.0049
复制
发表时间：
2001
期刊：
影响因子：
--
通讯作者：
T. Ozeki
中科院分区：
文献类型：
--
作者：
S. Amari;Hyeyoung Park;T. Ozeki
奇点在分层模型（例如多层感知器）的参数空间中普遍存在。在奇点处，Fisher 信息矩阵退化，Cramer-Rao 范式不再成立，这意味着 AIC 和 MDL 等经典模型选择理论无法应用。研究奇点处的泛化误差和训练误差之间的关系非常重要。本文演示了一种通过使用简单模型来分析最大似然估计量和高斯随机场的贝叶斯预测分布的这些误差的方法。
Singularities are ubiquitous in the parameter space of hierarchical models such as multilayer perceptrons. At singularities, the Fisher information matrix degenerates, and the Cramer-Rao paradigm does no more hold, implying that the classical model selection theory such as AIC and MDL cannot be applied. It is important to study the relation between the generalization error and the training error at singularities. The present paper demonstrates a method of analyzing these errors both for the maximum likelihood estimator and the Bayesian predictive distribution in terms of Gaussian random fields, by using simple models.