Sovable Models of Finite Multi-resolution analysis

有限多分辨率分析的可解模型

• 批准号: 09680362
• 负责人: WATANABE Sumio
• 金额: $ 1.34万
• 依托单位: Tokyo Institute of Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 1997
• 资助国家: 日本
• 起止时间: 1997 至 1999
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-09680362/
• 关键词: Multi-Resolution Analysis; Neural networks; Learning Theory; Function approximation; Statistical Estimation; b-function; Singularities; Hironaka's resolution; ウェーブレット; ニューラルネットワーク; 代数解析; 自由エネルギー; 確率的複雑さ; 可解モデル; 汎関数測度

Learning models with finite multi-resolution functions, for example, neural networks, gaussian mixtures, and finite wavelets, are often used in pattern recognition, robotic control, and time sequence prediction. However, their mathematical foundation has not been established because they are not linear or not regular models. In this research, we clarified the two aspects of their mathematical properties, (1) function approximation abilities, and (2) statistical estimation effciencies.(1) It is well known that function approximation errors by their models depend on the functional topologies. In this research, we proposed a method to clarify the function approximation errors based on the assumption that the target functions are randomly taken from the function probability measures. Based on this assumption, we proved that the average function approximation errors are determined by the covariance of the coeficients of the functions, or the sparseness of the functions. This result shows a critrion whether the multi-resolution analysis is useful or not.(2) In order to clarify the statistical estimation errors, we have shown that the stochastic complexity of the learning model is determined by the deepest singularities of the model, and we developed an algorithm to calculate the learning efficiency based on the Sato-Bernstein's b-function and Hironaka's resolution of singularities.The problems for the future are to depelop a method to estimate the functional probability measure of the images and sounds and to establish the mathematical foundation of the maximum likelihood method.

Learning models with finite multi-resolution functions, for example, neural networks, gaussian mixtures, and finite wavelets, are often used in pattern recognition, robotic control, and time sequence prediction. However, their mathematical foundation has not been established because they are not linear or not regular models. In this research, we clarified the two aspects of their mathematical properties, (1) function approximation abilities, and (2) statistical estimation effciencies.(1) It is well known that function approximation errors by their models depend on the functional topologies. In this research, we proposed a method to clarify the function approximation errors based on the assumption that the target functions are randomly taken from the function probability measures. Based on this assumption, we proved that the average function approximation errors are determined by the covariance of the coeficients of the functions, or the sparseness of the functions. This result shows a critrion whether the multi-resolution analysis is useful or not.(2) In order to clarify the statistical estimation errors, we have shown that the stochastic complexity of the learning model is determined by the deepest singularities of the model, and we developed an algorithm to calculate the learning efficiency based on the Sato-Bernstein's b-function and Hironaka's resolution of singularities.The problems for the future are to depelop a method to estimate the functional probability measure of the images and sounds and to establish the mathematical foundation of the maximum likelihood method.

项目成果

Sumio Watanabe: "Algebraic analysis for singular statistical estimation"Lecture Notes on Computer Science. 1720. 39-50 (1999)

Sumio Watanabe：“奇异统计估计的代数分析”计算机科学讲义。

渡邊澄夫: "Inequalities of Generalization Errors for Layered Neural Networks in Bayesian Learning" Proceedings of Fifth International Conference on Neural Information Processing. 1巻. 59-62 (1998)

Sumio Watanabe：“贝叶斯学习中分层神经网络的泛化误差不等式”第五届国际神经信息处理会议论文集第 1 卷 59-62（1998 年）。

S. Watanabe: "Algebraic analysis for neural network learning"Proc. of IEEE Systems, Man Cybernetics. (1999)

S. Watanabe：“神经网络学习的代数分析”Proc。

Sumio Watanabe: "Algebraic analysis for non-regular learning machines"Advances in Neural Information Processing Systems. 12巻. 356-362 (2000)

Sumio Watanabe：“非常规学习机的代数分析”神经信息处理系统的进展，第 12 卷，356-362 (2000)。

渡邊澄夫: "代数解析に基づく特異点を持つモデルの学習理論" 電子情報通信学会技術報告. NC98-64. 73-80 (1998)

Sumio Watanabe：“基于代数分析的奇点模型学习理论”IEICE 技术报告 73-80 (1998)。