Network Complexity and Generalization Performance of Large Function Approximating Neural Networks

JI 9312504这项研究将解决前馈多层神经网络监督学习中一个重要的公开问题：如何利用网络参数相关的信息来评估网络的泛化性能。重点放在用于函数逼近的大型网络上。其目标是：1)利用网络参数相关的信息来评估泛化误差；2)将训练算法的动态特性纳入泛化误差的评估中；3)设计网络复杂性，以便在具有固定结构的大型网络的小训练集上实现更可预测的泛化。为了实现这些目标，将使用统计方法来建立泛化误差与预期网络复杂性之间的一般关系。并利用贝叶斯网络将训练算法的动态性与网络的泛化性能联系起来。本文的研究成果对组合监督强化学习中的重要公开问题，以及高速、大容量的大型光学神经网络的研究具有重要意义。*v S t app hlp@j N=Argyle BMP@j o v Cars BMP@j p v CHARMAP EXE@j q V弦波@j|a扩展EXE@j；Flock BMP@j^GLOSSARYHLP@j MPLAYER HLP@j`2 NETWORKSWRI@j Notepad EXE@j Ji 9312504这项研究将解决使用前馈多层神经网络进行监督学习中的一个重要问题$$(F//1信使符号&amp；Arial 5 Courier New“h%%*V R：\WW20USER\ABSTRACT.DOT ji摘要Alicia E.Harris Alicia E.Harris

Ji 9312504 This research will address an important open problem in supervised learning using feedforward multi-layer neural networks: how to evaluate generalization performance of the networks using network-parameter-dependent information. The focus is on large networks used for the purpose of function approximation. The objectives are: 1) to evaluate the generalization error utilizing the network-parameter-dependent information; 2) to incorporate dynamics of training algorithms into evaluation of generalization error; 3) to design network complexity in order to achieve more predictable generalization on a small training set for large networks with fixed architecture. To achieve these objectives, statistical methods will be used to develop a general relationship between the generalization error and the expected network complexity. And a Bayesian network is utilized to relate dynamics of training algorithms to the generalization performance of the networks. The impact of the proposed research is to important open problems in combined supervised and reinforcement learning, and high-speed high-capacity large optical neural networks. *** v s t APPS HLP @ j N= ARGYLE BMP @ j o v CARS BMP @ j p v CHARMAP EXE @ j q V CHORD WAV @ j | a EXPAND EXE @ j ; FLOCK BMP @ j ^ GLOSSARYHLP @ j MPLAYER HLP @ j `2 NETWORKSWRI @ j NOTEPAD EXE @ j Ji 9312504 This research will address an important open problem in supervised learning using feedforward mul ti-layer neural ne $ $ ( F / / 1 Courier Symbol & Arial 5 Courier New " h % % * V R:\WW20USER\ABSTRACT.DOT ji abstract Alicia E. Harris Alicia E. Harris