Exploring constructive cascade networks
Exploring constructive cascade networks
复制标题
探索建设性级联网络
DOI：
10.1109/72.809079
复制
发表时间：
1999
影响因子：
--
通讯作者：
Tom Gedeon
中科院分区：
文献类型：
--
作者：
N. Treadgold;Tom Gedeon
构造性算法已被证明是训练前向神经网络的有效方法。这些算法的一个重要性质是泛化。对构造级联算法中正则化对泛化的影响进行了一系列的实证研究。研究发现，早期停止和正则化的结合比单独使用早期停止有更好的泛化效果。三次罚项大大地惩罚了大权重，这对级联网络中的泛化是有利的。介绍了一种在构造算法中设置正则化幅度的自适应方法，并证明了该方法产生的泛化结果类似于固定的、用户优化的正则化设置所获得的结果。这种自适应的方法还导致了为更复杂的问题构建更小的网络。ACASPER算法融合了从实证研究中获得的见解，具有良好的泛化和网络结构特性。在Proben 1和其他回归数据集上，将该算法与级联相关算法进行了比较。
Constructive algorithms have proved to be powerful methods for training feedforward neural networks. An important property of these algorithms is generalization. A series of empirical studies were performed to examine the effect of regularization on generalization in constructive cascade algorithms. It was found that the combination of early stopping and regularization resulted in better generalization than the use of early stopping alone. A cubic penalty term that greatly penalizes large weights was shown to be beneficial for generalization in cascade networks. An adaptive method of setting the regularization magnitude in constructive algorithms was introduced and shown to produce generalization results similar to those obtained with a fixed, user-optimized regularization setting. This adaptive method also resulted in the construction of smaller networks for more complex problems. The acasper algorithm, which incorporates the insights obtained from the empirical studies, was shown to have good generalization and network construction properties. This algorithm was compared to the cascade correlation algorithm on the Proben 1 and additional regression data sets.