A theoretical study of the relationship between an ELM network and its subnetworks
A theoretical study of the relationship between an ELM network and its subnetworks
复制标题
DOI：
10.1109/ijcnn.2017.7966068
复制
发表时间：
2017-05
期刊：
影响因子：
--
通讯作者：
E. Tu;Guanghao Zhang;L. Rachmawati;E. Rajabally;Shangbo Mao;G. Huang
中科院分区：
文献类型：
--
作者：
E. Tu;Guanghao Zhang;L. Rachmawati;E. Rajabally;Shangbo Mao;G. Huang
生物神经网络由许多具有不同功能的子网络和模块构成。对于人工神经网络，网络与其子网络之间的关系对于理论和算法研究也是非常重要和有用的，即可以利用它来开发增量式网络训练算法或并行网络训练算法。在本文中，我们探讨了极端学习机（ELM）训练的神经网络和它的子网络之间的关系。据我们所知，我们是第一个证明一个定理，表明ELM训练的神经网络可以分散成子网络，其最优解可以递归地构造这些子网络的最优解。在此基础上，我们还提出了两种有效训练大型ELM神经网络的算法：一种是并行网络训练算法，另一种是增量网络训练算法。实验结果证明了定理的有效性和算法的有效性。
A biological neural network is constituted by numerous subnetworks and modules with different functionalities. For an artificial neural network, the relationship between a network and its subnetworks is also important and useful for both theoretical and algorithmic research, i.e. it can be exploited to develop incremental network training algorithm or parallel network training algorithm. In this paper we explore the relationship between an Extreme Learning Machine (ELM) trained neural network and its subnetworks. To the best of our knowledge, we are the first to prove a theorem that shows an ELM trained neural network can be scattered into subnetworks and its optimal solution can be constructed recursively by the optimal solutions of these subnetworks. Based on the theorem we also present two algorithms to train a large ELM neural network efficiently: one is a parallel network training algorithm and the other is an incremental network training algorithm. The experimental results demonstrate the usefulness of the theorem and the validity of the developed algorithms.