Can threshold networks be trained directly?
Can threshold networks be trained directly?
复制标题
DOI：
10.1109/tcsii.2005.857540
复制
发表时间：
2006-03-01
影响因子：
4.4
通讯作者：
Sundararajan, N
中科院分区：
文献类型：
--
作者：
Huang, GB;Zhu, QY;Sundararajan, N
具有阈值激活功能的神经网络非常需要，因为硬件实现的易用性，但是，由于阈值函数不可分割，因此流行的基于梯度的学习算法不能直接用于训练这些网络。文献中可用的方法主要集中于通过使用Sigmoid函数近似阈值激活函数。在本文中，我们从理论上讲，最近开发的极限学习机（ELM）算法可用于直接训练具有阈值功能的神经网络，而不是使用Sigmoid函数近似它们。基于现实世界基准回归问题的实验结果表明，ELM获得的概括性能比阈值网络中使用的其他算法更好。同样，ELM方法不需要控制变量（手动调谐参数），并且更快。
Neural networks with threshold activation functions are highly desirable because of the ease of hardware implementation, However, the popular gradient-based learning algorithms cannot be directly used to train these networks as the threshold functions are nondifferentiable. Methods available in the literature mainly focus on approximating the threshold activation functions by using sigmoid functions. In this paper, we show theoretically that the recently developed extreme learning machine (ELM) algorithm can be used to train the neural networks with threshold functions directly instead of approximating them with sigmoid functions. Experimental results based on real-world benchmark regression problems demonstrate that the generalization performance obtained by ELM is better than other algorithms used in threshold networks. Also, the ELM method does not need control variables (manually tuned parameters) and is much faster.