On computing the hyperparameter of extreme learning machines: Algorithm and application to computational PDEs, and comparison with classical and high-order finite elements
On computing the hyperparameter of extreme learning machines: Algorithm and application to computational PDEs, and comparison with classical and high-order finite elements
复制标题
关于计算极限学习机的超参数：计算偏微分方程的算法和应用，以及与经典和高阶有限元的比较
DOI：
10.1016/j.jcp.2022.111290
复制
发表时间：
2022
影响因子：
4.1
通讯作者：
Yang, Jielin
中科院分区：
文献类型：
--
作者：
Dong, Suchuan;Yang, Jielin
我们考虑使用极端学习机（ELM）计算偏微分方程（PDE）。在ELM中，神经网络中的隐藏层系数被分配给在[− R m，R m]上生成的随机值并固定，其中R m是用户提供的常数，输出层系数通过线性或非线性最小二乘计算进行训练。我们提出了一种基于差分进化算法的计算最优或近优值的R m的方法。所提出的方法使我们能够照亮的两种类型的ELM配置的最佳R m的特性：（i）单Rm-ELM，对应于传统的ELM方法，其中一个单一的R m是用于产生的随机系数在所有的隐藏层，和（ii）多Rm-ELM，对应于一个修改的ELM方法，其中涉及多个R m常数，每个用于产生不同的隐藏层的随机系数。我们采用这种方法的最佳R m，并将其他改进到ELM的实现。特别是，在这里，我们计算所有的微分算子涉及的输出字段的最后一个隐藏层的前向模式的自动微分，而不是反向模式的自动微分在以前的工作。这些改进显著减少了网络训练时间，提高了ELM性能。我们系统地比较了当前改进的ELM的计算性能与有限元方法（FEM），无论是经典的二阶有限元和高阶有限元与高次的拉格朗日元素，解决了一些线性和非线性偏微分方程。结果表明，改进的ELM远优于经典有限元法。对于较小的问题规模，其计算性能与高阶有限元相当，对于较大的问题规模，ELM明显优于高阶有限元。
We consider the use of extreme learning machines (ELM) for computational partial differential equations (PDE). In ELM the hidden-layer coefficients in the neural network are assigned to random values generated on [− R m, R m] and fixed, where R m is a user-provided constant, and the output-layer coefficients are trained by a linear or nonlinear least squares computation. We present a method for computing the optimal or near-optimal value of R m based on the differential evolution algorithm. The presented method enables us to illuminate the characteristics of the optimal R m for two types of ELM configurations:(i) Single-Rm-ELM, corresponding to the conventional ELM method in which a single R m is used for generating the random coefficients in all the hidden layers, and (ii) Multi-Rm-ELM, corresponding to a modified ELM method in which multiple R m constants are involved with each used for generating the random coefficients of a different hidden layer. We adopt the optimal R m from this method and also incorporate other improvements into the ELM implementation. In particular, here we compute all the differential operators involving the output fields of the last hidden layer by a forward-mode auto-differentiation, as opposed to the reverse-mode auto-differentiation in a previous work. These improvements significantly reduce the network training time and enhance the ELM performance. We systematically compare the computational performance of the current improved ELM with that of the finite element method (FEM), both the classical second-order FEM and the high-order FEM with Lagrange elements of higher degrees, for solving a number of linear and nonlinear PDEs. It is shown that the current improved ELM far outperforms the classical FEM. Its computational performance is comparable to that of the high-order FEM for smaller problem sizes, and for larger problem sizes the ELM markedly outperforms the high-order FEM.