Self-adaptive deep neural network: Numerical approximation to functions and PDEs
Self-adaptive deep neural network: Numerical approximation to functions and PDEs
复制标题
自适应深度神经网络：函数和偏微分方程的数值逼近
DOI：
10.1016/j.jcp.2022.111021
复制
发表时间：
2022
影响因子：
4.1
通讯作者：
Liu, Min
中科院分区：
文献类型：
--
作者：
Cai, Zhiqiang;Chen, Jingshuang;Liu, Min
在许多机器学习应用中，为给定任务设计最佳深度神经网络是重要且具有挑战性的。为了解决这个问题，我们引入了一种自适应算法：自适应网络增强（ANE）方法，写为训练→估计→增强形式的循环。从一个小的两层神经网络（NN）开始，步骤训练是解决当前NN的优化问题;步骤估计是使用当前NN的解决方案计算后验估计器/指标;步骤增强是向当前NN添加新的神经元。新的网络增强策略的基础上计算的估计/指标，本文开发，以确定有多少新的神经元，当一个新的层应添加到当前NN。ANE方法提供了一个在训练当前NN时获得良好初始化的自然过程;此外，我们还介绍了一个关于如何初始化新添加的神经元以获得更好近似的高级过程。我们证明，ANE方法可以自动设计一个几乎最小的神经网络学习功能表现出尖锐的过渡层，以及双曲型偏微分方程的不连续的解决方案。
Designing an optimal deep neural network for a given task is important and challenging in many machine learning applications. To address this issue, we introduce a self-adaptive algorithm: the adaptive network enhancement (ANE) method, written as loops of the form train→ estimate→ enhance. Starting with a small two-layer neural network (NN), the step train is to solve the optimization problem at the current NN; the step estimate is to compute a posteriori estimator/indicators using the solution at the current NN; the step enhance is to add new neurons to the current NN. Novel network enhancement strategies based on the computed estimator/indicators are developed in this paper to determine how many new neurons and when a new layer should be added to the current NN. The ANE method provides a natural process for obtaining a good initialization in training the current NN; in addition, we introduce an advanced procedure on how to initialize newly added neurons for a better approximation. We demonstrate that the ANE method can automatically design a nearly minimal NN for learning functions exhibiting sharp transitional layers as well as discontinuous solutions of hyperbolic partial differential equations.
影响因子：
4.1
作者：
Raissi, M.;Perdikaris, P.;Karniadakis, G. E.
通讯作者：
Karniadakis, G. E.