Simultaneous neural network approximation for smooth functions
Simultaneous neural network approximation for smooth functions
复制标题
DOI：
10.1016/j.neunet.2022.06.040
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
S. Hon;Haizhao Yang
中科院分区：
文献类型：
--
作者：
S. Hon;Haizhao Yang
我们在这项工作中建立了深度神经网络的近似结果，用于以 Sobolev 范数测量的平滑函数，其动机是最近开发的使用深度神经网络的偏微分方程数值求解器。我们的近似结果是非渐近的，因为误差范围是根据网络的宽度和深度同时明确确定的所有涉及的常数来明确表征的。也就是说，对于 f∈ C s ([0, 1] d)，我们证明宽度为 O (N log N) 和深度为 O (L log L) 的深度 ReLU 网络可以相对于 p∈[1,∞）的 W 1, p ([0, 1] d) 范数实现 O (N− 2 (s− 1)/d L− 2 (s− 1)/d) 的非渐近逼近率。如果使用 ReLU 函数或其平方作为激活函数来构建宽度为 O (N log N) 和深度为 O (L log L) 的深度神经网络来逼近 fε C s ([0, 1] d)，则相对于 pε[1,∞] 的 W n, p ([0, 1] d) 范数，逼近率为 O (N− 2 (s− n)/d L− 2 (s− n)/d)。
We establish in this work approximation results of deep neural networks for smooth functions measured in Sobolev norms, motivated by recent development of numerical solvers for partial differential equations using deep neural networks. Our approximation results are nonasymptotic in the sense that the error bounds are explicitly characterized in terms of both the width and depth of the networks simultaneously with all involved constants explicitly determined. Namely, for f∈ C s ([0, 1] d), we show that deep ReLU networks of width O (N log N) and of depth O (L log L) can achieve a nonasymptotic approximation rate of O (N− 2 (s− 1)/d L− 2 (s− 1)/d) with respect to the W 1, p ([0, 1] d) norm for p∈[1,∞). If either the ReLU function or its square is applied as activation functions to construct deep neural networks of width O (N log N) and of depth O (L log L) to approximate f∈ C s ([0, 1] d), the approximation rate is O (N− 2 (s− n)/d L− 2 (s− n)/d) with respect to the W n, p ([0, 1] d) norm for p∈[1,∞).