ReLU Networks Are Universal Approximators via Piecewise Linear or Constant Functions
ReLU Networks Are Universal Approximators via Piecewise Linear or Constant Functions
复制标题
ReLU 网络是通过分段线性或常数函数实现的通用逼近器
DOI：
10.1162/neco_a_01316
复制
发表时间：
2020
影响因子：
2.9
通讯作者：
Changcun Huang
中科院分区：
文献类型：
--
作者：
Changcun Huang
这封信证明了一个ReLU网络能够通过分段线性或常数逼近的方式以任意精度逼近任何连续函数。对于单变量函数$f(x)$，我们使用ReLU的组合来生成一条线段；所有线段的子网络构成一个ReLU网络，它是$f(x)$的一个分段线性逼近。对于多变量函数$f(x)$，构建ReLU网络来逼近由逼近$f(x)$的三角剖分方法导出的一个分段线性函数。通过一个ReLU网络设计了一种称为TRLU的神经单元；通过TRLU对ReLU网络的线性输出进行整流来实现分段常数逼近，例如哈尔小波。还给出了对深层的新解释以及一些其他结果。
This letter proves that a ReLU network can approximate any continuous function with arbitrary precision by means of piecewise linear or constant approximations. For univariate function f(x), we use the composite of ReLUs to produce a line segment; all of the subnetworks of line segments comprise a ReLU network, which is a piecewise linear approximation to f(x). For multivariate function f(x), ReLU networks are constructed to approximate a piecewise linear function derived from triangulation methods approximating f(x). A neural unit called TRLU is designed by a ReLU network; the piecewise constant approximation, such as Haar wavelets, is implemented by rectifying the linear output of a ReLU network via TRLUs. New interpretations of deep layers, as well as some other results, are also presented.