Nonparametric regression on low-dimensional manifolds using deep ReLU networks
Nonparametric regression on low-dimensional manifolds using deep ReLU networks
复制标题
使用深度 ReLU 网络对低维流形进行非参数回归
DOI：
10.1093/imaiai/iaac001
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Zhao, Tuo
中科院分区：
文献类型：
--
作者：
Chen, Minshuo;Jiang, Haomin;Liao, Wenjing;Zhao, Tuo
真实世界的数据通常表现出低维的几何结构，并且可以被视为低维流形附近的样本。研究了低维流形上Hölder函数的深度纠错线性单元(RELU)网络的非参数回归问题。假设训练数据取自等距嵌入的二维黎曼流形上支持的Hölder函数。设计了一种深度RELU网络结构，以从训练数据估计潜在的功能。证明了经验估计的均方误差按…的数量级收敛。这一结果表明，深度RELU网络的收敛速度依赖于数据的本征维度，而数据本征维度通常比环境维度小得多。因此，它证明了深层REU网络对数据中低维几何结构的适应性，并部分解释了深层REU网络在处理具有低维几何结构的高维数据方面的能力。
Real-world data often exhibit low-dimensional geometric structures and can be viewed as samples near a low-dimensional manifold. This paper studies nonparametric regression of Hölder functions on low-dimensional manifolds using deep Rectified Linear Unit (ReLU) networks. Supposetraining data are sampled from a Hölder function insupported on a-dimensional Riemannian manifold isometrically embedded in. A deep ReLU network architecture is designed to estimate the underlying function from the training data. The mean squared error of the empirical estimator is proved to converge in the order of. This result shows that deep ReLU networks give rise to a fast convergence rate depending on the data intrinsic dimension, which is usually much smaller than the ambient dimension. It therefore demonstrates the adaptivity of deep ReLU networks to low-dimensional geometric structures in data and partially explains the power of deep ReLU networks in tackling high-dimensional data with low-dimensional geometric structures.