High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
复制标题
DOI：
10.1287/opre.2021.2217
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Hongcheng Liu;Y. Ye;H. Lee
中科院分区：
文献类型：
--
作者：
Hongcheng Liu;Y. Ye;H. Lee
在《近似稀疏性下的高维学习及其在非光滑估计和正则化神经网络中的应用》一书中，Liu、Ye和Lee研究了一个数据比问题维度少得多的模型拟合问题。他们特别关注的情况是，通常强加的稀疏性假设被放松，而限制的强凸性的通常条件不存在。结果表明，即使问题维度呈指数增长，在这种情况下仍能保证泛化性能。进一步研究了高维非光滑估计和神经网络的样本复杂性。特别是对于后者，证明了在显式正则化的情况下，即使样本大小只是拟合参数个数的多对数，神经网络也是可证明的泛化。
In “High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks,” Liu, Ye, and Lee study a model fitting problem where there are much fewer data than problem dimensions. Of their particular focus are the scenarios where the commonly imposed sparsity assumption is relaxed, and the usual condition of the restricted strong convexity is absent. The results show that generalization performance can still be ensured in such settings, even if the problem dimensions grow exponentially. The authors further study the sample complexities of high-dimensional nonsmooth estimation and neural networks. Particularly for the latter, it is shown that, with explicit regularization, a neural network is provably generalizable, even if the sample size is only poly-logarithmic in the number of fitting parameters.