Sparse Reduced Rank Huber Regression in High Dimensions
Sparse Reduced Rank Huber Regression in High Dimensions
复制标题
DOI：
10.1080/01621459.2022.2050243
复制
发表时间：
2022-03
影响因子：
3.7
通讯作者：
Kean Ming Tan;Qiang Sun;D. Witten
中科院分区：
文献类型：
--
作者：
Kean Ming Tan;Qiang Sun;D. Witten
摘要提出了一种稀疏降秩Huber回归方法，用于分析带有重尾随机噪声的大型复杂高维数据。该方法是基于一个凸松弛的秩和稀疏约束的非凸优化问题，然后使用块坐标下降和交替方向法的乘子算法来解决。我们建立非渐近估计误差界下的Frobenius和核规范在高维设置。这是对降秩回归现有结果的重大贡献，降秩回归主要集中在秩选择和预测一致性上。我们的理论结果量化的随机噪声和统计偏差的重尾之间的权衡。对于具有有界（1+δ）阶矩且δ∈（0，1）的随机噪声，收敛速度是δ的函数，并且比亚高斯型偏差界慢;对于具有有界二阶矩的随机噪声，我们得到了与假设亚高斯噪声一样的收敛速度.我们通过大量的数值研究和数据应用说明了所提出的方法的性能。本文的补充材料可在网上查阅。
Abstract We propose a sparse reduced rank Huber regression for analyzing large and complex high-dimensional data with heavy-tailed random noise. The proposed method is based on a convex relaxation of a rank- and sparsity-constrained nonconvex optimization problem, which is then solved using a block coordinate descent and an alternating direction method of multipliers algorithm. We establish nonasymptotic estimation error bounds under both Frobenius and nuclear norms in the high-dimensional setting. This is a major contribution over existing results in reduced rank regression, which mainly focus on rank selection and prediction consistency. Our theoretical results quantify the tradeoff between heavy-tailedness of the random noise and statistical bias. For random noise with bounded (1+δ)th moment with δ∈(0,1), the rate of convergence is a function of δ, and is slower than the sub-Gaussian-type deviation bounds; for random noise with bounded second moment, we obtain a rate of convergence as if sub-Gaussian noise were assumed. We illustrate the performance of the proposed method via extensive numerical studies and a data application. Supplementary materials for this article are available online.