Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions.
Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions.
复制标题
DOI：
10.1111/rssb.12166
复制
发表时间：
2017-01
期刊：
影响因子：
--
通讯作者：
Wang Y
中科院分区：
文献类型：
--
作者：
Fan J;Li Q;Wang Y
在各种科学领域中，经常会遇到含有重尾误差的数据。为了解决这一问题，近年来发展了基于分位数回归和最小绝对偏差(LAD)回归的方法。这些方法实质上是估计条件中位数(或分位数)函数。它们可能与条件平均函数有很大不同，特别是当分布是不对称和异方差的时候。在只存在二阶矩的情况下，如何有效地估计超高维环境下的平均回归函数？为了解决这一问题，我们提出了一种带有发散参数的惩罚性Huber损失，以减少传统Huber损失带来的偏差。这样的惩罚稳健近似二次(RA-二次)损失将被称为RA-Lasso。在维度可以随样本大小呈指数增长的超高维环境下，我们的结果表明RA-Lasso估计器在轻尾情况下以与最优估计相同的速度产生一致的估计量。我们进一步研究了RA-Lasso算法的计算收敛问题，证明了复合梯度下降算法在充分迭代后确实能得到一个具有相同最优解的解。作为副产品，我们还建立了仅存在二阶矩时估计总体均值的集中度不等式。我们将RA-Lasso与其他基于分位数回归和LAD回归的正则化稳健估计进行了比较。大量的仿真研究表明，RA-Lasso具有令人满意的有限样本性能。
Data subject to heavy-tailed errors are commonly encountered in various scientific fields. To address this problem, procedures based on quantile regression and Least Absolute Deviation (LAD) regression have been developed in recent years. These methods essentially estimate the conditional median (or quantile) function. They can be very different from the conditional mean functions, especially when distributions are asymmetric and heteroscedastic. How can we efficiently estimate the mean regression functions in ultra-high dimensional setting with existence of only the second moment? To solve this problem, we propose a penalized Huber loss with diverging parameter to reduce biases created by the traditional Huber loss. Such a penalized robust approximate quadratic (RA-quadratic) loss will be called RA-Lasso. In the ultra-high dimensional setting, where the dimensionality can grow exponentially with the sample size, our results reveal that the RA-lasso estimator produces a consistent estimator at the same rate as the optimal rate under the light-tail situation. We further study the computational convergence of RA-Lasso and show that the composite gradient descent algorithm indeed produces a solution that admits the same optimal rate after sufficient iterations. As a byproduct, we also establish the concentration inequality for estimating population mean when there exists only the second moment. We compare RA-Lasso with other regularized robust estimators based on quantile regression and LAD regression. Extensive simulation studies demonstrate the satisfactory finite-sample performance of RA-Lasso.