A Tuning-free Robust and Efficient Approach to High-dimensional Regression
A Tuning-free Robust and Efficient Approach to High-dimensional Regression
复制标题
DOI：
10.1080/01621459.2020.1840989
复制
发表时间：
2020-10
影响因子：
3.7
通讯作者：
Lan Wang;Bo Peng;Jelena Bradic;Runze Li;Y. Wu
中科院分区：
文献类型：
--
作者：
Lan Wang;Bo Peng;Jelena Bradic;Runze Li;Y. Wu
摘要介绍了一种理论上有保证的高维回归新方法。新方法克服了套索参数选择的困难，并具有几个吸引人的性质。它使用一种易于模拟的调整参数，该参数可自动适应未知随机误差分布和设计矩阵的相关结构。该算法对重尾随机误差具有较大的效率增益，同时对正常随机误差保持较高的效率，具有较强的鲁棒性。与其他稳健回归方法相比，当响应变量经过尺度变换时，它还具有等变性质。在计算上，该问题可以通过线性规划有效地求解。理论上，在随机误差分布弱的条件下，我们给出了具有模拟调谐参数的新估计量的有限样本误差界。我们的结果对弥补套索及其变种的实践与理论之间的差距做出了有益的贡献。我们还证明，通过第二阶段的增强和一些光调可以实现效率的进一步提高。我们的仿真结果表明，在不同的设置下，所提出的方法往往优于交叉验证的Lasso方法。
Abstract We introduce a novel approach for high-dimensional regression with theoretical guarantees. The new procedure overcomes the challenge of tuning parameter selection of Lasso and possesses several appealing properties. It uses an easily simulated tuning parameter that automatically adapts to both the unknown random error distribution and the correlation structure of the design matrix. It is robust with substantial efficiency gain for heavy-tailed random errors while maintaining high efficiency for normal random errors. Comparing with other alternative robust regression procedures, it also enjoys the property of being equivariant when the response variable undergoes a scale transformation. Computationally, it can be efficiently solved via linear programming. Theoretically, under weak conditions on the random error distribution, we establish a finite-sample error bound with a near-oracle rate for the new estimator with the simulated tuning parameter. Our results make useful contributions to mending the gap between the practice and theory of Lasso and its variants. We also prove that further improvement in efficiency can be achieved by a second-stage enhancement with some light tuning. Our simulation results demonstrate that the proposed methods often outperform cross-validated Lasso in various settings.