A Statistical Learning Assessment of Huber Regression
A Statistical Learning Assessment of Huber Regression
复制标题
DOI：
10.1016/j.jat.2021.105660
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Yunlong Feng;Qiang Wu
中科院分区：
文献类型：
--
作者：
Yunlong Feng;Qiang Wu
作为稳健统计的胜利和里程碑之一，Huber 回归在稳健推理和估计中发挥着重要作用。它还在机器学习中找到了各种各样的应用。在参数设置中，它已被广泛研究。然而，在通常以非参数方式学习函数的统计学习环境中，对于 Huber 回归估计器如何学习条件均值函数以及它为何在没有轻尾噪声假设的情况下起作用，仍然缺乏理论理解。为了解决这些基本问题，本文从统计学习的角度对 Huber 回归进行了评估。首先，我们表明，机器学习中通常追求的 Huber 回归估计量的通常风险一致性属性不能保证其在均值回归中的可学习性。其次，我们认为 Huber 回归应该以一种自适应的方式来实现均值回归，这意味着需要根据样本大小和噪声的矩条件来调整尺度参数。第三，通过自适应选择尺度参数，我们证明了 Huber 回归估计量可以在条件分布的 (1+ ε) 矩条件 (ε> 0) 下进行渐近均值回归校准。最后但并非最不重要的一点是，在相同时刻的条件下，我们为 Huber 回归估计器建立了几乎确定的收敛率。请注意，(1+ ε) 矩条件适应响应变量具有无限方差的特殊情况，因此所建立的收敛率证明了 Huber 回归估计器的鲁棒性特征。在上述意义上，本研究提供了 Huber 回归估计器的系统统计学习评估，并从理论角度证明了它们在鲁棒性方面的优点。
As one of the triumphs and milestones of robust statistics, Huber regression plays an important role in robust inference and estimation. It has also been finding a great variety of applications in machine learning. In a parametric setup, it has been extensively studied. However, in the statistical learning context where a function is typically learned in a nonparametric way, there is still a lack of theoretical understanding of how Huber regression estimators learn the conditional mean function and why it works in the absence of light-tailed noise assumptions. To address these fundamental questions, this paper conducts an assessment of Huber regression from a statistical learning viewpoint. First, we show that the usual risk consistency property of Huber regression estimators, which is usually pursued in machine learning, cannot guarantee their learnability in mean regression. Second, we argue that Huber regression should be implemented in an adaptive way to perform mean regression, implying that one needs to tune the scale parameter in accordance with the sample size and the moment condition of the noise. Third, with an adaptive choice of the scale parameter, we demonstrate that Huber regression estimators can be asymptotic mean regression calibrated under (1+ ε)-moment conditions (ε> 0) on the conditional distribution. Last but not least, under the same moment conditions, we establish almost sure convergence rates for Huber regression estimators. Note that the (1+ ε)-moment conditions accommodate the special case where the response variable possesses infinite variance and so the established convergence rates justify the robustness feature of Huber regression estimators. In the above senses, the present study provides a systematic statistical learning assessment of Huber regression estimators and justifies their merits in terms of robustness from a theoretical viewpoint.