Distributed adaptive Huber regression
Distributed adaptive Huber regression
复制标题
DOI：
10.1016/j.csda.2021.107419
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Jiyun Luo;Qiang Sun;Wen-Xin Zhou
中科院分区：
文献类型：
--
作者：
Jiyun Luo;Qiang Sun;Wen-Xin Zhou
分布式数据自然会出现在涉及多个观察源的场景中，每个观察源都存储在不同的位置。由于带宽和存储有限，或者由于隐私协议，直接将所有数据集中在一起通常是被禁止的。针对数据存在重尾和/或非对称误差且具有有限二阶矩的情况，提出了一种新的稳健的分布式线性回归拟合算法。该算法在每次迭代中只传递梯度信息，因此通信效率很高。要实现偏差-稳健性的折衷，关键是一种新的双鲁棒性方法，该方法同时适用于局部和全局目标函数。从统计学上讲，所得到的估计器达到了集中的非渐近误差界，就像所有的数据被汇集在一起并且来自具有亚高斯尾部的分布一样。在有限的(2+δ)阶矩条件下，建立了分布估计的Berry-Esseen界，并在此基础上构造了稳健的可信区间。在高维中，提出的双粗化损失函数与ℓ-1惩罚相补充，用于拟合稀疏线性模型与分布数据。数值研究进一步证实，与现有的分布式方法相比，该方法以较小的变异性获得了接近最优的精度，以较小的置信度宽度获得了更好的覆盖范围。
Distributed data naturally arise in scenarios involving multiple sources of observations, each stored at a different location. Directly pooling all the data together is often prohibited due to limited bandwidth and storage, or due to privacy protocols. A new robust distributed algorithm is introduced for fitting linear regressions when data are subject to heavy-tailed and/or asymmetric errors with finite second moments. The algorithm only communicates gradient information at each iteration, and therefore is communication-efficient. To achieve the bias-robustness tradeoff, the key is a novel double-robustification approach that applies on both the local and global objective functions. Statistically, the resulting estimator achieves the centralized nonasymptotic error bound as if all the data were pooled together and came from a distribution with sub-Gaussian tails. Under a finite (2+ δ)-th moment condition, a Berry-Esseen bound for the distributed estimator is established, based on which robust confidence intervals are constructed. In high dimensions, the proposed doubly-robustified loss function is complemented with ℓ 1-penalization for fitting sparse linear models with distributed data. Numerical studies further confirm that compared with extant distributed methods, the proposed methods achieve near-optimal accuracy with low variability and better coverage with tighter confidence width.