Communication-Efficient Accurate Statistical Estimation
Communication-Efficient Accurate Statistical Estimation
复制标题
DOI：
10.1080/01621459.2021.1969238
复制
发表时间：
2019-06
影响因子：
3.7
通讯作者：
Jianqing Fan;Yongyi Guo;Kaizheng Wang
中科院分区：
文献类型：
--
作者：
Jianqing Fan;Yongyi Guo;Kaizheng Wang
摘要当数据以分布式方式存储时，由于通信成本和隐私问题，传统统计推断程序的直接应用往往是禁止的。本文开发和研究了两个通信高效的精确统计估计（CEASE），通过迭代算法实现分布式优化。在每次迭代中，节点机并行执行计算并与中央处理器通信，然后中央处理器将聚合信息广播到节点机以进行新的更新。算法适应节点机上损失函数之间的相似性，当每个节点机具有足够大的样本量时，算法收敛速度较快。此外，他们不需要良好的初始化，并享有线性收敛保证在一般情况下。给出了优化误差的收缩率，揭示了优化误差对局部样本容量的依赖性。此外，改进的统计精度每次迭代推导。通过将所提出的方法视为多步统计估计，我们表明，在典型的统计应用中，可以在有限的步骤中实现统计效率。此外，我们给出了一步CEASE估计是统计有效的条件。大量的数值实验的合成和真实的数据验证了理论结果，并证明了我们的算法的上级性能。
Abstract When the data are stored in a distributed manner, direct applications of traditional statistical inference procedures are often prohibitive due to communication costs and privacy concerns. This article develops and investigates two communication-efficient accurate statistical estimators (CEASE), implemented through iterative algorithms for distributed optimization. In each iteration, node machines carry out computation in parallel and communicate with the central processor, which then broadcasts aggregated information to node machines for new updates. The algorithms adapt to the similarity among loss functions on node machines, and converge rapidly when each node machine has large enough sample size. Moreover, they do not require good initialization and enjoy linear converge guarantees under general conditions. The contraction rate of optimization errors is presented explicitly, with dependence on the local sample size unveiled. In addition, the improved statistical accuracy per iteration is derived. By regarding the proposed method as a multistep statistical estimator, we show that statistical efficiency can be achieved in finite steps in typical statistical applications. In addition, we give the conditions under which the one-step CEASE estimator is statistically efficient. Extensive numerical experiments on both synthetic and real data validate the theoretical results and demonstrate the superior performance of our algorithms.