Gradient Coding with Dynamic Clustering for Straggler Mitigation
Gradient Coding with Dynamic Clustering for Straggler Mitigation
复制标题
DOI：
10.1109/icc42927.2021.9500346
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Baturalp Buyukates;Emre Ozfatura;S. Ulukus;Deniz Gündüz
中科院分区：
文献类型：
--
作者：
Baturalp Buyukates;Emre Ozfatura;S. Ulukus;Deniz Gündüz
在分布式同步梯度下降（GD）中，每次迭代完成时间的主要性能瓶颈是最慢的离散工人。为了加快GD迭代中存在的掉队者，编码分布式计算技术的实现分配冗余计算的工人。在本文中，我们提出了一种新的梯度编码（GC）计划，利用动态聚类，表示为GC-DC，以加快梯度计算。在时间相关的离散行为下，GC-DC的目标是根据前一次迭代中的离散行为来调节每个集群中离散工人的数量。我们数值计算表明，GC-DC提供了显着的改善，在平均完成时间（每次迭代）的通信负载没有增加相比，原来的GC计划。
In distributed synchronous gradient descent (GD) the main performance bottleneck for the per-iteration completion time is the slowest straggling workers. To speed up GD iterations in the presence of stragglers, coded distributed computation techniques are implemented by assigning redundant computations to workers. In this paper, we propose a novel gradient coding (GC) scheme that utilizes dynamic clustering, denoted by GC-DC, to speed up gradient calculations. Under time-correlated straggling behavior, GC-DC aims at regulating the number of straggling workers in each cluster based on the straggler behavior in the previous iteration. We numerically show that GC-DC provides significant improvements in the average completion time (of each iteration) with no increase in the communication load compared to the original GC scheme.