A Fault-Tolerant Distributed Framework for Asynchronous Iterative Computations
A Fault-Tolerant Distributed Framework for Asynchronous Iterative Computations
复制标题
DOI：
10.1109/tpds.2021.3059420
复制
发表时间：
2021-08
影响因子：
5.3
通讯作者：
Tian Zhou;Lixin Gao;X. Guan
中科院分区：
文献类型：
--
作者：
Tian Zhou;Lixin Gao;X. Guan
异步迭代计算（AIC）在机器学习和数据挖掘系统中很常见。然而，异步处理中缺乏同步障碍给连续处理带来了挑战，工作者可能会失败。不存在所有工作线程都可以回滚到的全局同步点。在本文中，我们提出了异步迭代计算（FAIC）的容错框架。我们的框架在不停止任何worker的计算的情况下对AIC系统进行虚拟快照。证明了fac捕获虚拟快照可以正确地恢复AIC系统。我们在两个现有的AIC系统Maiter和NOMAD上评估了我们的fac框架。实验结果表明，fac的检查点开销比同步检查点方法减少了50%以上。FAIC比其他异步快照算法（如Chandy-Lamport算法）快10%左右。我们在大型集群上的实验表明，fac随工人数量的增加而扩大。
Asynchronous iterative computations (AIC) are common in machine learning and data mining systems. However, the lack of synchronization barriers in asynchronous processing brings challenges for continuous processing while workers might fail. There is no global synchronization point that all workers can roll back to. In this article, we propose a fault-tolerant framework for asynchronous iterative computations (FAIC). Our framework takes a virtual snapshot of the AIC system without halting the computation of any worker. We prove that the virtual snapshot capture by FAIC can recover the AIC system correctly. We evaluate our FAIC framework on two existing AIC systems, Maiter and NOMAD. Our experiment result shows that the checkpoint overhead of FAIC is more than 50 percent shorter than the synchronous checkpoint method. FAIC is around 10 percent faster than other asynchronous snapshot algorithms, such as the Chandy-Lamport algorithm. Our experiments on a large cluster demonstrate that FAIC scales with the number of workers.