Adaptive Verifiable Coded Computing: Towards Fast, Secure and Private Distributed Machine Learning
Adaptive Verifiable Coded Computing: Towards Fast, Secure and Private Distributed Machine Learning
复制标题
DOI：
10.1109/ipdps53621.2022.00067
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Ting-long Tang;Ramy E. Ali;H. Hashemi;Tynan Gangwani;A. Avestimehr;M. Annavaram
中科院分区：
文献类型：
--
作者：
Ting-long Tang;Ramy E. Ali;H. Hashemi;Tynan Gangwani;A. Avestimehr;M. Annavaram
落伍者、拜占庭工人和数据隐私是分布式云计算的主要瓶颈。之前的一些工作提出了编码计算策略来共同解决这三个挑战。它们需要大量的工人，显著的通信成本或显著的计算复杂性来容忍拜占庭工人。以前的方案中的大部分开销来自于这样一个事实，即它们将所有三个问题的编码紧密耦合到一个框架中。在本文中，我们提出了自适应可验证编码计算（AVCC）的框架，从落伍者的容忍度的拜占庭节点检测的挑战。AVCC利用编码计算来处理落伍者和隐私，然后使用正交方法来利用可验证计算来减轻拜占庭工人。此外，AVCC动态地调整其编码方案，以权衡掉队容忍与拜占庭保护。我们评估AVCC的计算密集型分布式逻辑回归应用程序。实验结果表明，AVCC比传统的拉格朗日编码计算方法（LCC）提高了4.2倍的加速比和5.1%的计算精度. AVCC还将分布式逻辑回归的传统未编码实现速度提高了7.6倍，并将测试精度提高了12.1%。
Stragglers, Byzantine workers, and data privacy are the main bottlenecks in distributed cloud computing. Some prior works proposed coded computing strategies to jointly address all three challenges. They require either a large number of workers, a significant communication cost or a significant computational complexity to tolerate Byzantine workers. Much of the overhead in prior schemes comes from the fact that they tightly couple coding for all three problems into a single framework. In this paper, we propose Adaptive Verifiable Coded Computing (AVCC) framework that decouples the Byzantine node detection challenge from the straggler tolerance. AVCC leverages coded computing just for handling stragglers and privacy, and then uses an orthogonal approach that leverages verifiable computing to mitigate Byzantine workers. Furthermore, AVCC dynamically adapts its coding scheme to trade-off straggler tolerance with Byzantine protection. We evaluate AVCC on a compute-intensive distributed logistic regression application. Our experiments show that AVCC achieves up to 4.2× speedup and up to 5.1% accuracy improvement over the state-of-the-art Lagrange coded computing approach (LCC). AVCC also speeds up the conventional uncoded implementation of distributed logistic regression by up to 7.6×, and improves the test accuracy by up to 12.1%.