Resilient and Communication Efficient Learning for Heterogeneous Federated Systems
Resilient and Communication Efficient Learning for Heterogeneous Federated Systems
复制标题
DOI：
--
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Zhuangdi Zhu;Junyuan Hong;S. Drew;Jiayu Zhou
中科院分区：
文献类型：
--
作者：
Zhuangdi Zhu;Junyuan Hong;S. Drew;Jiayu Zhou
联邦学习(FL)的兴起通过利用分散在边缘设备上的数据将机器学习带入了边缘计算。然而，边缘网络拓扑的异构性和无线传输的不确定性是FL在边缘计算中广泛应用的两大障碍，导致收敛时间过快和通信成本过高。在这项工作中，我们提出了一种同时应对这两个挑战的FL方案。具体地说，我们使边缘设备能够学习自我提取的神经网络，这些神经网络很容易修剪到任意大小，以嵌套和渐进的方式捕获学习领域的知识。我们的方法不仅通过为具有不同模型体系结构的边缘设备提供服务来解决系统异构性，而且通过允许在故障网络连接下传输部分模型参数而不浪费传输参数的贡献知识来缓解连接不确定性问题。大量的实证研究表明，在系统异构性和网络不稳定的情况下，我们的方法表现出了显著的弹性和更高的通信效率。
The rise of Federated Learning (FL) is bringing machine learning to edge computing by utilizing data scattered across edge devices. However, the heterogeneity of edge network topologies and the uncertainty of wireless transmission are two major obstructions of FL's wide application in edge computing, leading to prohibitive convergence time and high communication cost. In this work, we propose an FL scheme to address both challenges simultaneously. Specifically, we enable edge devices to learn self-distilled neural networks that are readily prunable to arbitrary sizes, which capture the knowledge of the learning domain in a nested and progressive manner. Not only does our approach tackle system heterogeneity by serving edge devices with varying model architectures, but it also alleviates the issue of connection uncertainty by allowing transmitting part of the model parameters under faulty network connections, without wasting the contributing knowledge of the transmitted parameters. Extensive empirical studies show that under system heterogeneity and network instability, our approach demonstrates significant resilience and higher communication efficiency compared to the state-of-the-art.