Network Adaptive Federated Learning: Congestion and Lossy Compression
Network Adaptive Federated Learning: Congestion and Lossy Compression
复制标题
DOI：
10.1109/infocom53939.2023.10228885
复制
发表时间：
2023-01
期刊：
影响因子：
--
通讯作者：
Parikshit Hegde;G. Veciana;Aryan Mokhtari
中科院分区：
文献类型：
--
作者：
Parikshit Hegde;G. Veciana;Aryan Mokhtari
为了实现分布式数据的隐私和学习的双重目标，联邦学习（FL）系统依赖于一组客户端和服务器之间频繁交换大文件（模型更新）。因此，FL系统暴露于跨广泛的网络资源集合的拥塞，或者实际上是跨广泛的网络资源集合的拥塞的原因。有损压缩可用于减少交换文件的大小和相关延迟，但代价是增加模型更新的噪声。通过明智地调整客户端的压缩以适应不同的网络拥塞，FL应用程序可以减少挂钟训练时间。为此，我们提出了一个网络自适应压缩（NAC-FL）的政策，它动态地改变客户端的有损压缩选择网络拥塞的变化。在适当的假设下，我们证明了NAC-FL在直接最小化预期挂钟训练时间方面是渐近最优的。此外，我们通过模拟表明，NAC-FL实现了鲁棒的性能改进，在设置中具有更高的增益与时间上的正相关延迟。
In order to achieve the dual goals of privacy and learning across distributed data, Federated Learning (FL) systems rely on frequent exchanges of large files (model updates) between a set of clients and the server. As such FL systems are exposed to, or indeed the cause of, congestion across a wide set of network resources. Lossy compression can be used to reduce the size of exchanged files and associated delays, at the cost of adding noise to model updates. By judiciously adapting clients’ compression to varying network congestion, an FL application can reduce wall clock training time. To that end, we propose a Network Adaptive Compression (NAC-FL) policy, which dynamically varies the client’s lossy compression choices to network congestion variations. We prove, under appropriate assumptions, that NAC-FL is asymptotically optimal in terms of directly minimizing the expected wall clock training time. Further, we show via simulation that NAC-FL achieves robust performance improvements with higher gains in settings with positively correlated delays across time.