SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression
SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression
复制标题
DOI：
10.48550/arxiv.2206.09888
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Zhize Li;Haoyu Zhao;Boyue Li;Yuejie Chi
中科院分区：
文献类型：
--
作者：
Zhize Li;Haoyu Zhao;Boyue Li;Yuejie Chi
为了在无线网络等带宽需求大的环境中实现大规模机器学习，最近在通信压缩的帮助下设计通信高效的联邦学习算法方面取得了重大进展。另一方面，隐私保护，特别是在客户端级别，是另一个重要的需求，但在先进的通信压缩技术的存在下尚未同时得到解决。在本文中，我们提出了一个统一的框架，通过通信压缩来提高私有联邦学习的通信效率。利用通用压缩算子和局部差分隐私，我们首先检查一种简单的算法，该算法直接将压缩应用于差分隐私随机梯度下降，并确定其局限性。然后，我们提出了一个用于私有联邦学习的统一框架 SoteriaFL，它容纳了一系列局部梯度估计器，包括流行的随机方差减少梯度方法和最先进的移位压缩方案。我们在隐私、实用性和通信复杂性方面对其性能权衡进行了全面的表征，其中与其他没有通信压缩的私有联邦学习算法相比，SoteraFL 在不牺牲隐私和实用性的情况下实现了更好的通信复杂性。
To enable large-scale machine learning in bandwidth-hungry environments such as wireless networks, significant progress has been made recently in designing communication-efficient federated learning algorithms with the aid of communication compression. On the other end, privacy-preserving, especially at the client level, is another important desideratum that has not been addressed simultaneously in the presence of advanced communication compression techniques yet. In this paper, we propose a unified framework that enhances the communication efficiency of private federated learning with communication compression. Exploiting both general compression operators and local differential privacy, we first examine a simple algorithm that applies compression directly to differentially-private stochastic gradient descent, and identify its limitations. We then propose a unified framework SoteriaFL for private federated learning, which accommodates a general family of local gradient estimators including popular stochastic variance-reduced gradient methods and the state-of-the-art shifted compression scheme. We provide a comprehensive characterization of its performance trade-offs in terms of privacy, utility, and communication complexity, where SoteraFL is shown to achieve better communication complexity without sacrificing privacy nor utility than other private federated learning algorithms without communication compression.