LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learning
LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learning
复制标题
DOI：
--
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Jinhyun So;Chaoyang He;Chien-Sheng Yang;Songze Li;Qian Yu;Ramy E. Ali;Basak Guler;S. Avestimehr
中科院分区：
文献类型：
--
作者：
Jinhyun So;Chaoyang He;Chien-Sheng Yang;Songze Li;Qian Yu;Ramy E. Ali;Basak Guler;S. Avestimehr
安全模型聚合是联邦学习 (FL) 的关键组成部分，旨在保护每个用户的个人模型的隐私，同时允许其全局聚合。它可以应用于任何基于聚合的 FL 方法来训练全局或个性化模型。模型聚合还需要能够抵御 FL 系统中可能出现的用户丢失，从而使其设计变得更加复杂。最先进的安全聚合协议依赖于用户处用于掩码生成的随机种子的秘密共享，以能够重建和取消属于丢失用户的种子。然而，此类方法的复杂性随着用户流失数量的增加而大幅增加。我们提出了一种名为 LightSecAgg 的新方法，通过将设计从“丢弃用户的随机种子重建”更改为“通过掩码编码/解码对活动用户进行一次性聚合掩码重建”来克服这一瓶颈。我们表明，LightSecAgg 实现了与最先进的协议相同的隐私和丢失弹性保证，同时显着降低了针对丢失用户的弹性开销。我们还证明，与现有方案不同，LightSecAgg 可以应用于异步 FL 设置中的安全聚合。此外，我们通过实现模型训练和设备上编码之间的计算重叠，以及提高并发接收和发送分块掩码的速度，提供模块化系统设计和优化的设备上并行化，以实现可扩展的实现。我们通过广泛的实验来评估 LightSecAgg，在具有大量用户的现实 FL 系统中的各种数据集上训练不同的模型，并证明 LightSecAgg 显着减少了总训练时间。
Secure model aggregation is a key component of federated learning (FL) that aims at protecting the privacy of each user's individual model while allowing for their global aggregation. It can be applied to any aggregation-based FL approach for training a global or personalized model. Model aggregation needs to also be resilient against likely user dropouts in FL systems, making its design substantially more complex. State-of-the-art secure aggregation protocols rely on secret sharing of the random-seeds used for mask generations at the users to enable the reconstruction and cancellation of those belonging to the dropped users. The complexity of such approaches, however, grows substantially with the number of dropped users. We propose a new approach, named LightSecAgg, to overcome this bottleneck by changing the design from"random-seed reconstruction of the dropped users"to"one-shot aggregate-mask reconstruction of the active users via mask encoding/decoding". We show that LightSecAgg achieves the same privacy and dropout-resiliency guarantees as the state-of-the-art protocols while significantly reducing the overhead for resiliency against dropped users. We also demonstrate that, unlike existing schemes, LightSecAgg can be applied to secure aggregation in the asynchronous FL setting. Furthermore, we provide a modular system design and optimized on-device parallelization for scalable implementation, by enabling computational overlapping between model training and on-device encoding, as well as improving the speed of concurrent receiving and sending of chunked masks. We evaluate LightSecAgg via extensive experiments for training diverse models on various datasets in a realistic FL system with large number of users and demonstrate that LightSecAgg significantly reduces the total training time.