BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning
BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Chengliang Zhang;Suyi Li;Junzhe Xia;Wei Wang;Feng Yan;Yang Liu
中科院分区：
文献类型：
--
作者：
Chengliang Zhang;Suyi Li;Junzhe Xia;Wei Wang;Feng Yan;Yang Liu
跨筒仓联邦学习（FL）使组织（例如金融或医疗组织）能够通过聚合来自每个客户端的局部梯度更新来协作训练机器学习模型，而无需共享隐私敏感数据。为了确保在聚合过程中不会泄露任何更新，工业联邦学习框架允许客户端使用加法同态加密（HE）来掩盖局部梯度更新。然而，这会导致计算和通信方面的巨大成本。在我们的特性描述中，同态加密操作主导了训练时间，同时使数据传输量增加了两个数量级。在本文中，我们提出了BatchCrypt，一种用于跨筒仓联邦学习的系统解决方案，它大幅降低了由同态加密导致的加密和通信开销。我们不是以全精度加密单个梯度，而是将一批量化梯度编码为一个长整数并一次性对其进行加密。为了能够对编码批次的密文进行按梯度聚合，我们开发了新的量化和编码方案以及一种新颖的梯度裁剪技术。我们将BatchCrypt作为一个插件模块在FATE（一个工业跨筒仓联邦学习框架）中实现。在地理分布式数据中心中使用EC2客户端进行的评估表明，BatchCrypt实现了23倍 - 93倍的训练加速，同时将通信开销降低了66倍 - 101倍。由于量化误差导致的精度损失小于1%。
Cross-silo federated learning (FL) enables organizations (e.g., financial or medical) to collaboratively train a machine learning model by aggregating local gradient updates from each client without sharing privacy-sensitive data. To ensure no update is revealed during aggregation, industrial FL frameworks allow clients to mask local gradient updates using ad-ditively homomorphic encryption (HE). However, this results in significant cost in computation and communication . In our characterization, HE operations dominate the training time, while inflating the data transfer amount by two orders of magnitude. In this paper, we present BatchCrypt, a system solution for cross-silo FL that substantially reduces the encryption and communication overhead caused by HE. Instead of encrypting individual gradients with full precision, we encode a batch of quantized gradients into a long integer and encrypt it in one go. To allow gradient-wise aggregation to be performed on ciphertexts of the encoded batches, we develop new quantization and encoding schemes along with a novel gradient clipping technique. We implemented BatchCrypt as a plug-in module in FATE, an industrial cross-silo FL framework. Evaluations with EC2 clients in geo-distributed datacenters show that BatchCrypt achieves 23 × -93 × training speedup while reducing the communication overhead by 66 × -101 × . The accuracy loss due to quantization errors is less than 1%.