Every Vote Counts: Ranking-Based Training of Federated Learning to Resist Poisoning Attacks
Every Vote Counts: Ranking-Based Training of Federated Learning to Resist Poisoning Attacks
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Hamid Mozaffari;Virat Shejwalkar;Amir Houmansadr
中科院分区：
文献类型：
--
作者：
Hamid Mozaffari;Virat Shejwalkar;Amir Houmansadr
联邦学习（FL）允许不受信任的客户端协作训练一个通用的机器学习模型，称为全局模型，而无需共享他们的私有/专有训练数据。然而，FL很容易受到恶意客户端的毒害，这些客户端旨在通过在FL的训练过程中提供恶意更新来阻碍全局模型的准确性。我们认为，针对现有FL系统的中毒攻击成功的关键因素是客户端可以选择的大量模型更新空间。为了解决这个问题，我们提出了联邦秩学习（FRL）。FRL将客户端的更新空间从标准FL中的模型参数更新空间（浮点数的连续空间）减少到参数排名空间（整数值的离散空间）。为了能够使用参数排名（而不是参数权重）来训练全局模型，FRL利用了最近的超掩模训练机制的思想。具体而言，FRL客户端根据其本地训练数据对随机初始化的神经网络（由服务器提供）的参数进行排名，FRL服务器使用投票机制将客户端提交的参数排名进行汇总。直观地说，我们基于投票的聚合机制可以防止恶意客户端对全局模型进行重大的对抗性修改，因为每个客户端都有一票！通过分析证明和实验证明了该算法对中毒的鲁棒性，并证明了其具有较高的通信效率。1 ．
Federated learning (FL) allows untrusted clients to collaboratively train a common machine learning model, called global model , without sharing their private/proprietary training data. However, FL is susceptible to poisoning by malicious clients who aim to hamper the accuracy of the global model by contributing malicious updates during FL’s training process. We argue that the key factor to the success of poisoning attacks against existing FL systems is the large space of model updates available to the clients to choose from. To address this, we propose Federated Rank Learning (FRL). FRL reduces the space of client updates from model parameter updates (a continuous space of float numbers) in standard FL to the space of parameter rankings (a discrete space of integer values). To be able to train the global model using parameter ranks (instead of parameter weights), FRL leverage ideas from recent supermasks training mechanisms. Specifically, FRL clients rank the parameters of a randomly initialized neural network (provided by the server) based on their local training data, and the FRL server uses a voting mechanism to aggregate the parameter rankings submitted by the clients. Intuitively, our voting-based aggregation mechanism prevents poisoning clients from making significant adversarial modifications to the global model, as each client will have a single vote! We demonstrate the robustness of FRL to poisoning through analytical proofs and experimentation, and we show its high communication efficiency. 1 .