Faithful Edge Federated Learning: Scalability and Privacy
Faithful Edge Federated Learning: Scalability and Privacy
复制标题
DOI：
10.1109/jsac.2021.3118423
复制
发表时间：
2021-06
影响因子：
16.4
通讯作者：
Meng Zhang;Ermin Wei;R. Berry
中科院分区：
文献类型：
--
作者：
Meng Zhang;Ermin Wei;R. Berry
联邦学习使机器学习算法能够在分散的边缘设备上进行训练，而无需交换本地数据集。成功部署联邦学习需要确保智能体（例如移动设备）忠实地执行预期的算法，这在文献中很大程度上被忽视了。在这项研究中，我们首先使用风险界限来分析联邦学习的关键特征——不平衡且非独立同分布的数据如何影响智能体自愿参与和顺从地遵循传统联邦学习算法的动机。更具体地说，我们的分析表明，具有不太典型的数据分布和相对较多样本的智能体更有可能退出或篡改联邦学习算法。为此，我们提出了联邦学习的第一个忠实执行问题，并设计了两种满足经济特性、可扩展性和隐私性的忠实联邦学习机制。首先，我们设计了一种忠实联邦学习（FFL）机制，它通过增量计算来近似维克里 - 克拉克 - 格罗夫斯（VCG）支付。我们表明它实现了（可能近似的）最优性、忠实执行、自愿参与以及一些其他经济特性（如预算平衡）。此外，智能体数量$K$的时间复杂度为$\mathcal{O}(\log(K))$。其次，通过将智能体划分为几个集群，我们提出了一种可扩展的VCG机制近似。我们进一步设计了一种可扩展且具有差分隐私的FFL（DP - FFL）机制，这是第一种具有差分隐私的忠实机制，它保持了经济特性。我们的DP - FFL机制使人们能够在隐私、所需迭代次数和支付准确性损失之间进行三方面的性能权衡。
Federated learning enables machine learning algorithms to be trained over decentralized edge devices without requiring the exchange of local datasets. Successfully deploying federated learning requires ensuring that agents (e.g., mobile devices) faithfully execute the intended algorithm, which has been largely overlooked in the literature. In this study, we first use risk bounds to analyze how the key feature of federated learning, unbalanced and non-i.i.d. data, affects agents’ incentives to voluntarily participate and obediently follow traditional federated learning algorithms. To be more specific, our analysis reveals that agents with less typical data distributions and relatively more samples are more likely to opt out of or tamper with federated learning algorithms. To this end, we formulate the first faithful implementation problem of federated learning and design two faithful federated learning mechanisms which satisfy economic properties, scalability, and privacy. First, we design a Faithful Federated Learning (FFL) mechanism which approximates the Vickrey–Clarke–Groves (VCG) payments via an incremental computation. We show that it achieves (probably approximate) optimality, faithful implementation, voluntary participation, and some other economic properties (such as budget balance). Further, the time complexity in the number of agents $K$ is $\mathcal {O}(\log (K))$ . Second, by partitioning agents into several clusters, we present a scalable VCG mechanism approximation. We further design a scalable and Differentially Private FFL (DP-FFL) mechanism, the first differentially private faithful mechanism, that maintains the economic properties. Our DP-FFL mechanism enables one to make three-way performance tradeoffs among privacy, the iterations needed, and payment accuracy loss.