SPACE: fully decentralised distributed learning for tradeoff of privacy, accuracy, communication complexity, and efficiency

SPACE：完全去中心化的分布式学习，以权衡隐私、准确性、通信复杂性和效率

• 批准号: 10046257
• 金额: $ 7.14万
• 依托单位: UNIVERSITY OF LIVERPOOL
• 依托单位国家: 英国
• 项目类别: CR&D Bilateral
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10046257
• 关键词: SPACE; fully; decentralised; distributed; learning

Training data of a machine learning application may be collected from geographically different places, but transferring distributed data to a central server for training can be legally or practically impossible. This has led to a fast-growing area in machine learning, i.e., federated learning. However, it was discovered that the federated learning in its original form, where a server iteratively aggregates local gradients received from the distributed users, may suffer from privacy leakage as an attacker can infer sensitive information from the local gradients. Since then, many improvements have been presented to mitigate this privacy leakage, but they often achieve better privacy with the compromise of other critical properties such as the accuracy and communication complexity. In this paper, we propose a novel solution that is based on a fully decentralised distributed learning. Our solution enables the optimisation over, and achieve a balance between, multiple properties, including privacy preservation, accuracy, communication complexity, efficiency, and tolerance to user failures. Our solution proceeds by first synthesising a communication topology between users according to the required properties and then applying a fully decentralised distributed learning where the server is not involved in the computation. In the decentralised learning, the aggregation of local gradients is reduced to a distributed consensus between users. Finally, the agreed value of the users is sent to the server after added a differential privacy noise. We are conducting experiments on both use cases in the US/UK privacy enhancing technologies challenge to validate our solution.

机器学习应用程序的训练数据可以从地理上不同的地方收集，但将分布式数据传输到中央服务器进行训练在法律上或实际上是不可能的。这导致了机器学习中一个快速增长的领域，即，联邦学习然而，人们发现，在其原始形式的联合学习中，服务器迭代地聚合从分布式用户接收的局部梯度，可能会遭受隐私泄露，因为攻击者可以从局部梯度中推断敏感信息。从那时起，已经提出了许多改进来减轻这种隐私泄漏，但它们通常在其他关键属性（如准确性和通信复杂性）的妥协下实现更好的隐私。在本文中，我们提出了一种基于完全分散的分布式学习的新解决方案。我们的解决方案能够优化多个属性，并在多个属性之间实现平衡，包括隐私保护，准确性，通信复杂性，效率和对用户故障的容忍度。我们的解决方案首先根据所需的属性合成用户之间的通信拓扑，然后应用完全分散的分布式学习，其中服务器不参与计算。在分散式学习中，局部梯度的聚合被简化为用户之间的分布式共识。最后，用户的同意值被发送到服务器后，添加一个差分隐私噪声。我们正在对美国/英国隐私增强技术挑战中的两个用例进行实验，以验证我们的解决方案。