SPACE: fully decentralised distributed learning for tradeoff of privacy, accuracy, communication complexity, and efficiency

SPACE：完全去中心化的分布式学习，以权衡隐私、准确性、通信复杂性和效率

• 批准号: 10046257
• 金额: $ 7.14万
• 依托单位: UNIVERSITY OF LIVERPOOL
• 依托单位国家: 英国
• 项目类别: CR&D Bilateral
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10046257
• 关键词: SPACE; fully; decentralised; distributed; learning

Training data of a machine learning application may be collected from geographically different places, but transferring distributed data to a central server for training can be legally or practically impossible. This has led to a fast-growing area in machine learning, i.e., federated learning. However, it was discovered that the federated learning in its original form, where a server iteratively aggregates local gradients received from the distributed users, may suffer from privacy leakage as an attacker can infer sensitive information from the local gradients. Since then, many improvements have been presented to mitigate this privacy leakage, but they often achieve better privacy with the compromise of other critical properties such as the accuracy and communication complexity. In this paper, we propose a novel solution that is based on a fully decentralised distributed learning. Our solution enables the optimisation over, and achieve a balance between, multiple properties, including privacy preservation, accuracy, communication complexity, efficiency, and tolerance to user failures. Our solution proceeds by first synthesising a communication topology between users according to the required properties and then applying a fully decentralised distributed learning where the server is not involved in the computation. In the decentralised learning, the aggregation of local gradients is reduced to a distributed consensus between users. Finally, the agreed value of the users is sent to the server after added a differential privacy noise. We are conducting experiments on both use cases in the US/UK privacy enhancing technologies challenge to validate our solution.

机器学习应用程序的培训数据可以从地理上不同的地方收集，但是将分布式数据传输到中央服务器进行培训是合法或实际上是不可能的。这导致了机器学习的快速发展领域，即联合学习。但是，发现以原始形式的联合学习，服务器迭代地汇总了从分布式用户接收的本地梯度，可能会遭受隐私泄漏，因为攻击者可以从本地梯度中推断敏感信息。从那时起，已经提出了许多改进，以减轻这种隐私泄漏，但是由于精度和沟通复杂性等其他关键属性的妥协，它们通常会获得更好的隐私。在本文中，我们提出了一种基于完全分散的分布式学习的新颖解决方案。我们的解决方案可以优化并在多个属性之间达到平衡，包括隐私保护，准确性，沟通复杂性，效率和对用户失败的耐受性。我们的解决方案是通过首先根据所需属性综合用户之间的通信拓扑来进行的，然后应用完全分散的分布式学习，其中服务器不参与计算。在分散的学习中，将本地梯度的聚合减少为用户之间的分布式共识。最后，在添加了不同的隐私噪声后，将用户的商定价值发送到服务器。我们正在对美国/英国隐私增强技术的两种用例进行实验，以验证我们的解决方案。