Real-Time Federated Learning at the Wireless Edge via Algorithm-Hardware Co-Design

在过去的几年里，智能手机和物联网（IoT）设备等无线连接设备的数量迅速增长，这些设备产生的数据量不断增加，推动了关键的人工智能（AI）应用。然而，用户越来越不愿意允许他们的私人数据（如媒体、位置或传感器数据）上传到中央位置（如云数据中心）以训练机器学习（ML）模型，数据保护法（如2018年数据保护法）对数据使用的限制越来越严格。联邦学习（FL）是一项改变游戏规则的技术，旨在通过将训练从数据中心转移到网络边缘的用户设备，从而允许敏感数据保留在生成数据的设备上，从而解决日益增长的数据隐私问题。FL在现实世界中具有巨大的隐私敏感应用潜力，如自动驾驶、诊断医疗保健和预测性维护。由于各种原因，边缘FL的操作环境极具挑战性：1)FL客户端拥有的数据是高度异构的（在数据分布、质量和数量方面）和动态的（数据分布随时间而变化）；2)硬件设备具有不同的计算和通信能力，并受到严格的资源限制（例如电池电量）；3) FL客户端工作在不可靠的无线边缘网络条件下。因此，尽管FL有着巨大的前景，但它在需要实时、按需响应的关键任务人工智能应用中的广泛应用存在相当大的障碍，这是由以下几个重大挑战造成的：挑战1)缺乏FL算法，无法同时为动态客户端数据、多样化的客户端硬件和不可靠的无线连接提供一致的性能；挑战2)缺乏对实时、真实FL算法的严格理论分析；挑战3)缺乏优化的、节能的、通用的实时FL硬件加速。为了解决这些重要的挑战，该项目将创造革命性的算法-硬件协同设计方法，使FL成为一个具有无与伦比的速度、性能和无线边缘能效的实时过程，能够满足关键任务应用的严格要求。这项研究将开创一系列原创方法和创新技术，包括：1)颠覆性的轻量级硬件感知FL算法，在实现快速模型更新的同时显著降低通信、计算和能源成本；2)对所提出的算法进行严格的数学分析，以证明它们的收敛速度，并为它们在各种边缘网络条件下的表现提供理论见解；3)集成专业训练加速和功耗降低方法的自动软硬件协同优化框架，实现优化、节能的硬件加速；4)一个独特的原型系统，将集成设计的FL硬件加速器和实时FL算法，并在现实的无线边缘网络测试平台上进行评估。这个项目有可能将FL从一个冗长和脱节的过程转变为一个连续的、实时的过程，同时进行模型训练和部署。拟议的研究将通过创造突破性技术来创造创新的人工智能产品，从而为英国的数字化转型和绿色经济做出贡献，这些产品将显著提高性能和能源效率，同时遵守严格的数据隐私保护。

The past years have witnessed a rapidly growing number of wirelessly-connected devices such as smartphones and Internet-of-Things (IoT) equipment, which generate ever-increasing amounts of data driving key Artificial Intelligence (AI) applications. However, users are increasingly unwilling to allow their private data (such as media, location, or sensor data) to be uploaded to a central location (e.g., cloud datacentre) for training Machine Learning (ML) models, and data-protection laws such as the Data Protection Act 2018 are growing more restrictive towards data usage. Federated Learning (FL) is a game-changing technology conceived to address the growing data privacy concern by moving training from the datacentre to user devices at the network edge, allowing sensitive data to remain on the devices where it was generated. FL has enormous potential for real-world, privacy-sensitive applications such as autonomous driving, diagnostic healthcare, and predictive maintenance.The operating environment for FL at the edge is extremely challenging for a variety of reasons: 1) the data owned by FL clients is highly heterogeneous (in regard to data distribution, quality, and quantity) and dynamic (data distributions change over time); 2) the hardware devices have diverse computing and communication capabilities with stringent resource constraints (e.g., battery power); and 3) FL clients work under unreliable wireless edge network conditions. Hence, despite FL's huge promise, there are considerable barriers to its wider real-world adoption for mission-critical AI applications that need real-time, on-demand responses, caused by several grand challenges: Challenge 1) lack of FL algorithms delivering consistent performance for dynamic client data, diverse client hardware, and unreliable wireless connections simultaneously; Challenge 2) lack of rigorous theoretical analyses of real-time, real-world FL algorithms; Challenge 3) lack of optimised, energy-efficient, versatile hardware acceleration for real-time FL.To address these important challenges, this project will create revolutionary algorithm-hardware co-design approaches to make FL a real-time process with unparalleled speed, performance, and energy-efficiency at the wireless edge, capable of meeting the stringent requirements of mission-critical applications. This research will pioneer a set of original methods and innovative technologies including: 1) disruptive lightweight hardware-aware FL algorithms that significantly reduce communication, computing, and energy costs while achieving fast model updates; 2) rigorous mathematical analyses of the proposed algorithms to prove their convergence rates and offer theoretical insights into how they perform under various edge network conditions; 3) an automatic hardware-software co-optimisation framework integrating specialised training-acceleration and power-reduction methods to realise optimised, energy-efficient hardware acceleration; and 4) a unique prototype system that will integrate the designed FL hardware accelerator and real-time FL algorithms and be evaluated in a realistic wireless edge networking testbed.This project has the potential to transform FL from a lengthy and disjointed process to a continuous, real-time procedure with concurrent model training and deployment. The proposed research will contribute to the UK's digital transformation and green economy by creating ground-breaking technologies for creating innovative AI-empowered products with significantly improved performance and energy-efficiency while complying with strict data-privacy protection.