MLWiNS: Democratizing AI through Multi-Hop Federated Learning Over-the-Air

联合学习(FL)已成为实现大规模保护隐私的下一代人工智能的关键技术，其中大量边缘设备(如移动电话)协作学习共享的全球模型，同时将其数据保存在本地以防止隐私泄露。在无线多跳网络上启用FL，如无线社区网状网和卫星星座上的无线互联网，不仅可以增强城市移动用户的人工智能体验，还可以使人工智能民主化，使每个人都能以低成本的方式访问它，包括低收入社区、农村地区、欠发达地区和灾区的人们。本项目的总体目标是开发一种新型的无线多跳FL系统，具有稳定性好、精度高、收敛速度快的特点。该项目有望推进分布式深度学习系统的设计，促进对分布式计算和分布式网络之间强大协同作用的理解，并弥合分布式深度学习的理论基础和实际应用之间的差距。该项目还将通过私人投资促进机构将开发和提供的研究工作和相关课程，为研究生和本科生提供独特的跨学科培训机会。该项目提出使用联邦学习和多智能体强化学习的概念来为在由于噪声和干扰丰富的无线链路而具有通信约束的无线多跳网络上训练DL模型提供最优解决方案。主要工作包括：1)提出了一种分层联邦计算、半异步模型聚合和正则化目标函数的层次化FL系统结构，显著提高了系统的可扩展性、通信效率和稳定性；2)通过多智能体强化学习对FL系统进行微调，在边缘设备的计算约束下以最小的收敛时间最大化FL的精度；3)为资源受限的边缘设备找到了高增益的计算轻便的联邦计算策略，包括高效的DL模型设计和资源感知的模型自适应；和4)开发一个开源无线FL框架(OpenWFL)，用于在模拟器和物理测试中快速原型、部署和评估建议的FL算法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Federated learning (FL) has emerged as a key technology for enabling next-generation privacy-preserving AI at-scale, where a large number of edge devices, e.g., mobile phones, collaboratively learn a shared global model while keeping their data locally to prevent privacy leakage. Enabling FL over wireless multi-hop networks, such as wireless community mesh networks and wireless Internet over satellite constellations, not only can augment AI experiences for urban mobile users, but also can democratize AI and make it accessible in a low-cost manner to everyone, including people in low-income communities, rural areas, under-developed regions, and disaster areas. The overall objective of this project is to develop a novel wireless multi-hop FL system with guaranteed stability, high accuracy and fast convergence speed. This project is expected to advance the design of distributed deep learning (DL) systems, to promote the understanding of the strong synergy between distributed computing and distributed networking, and to bridge the gap between the theoretical foundations of distributed DL and its real-life applications. The project will also provide unique interdisciplinary training opportunities for graduate and undergraduate students through both research work and related courses that the PIs will develop and offer. This project proposes to use concepts of federated learning and multi-agent reinforcement learning to provide optimal solutions for training DL models over wireless multi-hop networks that have communication constraints due to noisy and interference-rich wireless links. The main thrusts include: 1) developing a novel hierarchical FL system architecture with layered federated computation, semi-asynchronous model aggregation, and regularized objective function to significantly improve system scalability, communication efficiency, and stability; 2) fine-tuning the FL system via multi-agent reinforcement learning to maximize the FL accuracy with the minimum convergence time under the computing constraints of edge devices; 3) finding high-gain computation-light robust federated computing strategies for resource-constraint edge devices, including efficient DL model design and resource-aware model adaptation; and 4) developing an open-source wireless FL framework (OpenWFL) for fast prototyping, deploying, and evaluating the proposed FL algorithms in both an emulator and physical testbeds.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.