Collaborative Research: CNS Core: Small: Hierarchical Federated Learning Over Wireless Edge Networks: Performance Analysis and Optimization

联合学习(FL)通过催化从基于云的集中式学习向分布式、设备上边缘学习的范式转变，正在给机器学习带来革命性的变化。FL使设备能够通过使用本地处理和简单的学习参数交换来协作地训练和执行全局学习任务，从而避免了与远程云共享大数据量相关的通信和隐私问题。由于其诱人的隐私、可扩展性和通信特性，FL将成为物联网(IoT)服务(如自主系统)不可或缺的边缘组件。然而，当部署在无线物联网边缘时，FL的性能将在很大程度上受到用于交换本地和全局FL模型参数的无线链路的质量的限制。由于下一代物联网将由无线蜂窝系统(例如，5G)供电，因此，能否获得物联网FL的优势取决于了解衰落、干扰和延迟等无线因素如何影响FL的融合和性能(例如，准确性、可靠性和融合时间)。这项研究的目标是开发一个基本框架，严格回答有关在现实、大规模无线边缘网络上可实现的FL性能的基本问题，从而促进FL与现实世界物联网的整合。这项研究与精心设计的教育计划相结合，其中包括在通信和机器学习的交叉点开设一门新课程，以及各级研究生和本科生的大量参与。这项研究将通过几个研讨会、教程、外展活动和其他工具来确保广泛的传播和外延。这项研究将为大规模无线蜂窝边缘网络上的FL性能分析和优化开发一个新颖的、整体的框架。所提出的框架将在无线和FL领域产生重大创新：1)允许在无线蜂窝系统上大规模实施FL的可扩展层次化无线体系结构；2)无线边缘网络上的层次化FL的严格性能分析，其将产生结合学习性能指标的新颖FL性能度量，例如训练精度和收敛时间；3)无线网络上FL的新的可靠性概念，以使FL能够在极端网络条件下以及在存在物联网设备移动性和时空相关性的情况下运行；以及4)合适的资源分配算法，可以优化无线边缘网络上的层次化FL的性能。结果将使用各种模拟和实验手段进行验证。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Federated learning (FL) is revolutionizing machine learning by catalyzing a paradigm shift from cloud-based centralized learning towards distributed, on-device edge learning. FL enables devices to collaboratively train and execute a global learning task by using local processing and simple learning parameters exchange, thus avoiding the communication and privacy concerns associated with sharing large data volumes with a remote cloud. Owing to its attractive privacy, scalability, and communication features, FL will be an integral edge component of Internet of Things (IoT) services such as autonomous systems. However, when deployed over the wireless IoT edge, the performance of FL will be largely constrained by the quality of the wireless links used to exchange the local and global FL model parameters. Since the next-generation IoT will be powered by a wireless cellular system (e.g., 5G), reaping the benefits of FL for the IoT hinges on understanding how wireless factors, such as fading, interference, and delay, impact the convergence and performance of FL (e.g., accuracy, reliability, and convergence time). The goal of this research is to develop a foundational framework that rigorously answers fundamental questions on the achievable FL performance over realistic, large-scale wireless edge networks thus facilitating FL integration unto a real-world IoT. The research is coupled with a well-crafted educational plan that includes a new course at the intersection of communications and machine learning as well as a significant involvement of graduate and undergraduate students at all levels. Broad dissemination and outreach will be ensured via several workshops, tutorials, outreach events, and other tools.This research will develop a novel, holistic framework for performance analysis and optimization of FL over large-scale wireless cellular edge networks. The proposed framework will yield major innovations across both wireless and FL fields: 1) A scalable hierarchical wireless architecture that allows a large-scale implementation of FL over wireless cellular systems, 2) Rigorous performance analysis of hierarchical FL over wireless edge networks that will yield novel FL performance metrics that jointly couple learning performance indicators, such as training accuracy and convergence time, 3) Novel notions of reliability for FL over wireless networks to enable the operation of FL under extreme network conditions and in presence of IoT device mobility and spatio-temporal correlations, and 4) Suitable resource allocation algorithms that can optimize the performance of hierarchical FL over wireless edge networks. The results will be validated using various simulation and experimental means.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.