CAREER: Towards Efficient and Fast Hierarchical Federated Learning in Heterogeneous Wireless Edge Networks

基于机器学习(ML)的人工智能(AI)在各个领域的快速渗透和无线应用的爆炸性增长促使无线联邦学习(WFL)，它可以在无线边缘网络中通过联邦学习(FL)实现协作智能。本项目探索无线分层联邦学习(WHFL)，它利用分层通信结构来显著降低WFL的通信成本。它为WHFL开发了基本的理解以及自适应和高效的算法和方案，同时解决了几个以前主要未探索的独特挑战：1)参与设备的同质计算配置(包括本地迭代数、小批量大小、步长)缺乏灵活性；2)基于带宽共享的通信资源分配效率低下；3)分层通信结构对异类设备的复杂影响。该项目探索无线网络和机器学习的创新跨学科研究，旨在为未来基于数据分析的智能网络计算系统提供有用的网络研究见解。该项目的研究成果有可能实现无线网络的智能控制和管理，并支持无线网络系统上的各种新兴人工智能应用，如联网和自动驾驶车辆，以及协作机器人。各种实质性的教育项目被整合到拟议的研究中，包括面向大学生的实践无线和ML/AI项目，以及面向K-12学生的机器人外展活动。本项目研究无线边缘网络中的分层FL，用于具有不同计算和通信能力的设备。本文的研究是基于我们前期工作中的一些关键发现：1)异质计算配置，特别是异质局部迭代次数对FL的学习精度和学习代价有很大的影响；2)基于分时的通信资源分配比带宽共享更有效，但它导致了计算配置和通信调度之间的非平凡耦合；3)当设备具有异质计算配置时，分层模型的通信和聚合具有非平凡的影响。基于这些见解，建议的研究被组织为以下三个相互依赖的推动力：i)本地集群中自适应的成本感知设备选择和计算配置；ii)联合设计计算配置和通信调度，以实现本地集群中的快速收敛；iii)全局模型通信和聚合，以实现跨本地集群的准确学习。所提出的WHFL方案和算法将被实现以评估其实际性能。该项目由CNS和已建立的激励竞争性研究计划(EPSCoR)共同资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The accelerating penetration of machine learning (ML) based artificial intelligence (AI) in a variety of domains and the explosive growth of wireless applications spur wireless federated learning (WFL), which can achieve collaborative intelligence via federated learning (FL) in wireless edge networks. This project explores wireless hierarchical federated learning (WHFL), which leverages a hierarchical communication structure to substantially reduce the communication costs of WFL. It develops fundamental understandings as well as adaptive and efficient algorithms and schemes for WHFL while addressing several unique challenges that have been predominantly unexplored before: 1) inflexibility of homogeneous computation configurations (including local iteration number, mini-batch size, step size) for participating devices; 2) inefficiency of bandwidth-sharing based communication resource allocation; 3) complex impacts of the hierarchical communication structure for heterogeneous devices. The project explores innovative cross-disciplinary research of wireless networking and machine learning, and aims to provide useful insights of networking research for future intelligent networked computational systems based on data analytics. The research outcomes of this project have the potential to enable intelligent control and management of wireless networks, and also support various emerging AI applications over wireless networked systems, such as connected and autonomous vehicles, and collaborative robots. Various substantial education programs are integrated with the proposed research, including hands-on wireless and ML/AI projects for college students, and outreach activities on robotics for K-12 students.This project studies hierarchical FL in wireless edge networks for devices with heterogeneous computation and communication capabilities. The proposed research is motivated by some key insights obtained from our preliminary work: 1) heterogeneous computation configurations, particularly heterogeneous local iteration numbers, have non-trivial impacts on the learning accuracy and learning cost of FL; 2) time-sharing based communication resource allocation is more efficient than bandwidth-sharing, while it results in non-trivial coupling between computation configuration and communication scheduling; 3) the hierarchical model communication and aggregation have non-trivial impacts when devices have heterogeneous computation configurations. With these insights, the proposed research is organized into the following three interdependent thrusts: i) adaptive cost-aware device selection and computation configuration in a local cluster; ii) co-design of computation configuration and communication scheduling for fast convergence in a local cluster; iii) global model communication and aggregation for accurate learning across local clusters. The proposed schemes and algorithms for WHFL will be implemented to evaluate their practical performance. This project is jointly funded by CNS and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.