CCSS: Collaborative Research: Quality-Aware Distributed Computation for Wireless Federated Learning: Channel-Aware User Selection, Mini-Batch Size Adaptation, and Scheduling

随着ML/AI技术的爆炸性增长，发展网络技术以实现联网系统上的分布式ML/AI数据分析具有巨大的潜力。该项目将在无线网络和机器学习的交叉点探索创新的跨学科研究，并研究无线联邦学习(FL)在无线网络中实现协作智能。这将促进对无线FL的质量感知动态分布式计算和计算-通信协同设计的基本理解。该项目将激发新的思路和提供新的见解，以支持无线网络系统上的各种新兴ML/AI应用，如协作机器人、多用户混合现实以及无线网络的智能控制和管理。拟议的研究还将通过课程开发、研究经验和外联，与私人投资机构的研究生、本科生和K-12学生的教育活动相结合。PIS将认真努力招收少数族裔研究生。本项目将研究无线FL的质量感知分布式计算，重点是信道感知用户选择、通信调度和自适应小批量设计。所提出的研究是建立在这样一个关键观察之上的：FL中训练模型的学习精度在很大程度上取决于参与学习过程的用户的动态选择以及他们的局部模型更新的质量(这由他们的小批量大小决定)。本地更新的质量可以被视为设计参数，并用作基于用户的通信和计算成本以及能力的跨用户和随时间的自适应控制的旋钮。基于这一认识，PI将1)在包括非IID数据、非凸损失函数和异步分布式学习的一般设置下，量化用户局部随机梯度更新的方差对学习精度的影响；2)开发自适应算法，根据用户的信道条件及其局部更新对训练损失的影响，在FL算法的每一轮中选择参与用户并设置他们的小批量大小；3)通过研究计算工作量和通信调度之间的复杂耦合，联合设计用户的小批量并调度他们的通信以减少学习时间。多目标优化将被用来在学习准确性和学习成本(或学习时间)之间取得适当的平衡。该项目由电气、通信和网络系统部门(ECCS)和既定的激励竞争研究计划(EPSCoR)联合资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

With the explosive growth of ML/AI technologies, there is enormous potential to advance networking technologies to enable distributed ML/AI data analytics over networked systems. This project will explore innovative cross-disciplinary research at the intersections of wireless networking and machine learning, and study wireless federated learning (FL) for achieving collaborative intelligence in wireless networks. It will advance the fundamental understanding of quality-aware dynamic distributed computation and computation-communication co-design for wireless FL. This project will spur a new line of thinking and provide new insights to support various emerging ML/AI applications over wireless networked systems, such as collaborative robotics, multi-user mixed reality, and intelligent control and management of wireless networks. The proposed research will also be integrated with education activities at the PIs' institutions for graduate, undergraduate, and K-12 students via curriculum development, research experiences, and outreach. The PIs will make conscientious effort to recruit minority graduate students.This project will study quality-aware distributed computation for wireless FL, with focuses on channel-aware user selection, communication scheduling, and adaptive mini-batch size design. The proposed research is built on the key observation that the learning accuracy of the trained model in FL depends heavily on dynamic selection of users participating in the learning process and the quality of their local model updates (which is determined by their mini-batch sizes). The quality of local updates can be treated as a design parameter and used as a knob for adaptive control across users and over time based on users' communication and computation costs as well as capabilities. With this insight, the PIs will 1) quantify the impacts of the variances of users' local stochastic gradient updates on learning accuracy over the learning process, for general settings including non-IID data, non-convex loss functions, and asynchronous distributed learning; 2) develop adaptive algorithms that select the participating users and set their mini-batch sizes in each round of the FL algorithm, based on users' channel conditions and the impacts of their local updates on the training loss; 3) jointly design users' mini-batch sizes and schedule their communications to reduce the learning time, by investigating the intricate coupling between computation workloads and communication scheduling. Multi-objective optimization will be used to strike the right balance between learning accuracy and learning cost (or learning time).This project is jointly funded by the Division of Electrical, Communications and Cyber Systems (ECCS), and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.