CCSS: Collaborative Research: Quality-Aware Distributed Computation for Wireless Federated Learning: Channel-Aware User Selection, Mini-Batch Size Adaptation, and Scheduling

随着ML/AI技术的爆炸式增长，推进网络技术以实现网络系统上的分布式ML/AI数据分析的潜力巨大。该项目将探索无线网络和机器学习交叉点的创新跨学科研究，并研究无线联邦学习（FL），以实现无线网络中的协同智能。它将促进对无线FL的质量感知动态分布式计算和计算通信协同设计的基本理解。该项目将激发新的思路，并提供新的见解，以支持无线网络系统上各种新兴的ML/AI应用，如协作机器人、多用户混合现实以及无线网络的智能控制和管理。拟议的研究还将通过课程开发、研究经验和外展，与pi机构的研究生、本科生和K-12学生的教育活动相结合。私立大学将认真招收少数民族研究生。该项目将研究无线FL的质量感知分布式计算，重点是信道感知用户选择，通信调度和自适应小批量大小设计。提出的研究是建立在关键观察的基础上的，即FL中训练模型的学习精度在很大程度上取决于参与学习过程的用户的动态选择和他们的局部模型更新的质量（这是由他们的小批量大小决定的）。本地更新的质量可以作为一个设计参数，并根据用户的通信和计算成本以及能力，用作跨用户和随时间的自适应控制的旋钮。有了这一见解，pi将1)量化用户局部随机梯度更新的方差对学习过程中学习精度的影响，包括非iid数据、非凸损失函数和异步分布式学习；2)开发自适应算法，根据用户的信道条件及其局部更新对训练损失的影响，在FL算法的每轮中选择参与的用户并设置其mini-batch大小；3)通过研究计算工作量与通信调度之间的复杂耦合，共同设计用户的小批量大小并调度其通信，以减少学习时间。多目标优化将用于在学习精度和学习成本（或学习时间）之间取得适当的平衡。该项目由电气、通信和网络系统部（ECCS）和促进竞争研究的既定计划（EPSCoR）共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With the explosive growth of ML/AI technologies, there is enormous potential to advance networking technologies to enable distributed ML/AI data analytics over networked systems. This project will explore innovative cross-disciplinary research at the intersections of wireless networking and machine learning, and study wireless federated learning (FL) for achieving collaborative intelligence in wireless networks. It will advance the fundamental understanding of quality-aware dynamic distributed computation and computation-communication co-design for wireless FL. This project will spur a new line of thinking and provide new insights to support various emerging ML/AI applications over wireless networked systems, such as collaborative robotics, multi-user mixed reality, and intelligent control and management of wireless networks. The proposed research will also be integrated with education activities at the PIs' institutions for graduate, undergraduate, and K-12 students via curriculum development, research experiences, and outreach. The PIs will make conscientious effort to recruit minority graduate students.This project will study quality-aware distributed computation for wireless FL, with focuses on channel-aware user selection, communication scheduling, and adaptive mini-batch size design. The proposed research is built on the key observation that the learning accuracy of the trained model in FL depends heavily on dynamic selection of users participating in the learning process and the quality of their local model updates (which is determined by their mini-batch sizes). The quality of local updates can be treated as a design parameter and used as a knob for adaptive control across users and over time based on users' communication and computation costs as well as capabilities. With this insight, the PIs will 1) quantify the impacts of the variances of users' local stochastic gradient updates on learning accuracy over the learning process, for general settings including non-IID data, non-convex loss functions, and asynchronous distributed learning; 2) develop adaptive algorithms that select the participating users and set their mini-batch sizes in each round of the FL algorithm, based on users' channel conditions and the impacts of their local updates on the training loss; 3) jointly design users' mini-batch sizes and schedule their communications to reduce the learning time, by investigating the intricate coupling between computation workloads and communication scheduling. Multi-objective optimization will be used to strike the right balance between learning accuracy and learning cost (or learning time).This project is jointly funded by the Division of Electrical, Communications and Cyber Systems (ECCS), and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.