Collaborative Research: CNS Core: Medium: Towards Federated Learning over 5G Mobile Devices: High Efficiency, Low Latency, and Good Privacy

最近出现的联邦学习（FL）允许分布式数据源在不共享其隐私敏感原始数据的情况下协作训练全局模型。然而，由于深度学习模型的庞大规模，模型的下载和更新会产生大量的网络流量，这给现有的电信基础设施带来了巨大的负担。本项目将基于5G移动设备的FL作为一个可行的应用场景来解决这一困境，这将显著提高基于5G移动设备的FL的设计、分析和实现。研究成果将大大丰富机器学习技术和5G系统及以后的知识。此外，该项目是多学科的，涉及机器学习/深度学习/联邦学习、边缘计算、无线通信与网络、安全与隐私、计算机架构设计等，将为研究生和本科生提供富有成效的培训基地，使他们具备未来劳动力的多学科技能，以推动国民经济。此外，针对高中学生的外展活动将增加女性和少数民族学生对理工科的参与。具体而言，通过观察迭代模型更新倾向于显示高稀疏性，研究人员利用模型更新稀疏性设计模型修剪和量化方案，以优化局部训练和隐私保护模型更新，从而降低能耗和模型更新流量。为了实现这一设计目标，他们开展了四项研究工作：(1)在单个5G移动设备中，根据本地数据和模型稀疏性，设计软硬件协同设计的模型修剪方案和自适应量化技术，以减少本地计算和内存访问；(2)在“工作”（即本地计算）和“通话”（即5G无线传输）之间进行合理权衡，以提高FL比5G移动设备的整体能源/通信效率；(3)开发基于稀疏性和量化适应性的差分私有压缩方案，在保持高模型精度和通信效率的同时严格保护数据隐私；(4)建立测试平台，对设计方案进行全面评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent emerging federated learning (FL) allows distributed data sources to collaboratively train a global model without sharing their privacy sensitive raw data. However, due to the huge size of the deep learning model, the model downloads and updates generate significant amount of network traffic which exerts tremendous burden to existing telecommunication infrastructure. This project takes FL over 5G mobile devices as a workable application scenario to address this dilemma, which will significantly improve the design, analysis and implementation of FL over 5G mobile devices. The research outcomes will substantially enrich the knowledge of machine learning technologies and 5G systems and beyond. Moreover, this project is multidisciplinary, involving machine learning/deep learning/federated learning, edge computing, wireless communications and networking, security and privacy, computer architectural design, etc., which will serve as a fruitful training ground for both graduate and undergraduate students to equip them with multidisciplinary skills for future work force to boost the national economy. Furthermore, outreach activities to high school students will increase the participation of female and minority students in science and engineering.Specifically, by observing that iterative model updates tend to show high sparsity, the investigators leverage model update sparsity to design model pruning and quantization schemes to optimize local training and privacy-preserving model updating in order to lower both energy consumption and model update traffic. They achieve this design goal by conducting the four research tasks: (1) designing software-hardware co-designed model pruning schemes and adaptive quantization techniques in FL within a single 5G mobile device according to the local data and model sparsity property to reduce the local computation and memory access; (2) making sound trade-off between "working" (i.e., local computing) and "talking" (i.e., 5G wireless transmissions) to boost the overall energy/communications efficiency for FL over 5G mobile devices; (3) developing novel differentially private compression schemes based on sparsification property and quantization adaptability to rigorously protect data privacy while maintaining high model accuracy and communication efficiency in FL; and (4) building a testbed to thoroughly evaluate the proposed designs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.