Scalable Federated Learning and Analytics with Communication Efficiency in Mobile Cloud Computing

移动云计算中具有通信效率的可扩展联合学习和分析

• 批准号: RGPIN-2022-04782
• 负责人: Li, Baochun
• 金额: $ 5.54万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=751618
• 关键词: Scalable; Federated; Learning; Analytics; Communication

Today, mobile cloud computing becomes the norm and machine learning (ML) becomes a necessity. Millions of devices at the network edge, from smartphones to Internet of Things (IoT) sensors, need to collaborate autonomously to perform complex tasks with the assistance of the cloud. These complex tasks, such as object detection and question-answering, may involve both ML training and analytics at the edge devices. This research focuses on the design of scalable, personalized, and communication-efficient mechanisms for next-generation federated learning and analytics with large numbers of edge devices. Although both federated learning and federated analytics have drawn much recent attention, they remain at a nascent stage of research, especially as the number of edge devices scales up in mobile cloud computing systems. The majority of existing studies have focused on traditional client-server architectures, which is far from scalable. As the number of devices scales up and the network traffic load increases, devices become more heterogeneous in their capabilities and data distribution, and communication efficiency is far more difficult to achieve. Further exacerbating this challenge, as a shared global model is trained and then deployed for inference, existing work failed to consider the need for personalization on individual devices, customized for its locally collected dataset and tailored to its own requirements. The long-term goal of this research program is to improve the scalability of both federated learning and analytics, by substantially improving both communication efficiency and the effectiveness of training personalized models and using them for local analytics. Towards such a long-term goal, this research proposes to achieve a number of specific objectives. First, we aim to achieve a balanced trade-off between model performance, privacy preservation, and communication efficiency, and design new local differential privacy, model quantization and aggregation mechanisms. Second, we will investigate the design of new algorithms for fine-tuning meta-learning models for personalized for individual needs on each edge device. Third, we will study how inference workload with large and complex models in federated analytics can be best distributed across multiple edge devices to satisfy their power and capability constraints. Finally, to demonstrate real-world feasibility of this proposed research, we will build a production-quality open-source framework that encompasses our proposed solutions in this research program, so that they can be tested in a real-world production environment. The program will train highly qualified personnel (HQP) in the intersection of deep learning theory and distributed computing systems, offering a mix of theoretical and hands-on skill sets that are in high demand today. A strong emphasis on equity, diversity, and inclusion (EDI) will be consistently carried out during the program.

今天，移动云计算成为常态，机器学习(ML)成为必要。从智能手机到物联网(IoT)传感器，数百万处于网络边缘的设备需要在云的帮助下自主协作来执行复杂的任务。这些复杂的任务，如对象检测和问题回答，可能涉及边缘设备上的ML训练和分析。这项研究的重点是设计可扩展、个性化和通信高效的机制，用于使用大量边缘设备进行下一代联合学习和分析。尽管联合学习和联合分析最近都引起了很多关注，但它们仍处于研究的初级阶段，特别是在移动云计算系统中边缘设备数量不断增加的情况下。现有的大多数研究都集中在传统的客户端-服务器体系结构上，这远远不是可伸缩的。随着设备数量的增加和网络流量负载的增加，设备的能力和数据分布变得更加异构化，通信效率更难实现。进一步加剧这一挑战的是，由于训练了一个共享的全球模型，然后将其部署用于推理，现有的工作未能考虑到在个别设备上进行个性化的需要，这些设备是为其本地收集的数据集定制的，并根据其自身的要求进行定制。该研究计划的长期目标是通过大幅提高通信效率和培训个性化模型并将其用于本地分析的有效性，来提高联合学习和分析的可扩展性。为了达到这样的长远目标，本研究提出了一些具体的目标。首先，我们的目标是在模型性能、隐私保护和通信效率之间实现平衡，并设计新的局部差异隐私、模型量化和聚合机制。其次，我们将研究针对每个边缘设备上的个性化需求微调元学习模型的新算法的设计。第三，我们将研究联合分析中具有大型复杂模型的推理工作负载如何跨多个边缘设备进行最佳分配，以满足其功率和能力限制。最后，为了证明这项建议的研究在现实世界中的可行性，我们将构建一个生产质量的开源框架，该框架包含我们在此研究计划中建议的解决方案，以便它们可以在真实的生产环境中进行测试。该计划将培训深度学习理论和分布式计算系统交叉领域的高素质人才(HQP)，提供当今高度需求的理论和实践技能组合。该计划将始终如一地强调公平、多样性和包容性(EDI)。