Scalable Federated Learning and Analytics with Communication Efficiency in Mobile Cloud Computing

移动云计算中具有通信效率的可扩展联合学习和分析

• 批准号: RGPIN-2022-04782
• 负责人: Li, Baochun
• 金额: $ 5.54万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=751618
• 关键词: Scalable; Federated; Learning; Analytics; Communication

Today, mobile cloud computing becomes the norm and machine learning (ML) becomes a necessity. Millions of devices at the network edge, from smartphones to Internet of Things (IoT) sensors, need to collaborate autonomously to perform complex tasks with the assistance of the cloud. These complex tasks, such as object detection and question-answering, may involve both ML training and analytics at the edge devices. This research focuses on the design of scalable, personalized, and communication-efficient mechanisms for next-generation federated learning and analytics with large numbers of edge devices. Although both federated learning and federated analytics have drawn much recent attention, they remain at a nascent stage of research, especially as the number of edge devices scales up in mobile cloud computing systems. The majority of existing studies have focused on traditional client-server architectures, which is far from scalable. As the number of devices scales up and the network traffic load increases, devices become more heterogeneous in their capabilities and data distribution, and communication efficiency is far more difficult to achieve. Further exacerbating this challenge, as a shared global model is trained and then deployed for inference, existing work failed to consider the need for personalization on individual devices, customized for its locally collected dataset and tailored to its own requirements. The long-term goal of this research program is to improve the scalability of both federated learning and analytics, by substantially improving both communication efficiency and the effectiveness of training personalized models and using them for local analytics. Towards such a long-term goal, this research proposes to achieve a number of specific objectives. First, we aim to achieve a balanced trade-off between model performance, privacy preservation, and communication efficiency, and design new local differential privacy, model quantization and aggregation mechanisms. Second, we will investigate the design of new algorithms for fine-tuning meta-learning models for personalized for individual needs on each edge device. Third, we will study how inference workload with large and complex models in federated analytics can be best distributed across multiple edge devices to satisfy their power and capability constraints. Finally, to demonstrate real-world feasibility of this proposed research, we will build a production-quality open-source framework that encompasses our proposed solutions in this research program, so that they can be tested in a real-world production environment. The program will train highly qualified personnel (HQP) in the intersection of deep learning theory and distributed computing systems, offering a mix of theoretical and hands-on skill sets that are in high demand today. A strong emphasis on equity, diversity, and inclusion (EDI) will be consistently carried out during the program.

如今，移动云计算成为规范和机器学习（ML）成为必要的。从智能手机到物联网（IoT）传感器的网络边缘设备数百万个设备需要自动协作以在云的帮助下执行复杂的任务。这些复杂的任务，例如对象检测和提问，可能涉及边缘设备的ML培训和分析。这项研究着重于针对下一代联合学习和具有大量边缘设备的下一代联合学习和分析的可扩展，个性化和沟通效率的机制的设计。尽管联合学习和联合分析都引起了最近的关注，但它们仍处于研究的新兴阶段，尤其是随着移动云计算系统中边缘设备的数量扩展。现有的大多数研究都集中在传统的客户服务器架构上，这远非可扩展。随着设备的扩展和网络流量负载的增加，设备的功能和数据分配变得越来越多，并且通信效率更加难以实现。进一步加剧了这一挑战，随着共享的全球模型经过培训，然后进行了推断，现有工作未能考虑在单个设备上进行个性化的需求，这是针对其本地收集的数据集定制的，并根据其自身的要求进行了量身定制。该研究计划的长期目标是通过实质上提高沟通效率和培训个性化模型的有效性并将其用于本地分析，以提高联合学习和分析的可扩展性。达到这样一个长期目标，这项研究提出了实现许多特定目标。首先，我们旨在在模型性能，隐私保护和沟通效率和设计新的当地差异隐私，模型量化和聚合机制之间实现平衡的权衡。其次，我们将研究针对每个边缘设备上个性化的个性化的微调元学习模型的新算法的设计。第三，我们将研究如何最好地分布在联合分析中的大型和复杂模型的推理工作量如何在多个边缘设备上分布，以满足其功率和能力约束。最后，为了证明这项拟议研究的现实世界可行性，我们将建立一个生产质量的开源框架，该框架涵盖了我们在该研究计划中提出的解决方案，以便可以在现实世界中的生产环境中进行测试。该计划将在深度学习理论和分布式计算系统的交集中培训高素质的人员（HQP），并提供当今需求量很高的理论和动手技能集合。在计划期间，将始终如一地强调公平，多样性和包容性（EDI）。