Collaborative Research: CNS Core: Medium: Systems Support for Federated Learning

协作研究：CNS 核心：中：联邦学习的系统支持

• 批准号: 2106184
• 负责人: Mosharaf Chowdhury
• 金额: $ 80万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106184&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Medium

Traditional approaches toward applying machine learning techniques to end-user data often require copying all data to the cloud. This is not only expensive but faces data privacy risks as well. By analyzing data on the device where it is generated, federated learning aims to mitigate both cost and privacy concerns of centralized machine learning without sacrificing its benefits. This collaborative project brings together investigators from two institutions to develop building blocks for practical federated learning by addressing challenges arising from the diversity of user devices and the heterogeneity of data distributions in those devices. The project takes a three-pronged approach: (1) enable performance improvements for machine learning developers (e.g., judicious participant selection instead of randomly selecting participants); (2) provide efficiency improvements for service providers (e.g., redundancy elimination for data transfers); (3) enable end-users to control their data privacy (e.g., akin to app permissions in Android) without sacrificing device usability. Two core principles underpin these solutions: multi-tenancy both in the cloud and on individual devices; and maintaining theoretical correctness, convergence characteristics, and privacy/security guarantees of federated learning algorithms. Widespread adoption of practical federated learning can fundamentally change how we gather insights from end-user data and how users value data privacy, because users may not have to sacrifice privacy for convenience in many cases. This, in turn, can force large corporations to rethink their data collection and usage practices, and influence policy makers to consider stricter privacy regulations. All software from this project will be open source. Through outreach and new educational materials, this project will pioneer the training of privacy-aware systems builders.This collaborative project will produce software artifacts, experimental harnesses, benchmarks, and results of running those benchmarks and artifacts. These materials will be available for public use under permissive open-source licenses at multiple locations, including https://github.com/symbioticlab. They will be retained for at least three years after the completion of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

将机器学习技术应用于最终用户数据的传统方法通常需要将所有数据复制到云中。这不仅昂贵，而且还面临数据隐私风险。通过分析生成数据的设备上的数据，联邦学习旨在减轻集中式机器学习的成本和隐私问题，而不会牺牲其优势。这个合作项目汇集了来自两个机构的研究人员，通过解决用户设备的多样性和这些设备中数据分布的异质性所带来的挑战，为实际的联邦学习开发构建模块。该项目采取三管齐下的方法：（1）为机器学习开发人员提供性能改进（例如，明智的参与者选择而不是随机选择参与者）;（2）为服务提供商提供效率改进（例如，数据传输的冗余消除）;（3）使终端用户能够控制他们的数据隐私（例如，类似于Android中的应用程序权限），而不会牺牲设备的可用性。两个核心原则支撑着这些解决方案：在云中和单个设备上的多租户;以及保持理论正确性，收敛特性和联邦学习算法的隐私/安全保证。广泛采用实用的联邦学习可以从根本上改变我们从最终用户数据中收集见解的方式以及用户如何重视数据隐私，因为在许多情况下，用户可能不必为了方便而牺牲隐私。这反过来又会迫使大公司重新考虑其数据收集和使用做法，并影响政策制定者考虑更严格的隐私法规。该项目的所有软件都将是开源的。通过推广和新的教育材料，该项目将率先培训隐私感知系统的构建者。这个合作项目将产生软件工件，实验工具，基准测试以及运行这些基准测试和工件的结果。这些材料将在多个地点（包括https://github.com/symbioticlab）的许可开源许可下供公众使用。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

ModelKeeper: Accelerating DNN Training via Automated Training Warmup

• 发表时间: 2023
• 期刊: Journal of the American Chemical Society
• 影响因子: 15
• 作者: Fan Lai;Yinwei Dai;H. Madhyastha;Mosharaf Chowdhury

AdaEmbed: Adaptive Embedding for Large-Scale Recommendation Models

AdaEmbed：大规模推荐模型的自适应嵌入

• 发表时间: 2023
• 期刊: USENIX OSDI
• 作者: Lai, Fan;Zhang, Wei;Liu, Rui;Tsai, William;Wei, Xiaohan;Hu, Yuxi;Devkota, Sabin;Huang, Jianyu;Park, Jongsoo;Liu, Xing
• 通讯作者: Liu, Xing

Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates

• DOI: 10.1145/3600006.3613152
• 发表时间: 2023-09
• 期刊: Proceedings of the 29th Symposium on Operating Systems Principles
• 作者: Insu Jang;Zhenning Yang;Zhen Zhang;Xin Jin;Mosharaf Chowdhury

FedScale: Benchmarking Model and System Performance of Federated Learning at Scale

• 发表时间: 2021-05
• 作者: Fan Lai;Yinwei Dai;Sanjay Sri Vallabh Singapuram;Jiachen Liu;Xiangfeng Zhu;H. Madhyastha;Mosharaf Chowdhury

Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing

• DOI: 10.1145/3552326.3587451
• 发表时间: 2022-01
• 期刊: Proceedings of the Eighteenth European Conference on Computer Systems
• 作者: Yiding Wang;D. Sun;Kai Chen-;Fan Lai;Mosharaf Chowdhury