Federated Optimization over Bandwidth-Limited Heterogeneous Networks

带宽受限异构网络的联合优化

• 批准号: 2318441
• 负责人: Yuejie Chi
• 金额: $ 36万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318441&HistoricalAwards=false
• 关键词: Federated; Optimization; over; Bandwidth; Limited

Harnessing the power of data collected from a vast amount of geographically distributed and heterogeneous devices, in a manner without moving data around and violating privacy, has great potential in advancing science and technology and improving quality of life. Federated optimization lies at the heart of the practice realizing this vision, encompassing problems such as training large-scale machine learning or artificial intelligence models, delivering insightful data analytics, as well as facilitating decision making under uncertainty, all in distributed manners. There is a significant gap in the algorithmic foundation of federated optimization when interfacing with bandwidth-limited heterogeneous networks, such as internet-of-things, smart healthcare, and edge computing, to meet the unique challenges of taming heterogeneity, privacy, and uncertainty without sacrificing efficiency. This research project will also be tightly integrated with education and workforce developments, through offering new courses, mentoring students at all levels in research projects including underrepresented minorities and women, and disseminating the research outcomes at suitable conferences and workshops. The goal of the research program is to develop a federated optimization framework to learning and decision making by designing communication-efficient, computation-scalable, and privacy-preserving algorithms that converge provably over highly heterogeneous data and computing environments. Leveraging insights from machine learning, optimization theory, signal processing, and differential privacy, the research program offers an entirely new suite of theoretical and algorithmic tools to enable heterogeneity-embracing and privacy-preserving learning and decision making in federated environments under bandwidth constraints, unveiling fundamental trade-offs among computation, communication, privacy, and utility. The research program will gravitate around a semi-decentralized federated setting suitable to meet the diverse needs of bandwidth-limited heterogeneous networks, and focus on developing bandwidth-limited federated optimization algorithms that are efficient, resilient, and private with rigorous performance guarantees for a wide range of problems arising from machine learning, data analysis, and sequential decision making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

利用从大量地理分布和异构设备收集的数据的力量，在不移动数据和侵犯隐私的情况下，在推进科学技术和提高生活质量方面具有巨大潜力。联邦优化是实现这一愿景的实践的核心，包括训练大规模机器学习或人工智能模型、提供有洞察力的数据分析以及促进不确定性下的决策制定等问题，所有这些都以分布式方式进行。在与带宽有限的异构网络（如物联网、智能医疗保健和边缘计算）接口时，联邦优化的算法基础存在显著差距，无法在不牺牲效率的情况下应对控制异质性、隐私性和不确定性的独特挑战。该研究项目还将与教育和劳动力发展紧密结合，提供新课程，指导各级学生参与研究项目，包括代表性不足的少数民族和妇女，并在适当的会议和讲习班上传播研究成果。该研究计划的目标是通过设计通信高效、计算可扩展和隐私保护的算法，开发一个用于学习和决策的联邦优化框架，这些算法可证明在高度异构的数据和计算环境中收敛。利用来自机器学习、优化理论、信号处理和差分隐私的见解，该研究项目提供了一套全新的理论和算法工具，以实现带宽限制下联邦环境中包容异构和保护隐私的学习和决策，揭示计算、通信、隐私和效用之间的基本权衡。该研究计划将围绕半分散的联邦设置，适合满足带宽有限的异构网络的各种需求，并专注于开发带宽有限的联邦优化算法，该算法高效、有弹性、具有严格的性能保证，适用于机器学习、数据分析和顺序决策产生的各种问题。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compression

• DOI: 10.48550/arxiv.2310.19059
• 发表时间: 2023-10
• 作者: Sijin Chen;Zhize Li;Yuejie Chi