CAREER: Strengthening the Theoretical Foundations of Federated Learning: Utilizing Underlying Data Statistics in Mitigating Heterogeneity and Client Faults

从改进的机器学习中受益的实际应用包括广泛的行业和领域，如医疗保健，自动驾驶汽车，自然语言处理以及制造业和工业。近年来，由于对实时数据处理、低延迟和隐私问题的需求日益增加，分布式机器学习获得了巨大的发展势头。边缘设备的快速发展拓宽了分布式机器学习的适用性，但也带来了重大挑战，需要重新审视联邦学习的基本原理和算法设计。该项目的研究目标是巩固分布式机器学习的理论基础，丰富分布式机器学习的算法工具箱，重点是增强其对各种数据异构性，系统缺陷（或故障）和外部攻击的弹性。该项目的教育目标是在下一代机器学习从业者和研究人员中推广有原则的数学思维对于解决机器学习中的现实问题的重要性，重点是开发针对女性和代表性不足的少数群体的项目。 联邦学习是一种快速发展的分布式机器学习方法，它可以促进全局模型训练，而无需共享原始本地数据。大多数现有的联邦学习理论分析都是从优化的角度出发的，但数据集的底层统计结构大多被忽视了。这往往导致悲观的理论预测和经验成功之间的不一致。此外，最近的工作表明，联邦学习分析中经常采用的有界梯度相异度条件可能对实际应用过于悲观。在我们初步成功的推动下，并在大量先前工作的支持下，该提案旨在加强联邦学习的理论基础，并通过利用联邦数据集的底层结构和设计新算法，增强其对各种数据异构性和系统故障的弹性。为了实现这一目标，我们将采用和创新的工具，从统计学习，分布式计算，高维概率和优化。这个奖项反映了NSF的法定使命，并已被认为是值得的支持，通过评估使用基金会的智力价值和更广泛的影响审查标准。

Real-world applications that would benefit from improved machine learning encompass a wide range of industries and domains such as healthcare, autonomous vehicles, natural language processing, and manufacturing and industry. Distributed machine learning has gained significant momentum in recent years due to the increasing need for real-time data processing, low latency, and privacy concerns. The rapid development of edge devices broadens the applicability of distributed machine learning yet brings nontrivial challenges that call for revisiting the fundamental principles and algorithm designs for federated learning. The research goal of this project is to consolidate the theoretical foundations and to enrich the algorithmic toolbox of distributed machine learning with a focus on enhancing its resilience against a wide range of data heterogeneity, system imperfection (or faults), and external attacks. The educational objective of this project is to promote the importance of principled mathematical thinking for solving real-world problems in machine learning among the next generation of machine learning practitioners and researchers, with a focus on developing programs that target women and underrepresented minority groups. Federated learning is a rapidly evolving distributed machine learning approach that facilitates global model training without the necessity of sharing raw local data. Most existing theoretical analysis of federated learning is derived from an optimization perspective but the underlying statistical structure of the dataset is mostly overlooked. This often leads to misalignment between the pessimistic theoretical predictions and empirical success. In addition, recent work suggests that the bounded gradient dissimilarity conditions, which are frequently adopted in federated learning analysis, may be too pessimistic for practical applications. Motivated by our preliminary successes and backed by extensive prior work, this proposal aims to strengthen the theoretical foundations of federated learning and to enhance its resilience against a wide range of data heterogeneity and system failures, by leveraging the underlying structures of the federated datasets and by designing new algorithms. Towards this goal we will employ and innovate tools from statistical learning, distributed computing, high-dimensional probabilities, and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.