CAREER: Federated Learning: Statistical Optimality and Provable Security

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。机器学习和数据科学的快速发展迫使组织和个人越来越多地依赖数据来解决推理和决策问题。为了缓解数据所有者对隐私的担忧，研究人员和实践者一直在倡导一种新的学习范式-联邦学习。在此框架下，中央学习器通过与分布式用户通信并将训练数据存储在用户本地来训练模型。虽然在不影响数据隐私的情况下为训练机器学习模型开辟了新的机会，但由于分布式用户的异构性和不可靠性，联邦学习在维护统计效率和安全性方面面临着重大挑战。该项目的成功完成为高效和安全的联邦学习提供了关键的支持技术，并加速了其在安全和安全关键系统中的应用，如自动驾驶汽车和个性化医疗。拟议的教育活动包括专门针对代表性不足的群体成员的研究生和本科生的教学和辅导，以及旨在提高公众隐私和安全意识的社区外联活动。这项研究开发了一个跨学科的计划，以研究联邦学习的基本和算法方面，从统计效率到安全性和隐私性。广泛采用的算法的统计效率进行了分析超出他们的失败达到稳定点。新算法是基于元学习和集群开发的，即使在存在模型和数据异质性的情况下，也被证明是统计上最优的。此外，本研究还对拜占庭攻击下的分散学习进行了全面的研究。借用鲁棒统计的见解，拜占庭弹性梯度下降算法的指数收敛到最佳的错误率。最后，为了保护学习者的隐私免受窃听攻击，研究者的目的是设计最佳的私人学习策略，从信息论和学习者和对手之间的对偶理论的创新思想。作为对理论研究的补充，新的学习算法可作为联邦学习系统和真实数据应用程序的计算包提供。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Rapid developments in machine learning and data science have compelled organizations and individuals to rely more and more on data to solve inference and decision problems. To ease the privacy concerns of data owners, researchers and practitioners have been advocating a new learning paradigm – federated learning. Under this framework, the central learner trains a model by communicating with distributed users and keeping the training data stored locally at the users. While opening up a world of new opportunities for training machine-learning models without compromising data privacy, federated learning faces significant challenges in maintaining statistical efficiency and security due to the heterogeneity and unreliability of the distributed users. Successful completion of the project provides key enabling technologies for efficient and secure federated learning and accelerates its adoption in security- and safety-critical systems such as self-driving cars and personalized medicine. The proposed education activities include teaching and mentoring graduate and undergraduate students targeted specifically at members of under-represented groups and community outreach aiming at raising public privacy and security awareness. This research develops an interdisciplinary program to investigate the fundamental and algorithmic aspects of federated learning ranging from statistical efficiency to security and privacy. The statistical efficiency of the widely-adopted algorithms is analyzed beyond their failures of reaching stationary points. New algorithms are developed based on meta-learning and clustering, and shown to be statistically optimal even in the presence of model and data heterogeneity. Moreover, this research conducts a comprehensive study of decentralized learning under Byzantine attacks. By borrowing insights from robust statistics, byzantine-resilient gradient descent algorithms with exponential convergence to the optimal error rates are devised. Finally, to protect the learner's privacy against eavesdropping attacks, the investigator aims to design optimal private learning strategies by innovating ideas from information theory and duality theory between the learner and adversary. Complementing the theoretical investigation, the new learning algorithms are made available as computational packages for the federated learning systems and real-data applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.