RI: Small: Secure, Private, and Resource-Constrained Approaches to Federated Machine Learning

在一个越来越受数据驱动的机器学习（ML）影响的世界里，一个新出现的挑战是，数据通常以分布式的方式收集和存储——跨多个数据中心或设备。另一方面，出于安全和隐私方面的考虑，数据所有者之间的信任程度往往很低。为此，联邦ML支持使用分布式数据的ML，同时避免将私有数据从分布式设备传输到中央数据中心。为了实现机器学习民主化的目标，该项目将设计和实现新技术，使联邦机器学习安全和私有。特别令人感兴趣的是在计算能力或通信带宽有限的设备上启用联合ML的新系统设计，例如智能手机、智能健康监视器和智能手表等。这个项目的思想、软件和结果将直接影响工业和现实世界的应用。该项目将包括联邦机器学习的课程开发，并计划让来自代表性不足群体的研究生参与其中。该项目为联邦机器学习（ML）研究创造了一个变革性的新方向，通过在不受信任或薄弱的设备上、跨组织和希望维护数据隐私的用户上启用ML。该项目将包括理论基础、系统设计、实现和与流行ML软件集成的新工作。具体而言，该项目解决了联邦机器学习中的三个挑战。第一个挑战是容错机器学习算法，即当工人以任意恶意方式（称为拜占庭故障）行为时执行机器学习的新技术——特别是，该项目将表明，通过利用ML中的自然噪声容忍度，可以容忍比传统分布式计算文献所指出的更多的拜占庭工人。第二个挑战是开发保护隐私的机器学习算法，该算法引入来自工作人员的噪声，以保护参与者拥有的数据的隐私，同时在全球层面上导致正确和快速的机器学习。第三个挑战是通过采用新技术来研究资源受限的机器学习调度，以允许大型神经网络模型在具有内存约束的多个设备上运行。除了为这些方向开发算法和理论框架外，该项目还将构建和发布开放软件。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In a world increasingly shaped by data-driven machine learning (ML), one of the emerging challenges is that data are often collected and stored in a distributed manner -- across multiple datacenters or devices. On the other hand, due to security and privacy concerns, there are often low levels of trust between the data owners. To this end, federated ML enables ML with distributed data, while avoiding the transfer of private data from distributed devices to a central datacenter. Towards the goal of democratizing ML, this project will design and implement new techniques to make federated ML secure and private. Of particular interest are new system designs that enable federated ML on devices with limited computational power or communication bandwidth e.g., smartphones, smart health monitors, and smartwatches, among others. The ideas, software, and results of this project will directly impact industry and real-world applications. This project will include curriculum development for federated ML and plans to involve participation by graduate students from underrepresented groups. This project creates a transformative new direction for federated machine learning (ML) research, by enabling ML on devices that are untrusted or weak, and across organizations and for users who would like to maintain the privacy of their data. This project will include new work on theoretical foundations, systems design, implementation, and integration with popular ML software. Concretely, this project tackles three challenges in federated ML. The first challenge is fault-tolerant ML algorithms, i.e., new techniques to perform ML when workers act in arbitrarily malicious manners (called Byzantine failures) -- in particular, this project will show that by leveraging natural noise-tolerance in ML, it is possible to tolerate significantly more Byzantine workers than indicated by the traditional distributed computing literature. The second challenge is to develop privacy-preserving ML algorithms which introduce noise from workers to preserve the privacy of data owned by participants while leading to correct and fast ML at the global level. The third challenge is to investigate resource-constrained ML scheduling by including new techniques to allow large neural network models to run across multiple devices which have memory constraints. In addition to developing the algorithmic and theoretical frameworks for these directions, this project will also build and release open software.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.