RI: Small: Secure, Private, and Resource-Constrained Approaches to Federated Machine Learning

在一个越来越多地由数据驱动的机器学习（ML）塑造的世界中，一个新出现的挑战是数据通常以分布式方式收集和存储-跨多个数据中心或设备。另一方面，由于安全和隐私问题，数据所有者之间的信任程度往往很低。为此，联邦ML支持ML与分布式数据，同时避免将私有数据从分布式设备传输到中央数据中心。为了实现ML民主化的目标，该项目将设计和实施新技术，使联合ML安全且私密。特别令人感兴趣的是新的系统设计，这些设计使联合ML能够在具有有限计算能力或通信带宽的设备上实现，例如，智能手机、智能健康监测器和智能手表等。该项目的想法、软件和结果将直接影响行业和实际应用。该项目将包括联邦ML的课程开发，并计划让代表性不足的群体的研究生参与。该项目为联合机器学习（ML）研究创造了一个变革性的新方向，通过在不受信任或薄弱的设备上，跨组织和希望维护其数据隐私的用户启用ML。该项目将包括理论基础，系统设计，实现以及与流行ML软件集成的新工作。具体来说，这个项目解决了联邦ML中的三个挑战。第一个挑战是容错ML算法，即，当工作者以任意恶意方式（称为拜占庭故障）行事时执行ML的新技术-特别是，该项目将表明，通过利用ML中的自然噪声容忍度，可以容忍比传统分布式计算文献所指出的更多的拜占庭工作者。第二个挑战是开发保护隐私的机器学习算法，该算法引入来自工作人员的噪音，以保护参与者拥有的数据的隐私，同时在全球层面上实现正确和快速的机器学习。第三个挑战是研究资源受限的ML调度，包括新技术，允许大型神经网络模型在具有内存限制的多个设备上运行。除了为这些方向开发算法和理论框架外，该项目还将构建和发布开放软件。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In a world increasingly shaped by data-driven machine learning (ML), one of the emerging challenges is that data are often collected and stored in a distributed manner -- across multiple datacenters or devices. On the other hand, due to security and privacy concerns, there are often low levels of trust between the data owners. To this end, federated ML enables ML with distributed data, while avoiding the transfer of private data from distributed devices to a central datacenter. Towards the goal of democratizing ML, this project will design and implement new techniques to make federated ML secure and private. Of particular interest are new system designs that enable federated ML on devices with limited computational power or communication bandwidth e.g., smartphones, smart health monitors, and smartwatches, among others. The ideas, software, and results of this project will directly impact industry and real-world applications. This project will include curriculum development for federated ML and plans to involve participation by graduate students from underrepresented groups. This project creates a transformative new direction for federated machine learning (ML) research, by enabling ML on devices that are untrusted or weak, and across organizations and for users who would like to maintain the privacy of their data. This project will include new work on theoretical foundations, systems design, implementation, and integration with popular ML software. Concretely, this project tackles three challenges in federated ML. The first challenge is fault-tolerant ML algorithms, i.e., new techniques to perform ML when workers act in arbitrarily malicious manners (called Byzantine failures) -- in particular, this project will show that by leveraging natural noise-tolerance in ML, it is possible to tolerate significantly more Byzantine workers than indicated by the traditional distributed computing literature. The second challenge is to develop privacy-preserving ML algorithms which introduce noise from workers to preserve the privacy of data owned by participants while leading to correct and fast ML at the global level. The third challenge is to investigate resource-constrained ML scheduling by including new techniques to allow large neural network models to run across multiple devices which have memory constraints. In addition to developing the algorithmic and theoretical frameworks for these directions, this project will also build and release open software.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.