RINGS: Scalable and Resilient Networked Learning Systems

下一代学习系统支持医疗保健、能源、银行、增强/虚拟现实和汽车/机器人导航等应用，将是隐私驱动的、分布式的和大规模的，从而大大增加了网络拥塞和故障的风险。一个典型的场景是大量的实体（客户端，组织，社区）参与学习任务;这些实体中的每一个都拥有，可以访问，或生成一个整体数据集的一部分，这将是不切实际的或不可取的收集在一个中央位置。虽然实体从协作中受益，特别是在本地数据量相对较小并且因此实体自己学习模型的能力有限的情况下，但是这种协作需要主要的网络资源。这个研究项目的中心是开发新的，以及扩展传统的，弹性和可扩展的网络学习系统的设计工程原理。为此，该项目研究了跨越理论基础，网络架构，应用程序和协议设计的发展的相互关联的主题。更广泛的影响提供了教育的进步，提高多样性，参与社区和行业，并传播结果给更广泛的public.The项目将首先集中在联邦学习（FL）框架，并在主题1，研究如何实现弹性的不确定性在FL系统经历间歇性的客户端可用性和随时间变化的网络容量。这导致了一种新的方法，FL有效地“学习如何学习”在一个不确定/资源受限的环境。主题2解决了大规模FL中遇到的可扩展性挑战，依靠集群的“可交换”客户端。这通过利用集群人口的基于多播的估计/跟踪，结合集群中客户端的概率调度，从以客户端为中心的系统管理转变为以集群为中心的高效系统管理。这为可扩展性和弹性以及潜在的隐私增强提供了新的途径。主题3建立在率失真理论和可伸缩视频编码的思想基础上，并探讨了使用可伸缩/分层（学习）模型压缩作为自适应感知FL的基础。这里的一个关键思想是认识到积极的压缩会导致更快的交付，这促使人们寻找一个折衷的最佳点，FL执行更多的更新，但模型较差（噪声较大）。这项研究证实了从信息，学习和学习理论的想法对实现弹性和适应性的研究协同作用。该项目还追求覆盖数据聚合网络的设计和使用，该网络通过网络内更新聚合和相关数据压缩来利用模型更新的聚合特性。这可以被视为内容交付网络覆盖的双重，内容交付网络覆盖是管理当前网络基础设施中的成本和性能的核心元素。主题4认识到，在FL应用的基础是客户的参与，从而使重点客户和有限资源的管理在不确定的环境中的联合激励。总体而言，拟议的研究重点是新形式的网络智能和适应性，旨在通过设备到边缘到云的连续统一体来解决可扩展性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Next-generation learning systems enabling applications ranging from healthcare, energy, banking, augmented/virtual reality and car/robot navigation, will be privacy-driven, distributed and large-scale, resulting in substantially increased exposure to network congestion and failures. A typical scenario is one where a substantial number of entities (clients, organizations, communities thereof) participate in a learning task; each of these entities owns, has access to, or generates part of an overall data set which would be impractical or undesirable to gather at a central location. While the entities benefit from a collaboration, especially in settings where the amount of local data is relatively small and thus the ability of an entity to learn a model on its own is limited, such a collaboration requires major network resources. This research project centers on developing new, as well as expanding traditional, engineering principles for the design of resilient and scalable networked learning systems. To this end, the project investigates interrelated themes spanning the development of theoretical underpinnings, network architecture, applications and protocol design. The broader impacts offer advances in education, enhancing diversity, engaging the community and industry, and disseminating results to a wider public.The project will initially focus on Federated Learning (FL) frameworks and, in Theme 1, study how to achieve resilience to uncertainty in FL systems experiencing intermittent client availability and time-varying network capacity. This leads to a novel approach to FL which effectively "learns how to learn" in an uncertain/resource-constrained environment. Theme 2 addresses scalability challenges encountered in large-scale FL by relying on clustering of "exchangeable" clients. This moves from client-centric to an efficient cluster-centric system management by leveraging multicast-based estimation/tracking of cluster populations, combined with probabilistic scheduling of clients in the clusters. This offers new avenues to scalability and resiliency as well as potential privacy enhancements. Theme 3 builds on ideas from rate-distortion theory and scalable video coding, and explores the use of scalable/layered (learned) model compression as a basis for adaptive congestion-aware FL. A key idea here is recognizing that aggressive compression leads to faster delivery, which motivates the search for a tradeoff sweet spot where FL performs more updates but with poorer (noisier) models. This research exemplifies research synergies of ideas from information, queueing and learning theory towards achieving resilience and adaptability. The project also pursues the design and use of overlay Data Aggregation Networks which exploit the aggregative character of model updates via in-network update aggregation and associated data compression. This can be viewed as the dual of Content Delivery Network overlays which are a core element managing the cost and performance in current network infrastructure. Theme 4 recognizes that at the base of FL applications is client participation and thus brings into focus the joint incentivization of clients and management of limited resources in uncertain environments. Overall, the proposed research focuses on new forms of network intelligence and adaptability, aiming to address scalability through device-to-edge-to-cloud continuum.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.