A Multi-Rate Feedback Control Framework for Design and Analyzing of Decentralized and Federated Learning

分布式系统在实现我们现代生活所需的可扩展处理和实时智能方面前景广阔，彻底改变了我们与技术的交互方式，使我们能够轻松解决复杂问题。然而，尽管在分布式算法和系统中进行了广泛的研究，但在其合成和应用中仍然存在一些挑战。特别是，目前的算法设计过程是不可扩展的。实际上，需要设计一种新的算法，并针对每个应用场景和需求集开发相应的分析。有一个迫切需要统一的分布式算法的各种子类，以提供见解和简化的设计和分析。在这个项目中开发的框架将有利于机器学习以外的广泛应用，如控制理论和信号处理。建议的活动也提供了丰富的机会，让本科生参与跨学科的研究和K-12的推广activities.This建议倡导一个通用的“模型”的分布式算法，利用理论和技术，从随机多速率反馈控制系统。这样的通用模型可以将分布式算法的重要特征，如保证差分隐私，压缩通信或偶尔通信，抽象成易于处理的模块。在这些抽象模型的基础上，我们设计了一个具有上级建模能力的框架，以包含大量的分布式算法。该框架不仅可以用于分析整个算法类，更重要的是，可以将不同应用领域的功能轻松整合，从而简化新算法的设计。该奖项体现了NSF的法定使命，通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Distributed systems hold great promise to realize scalable processing and real-time intelligence required for our modern life, revolutionizing how we interact with technology and enabling us to easily tackle complex problems. However, despite extensive research in distributed algorithms and systems, several challenges persist in their synthesis and application. In particular, the current algorithm design process is not scalable. Indeed one needs to design a new algorithm and develop the corresponding analysis specific to each application scenario and set of requirements. There has been an urgent need to unify various subclasses of distributed algorithms to provide insights and streamline the design and analysis. The framework developed in this project will benefit a wide range of applications beyond machine learning, such as control theory and signal processing. The proposed activities also offer rich opportunities for engaging undergraduate students in cross-disciplinary research and K-12 outreach activities.This proposal advocates a generic “model” of distributed algorithms by leveraging theory and techniques from stochastic multi-rate feedback control systems. Such a generic model can abstract important features of distributed algorithms, such as guaranteed differential privacy, compressed communication, or occasional communication, into tractable modules. Building upon these abstract models, we design a framework with superior modeling power to encompass a substantial class of distributed algorithms. Such a framework can be used to analyze the entire algorithm class, but more importantly, it helps streamline the design of new algorithms in the sense that features arising from different application domains can be easily integrated.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.