CAREER: Toward Hierarchical Game Theory and Hybrid Learning Framework for Safe, Efficient Large-scale Multi-agent Systems

该奖项的全部或部分资金来自《2021年美国救援计划法案》(Public Law 117-2)。广域电力管理系统、智能交通、5G/6G超密集网络等大规模多智能体系统(LS-MAS)正在迅速改变我们的世界。在从这些LS-MAS中获益之前，有必要开发一种可行的方法，即使在不确定和干扰下也能实时提高LS-MAS的效率和弹性。尽管现有的博弈论、人工智能(AI)和机器学习(ML)在多智能体系统优化方面取得了令人振奋的成就，但将这些理论和技术应用于LS-MAS仍然是一个空白，因为大量的智能体将导致优化和学习中难以处理的计算复杂性，即众所周知的维灾。本项目旨在研究新的理论和高效可行的AI/ML方法，既能在理论上平衡LS-MAS的最优性、效率和计算复杂度，又能在保证弹性的情况下实时学习LS-MAS的最优解。这项研究还得到了研究和教育计划的整合，包括联合国研究中心和PVAMU(一个著名的HBCU)之间的双向教育/研究管道。通过一年一度的夏令营和研究生联合研究计划，北卡罗来纳大学和PVAMU可以交换学生和教师，特别是来自代表性不足群体的学生和教师，以增加多样性。通过产学互动，该项目还计划将成果转化为实践，并促进内华达州(EPSCoR州)的经济。该项目的目标是促进博弈论的基础知识和数据使能学习的科学方法，以提高大规模多智能体系统(LS-MAS)的弹性和效率。由于多智能体数量巨大，如何在计算复杂度和最优效率之间取得平衡是一项非常具有挑战性的工作。为了克服这一挑战，本项目将提供几个新的贡献，包括i)新的分层博弈理论(HGT)，它可以在保持LS-MAS最优效率的同时平衡计算复杂性，ii)新型的基于倒向随机微分方程的行动者-批评者强化学习来解决基于HGT的高维LS-MAS优化问题，以及iii)一个性能质量驱动的可靠、高效、安全的混合强化学习框架，它可以在学习效率和计算复杂性与安全性保证之间取得平衡，进一步为即使在恶劣环境中存在不确定性的情况下基于实时学习的LS-MAS优化铺平道路。该项目将引领机器学习、最优控制和实时LS-MAS优化的博弈论的新方向，并为各种新兴的LS-MAS做出贡献，例如智能交通、广域电力管理系统等，这些都是国家优先考虑的。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Large scale multi-agent systems (LS-MAS), such as wide area power management systems, smart transportation, ultra-dense network in 5G/6G, and so on, are transforming our world rapidly. Before harvesting the benefits from those LS-MAS, it is necessary to develop a feasible methodology that can enhance the efficiency and resiliency of LS-MAS in real-time even under uncertainties and disturbances. Although existing game theory, artificial intelligence (AI), and machine learning (ML) achievement in multi-agent systems optimization are exciting, there is still a gap for applying those theories and techniques to LS-MAS since a large number of agents will cause the intractable computational complexity in both optimization and learning, well-known as “curse-of-dimensionality”. This project aims to investigate the new theory along with efficient and feasible AI/ML approaches that cannot only balance the LS-MAS optimality efficiency and computational complexity theoretically but also learn the LS-MAS optimal solution in real-time with resilience guaranteed. The research is complemented by the integration of research and education plan including a two-way education/research pipeline between UNR and PVAMU (a renowned HBCU). Through the annual summer camp and graduate student joint-research program, UNR and PVAMU can exchange students and faculties especially from underrepresented groups to increase diversity. Through industrial-university interaction, this project also plans to translate outcomes to practice and boost Nevada’s (EPSCoR state) economy. The goal of this project is to advance foundational knowledge of game theory and scientific methodologies of data-enabled learning for enhancing the resiliency and efficiency in large scale multi-agent systems (LS-MAS). Due to ultra large number of agents, it is very challenging to balance the computational complexity and optimal efficiency in LS-MAS. To overcome this challenge, this project will provide several novel contributions, including i) A novel hierarchical game theory (HGT) that can maintain the LS-MAS optimal efficiency while simultaneously balancing computational complexity, ii) A new type of backward stochastic differential equation based actor-critic reinforcement learning to solve the high-dimensional HGT-based LS-MAS optimization problem, and iii) A quality-of-performance driven reliable, efficient, safe hybrid reinforcement learning framework that can balance learning efficiency and computational complexity with safety guaranteed and further pave the way to real-time learning-based LS-MAS optimization even with uncertainties from harsh environments. This project will lead a new direction in machine learning, optimal control, and game theory in real-time LS-MAS optimization, and also contribute to a variety of emerging LS-MAS, e.g. smart transportation, wide area power management systems, etc., which are of national priority.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.