CAREER: Toward Hierarchical Game Theory and Hybrid Learning Framework for Safe, Efficient Large-scale Multi-agent Systems

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。大规模多智能体系统（LS-MAS），如广域电源管理系统，智能交通，5G/6 G超密集网络等，正在迅速改变我们的世界。在收获这些LS-MAS的好处之前，有必要开发一种可行的方法，即使在不确定性和干扰下，也可以实时提高LS-MAS的效率和弹性。虽然现有的博弈论，人工智能（AI）和机器学习（ML）在多智能体系统优化方面的成就令人兴奋，但将这些理论和技术应用于LS-MAS仍然存在差距，因为大量的智能体将导致优化和学习中难以解决的计算复杂性，即众所周知的“维数灾难”。该项目旨在研究新的理论沿着高效可行的AI/ML方法，这些方法不仅可以在理论上平衡LS-MAS最优效率和计算复杂度，而且还可以实时学习LS-MAS最优解，并保证弹性。该研究是由研究和教育计划的整合，包括UNR和PVAMU（著名的HBCU）之间的双向教育/研究管道的补充。通过年度夏令营和研究生联合研究计划，UNR和PVAMU可以交换学生和教师，特别是来自代表性不足的群体，以增加多样性。通过产业与大学的互动，该项目还计划将成果转化为实践，并促进内华达州（EPSCoR州）的经济。 该项目的目标是推进博弈论的基础知识和数据支持学习的科学方法，以提高大规模多智能体系统（LS-MAS）的弹性和效率。由于LS-MAS中存在超大数量的智能体，因此在计算复杂度和最优效率之间取得平衡是非常具有挑战性的。为了克服这一挑战，本项目将提供几个新的贡献，包括i）一种新的分层博弈论（HGT），可以保持LS-MAS的最佳效率，同时平衡计算复杂性，ii）一种新型的基于倒向随机微分方程的行动者-评论家强化学习来解决高维HGT的LS-MAS优化问题，以及iii）性能质量驱动的可靠、高效、安全的混合强化学习框架，可以在安全性得到保证的情况下平衡学习效率和计算复杂性，并进一步为基于实时学习的LS-MAS优化铺平道路，即使在恶劣环境中也存在不确定性。该项目将引领机器学习、最优控制和博弈论在实时LS-MAS优化中的新方向，并为各种新兴的LS-MAS做出贡献，例如智能交通、广域电力管理系统等，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Large scale multi-agent systems (LS-MAS), such as wide area power management systems, smart transportation, ultra-dense network in 5G/6G, and so on, are transforming our world rapidly. Before harvesting the benefits from those LS-MAS, it is necessary to develop a feasible methodology that can enhance the efficiency and resiliency of LS-MAS in real-time even under uncertainties and disturbances. Although existing game theory, artificial intelligence (AI), and machine learning (ML) achievement in multi-agent systems optimization are exciting, there is still a gap for applying those theories and techniques to LS-MAS since a large number of agents will cause the intractable computational complexity in both optimization and learning, well-known as “curse-of-dimensionality”. This project aims to investigate the new theory along with efficient and feasible AI/ML approaches that cannot only balance the LS-MAS optimality efficiency and computational complexity theoretically but also learn the LS-MAS optimal solution in real-time with resilience guaranteed. The research is complemented by the integration of research and education plan including a two-way education/research pipeline between UNR and PVAMU (a renowned HBCU). Through the annual summer camp and graduate student joint-research program, UNR and PVAMU can exchange students and faculties especially from underrepresented groups to increase diversity. Through industrial-university interaction, this project also plans to translate outcomes to practice and boost Nevada’s (EPSCoR state) economy. The goal of this project is to advance foundational knowledge of game theory and scientific methodologies of data-enabled learning for enhancing the resiliency and efficiency in large scale multi-agent systems (LS-MAS). Due to ultra large number of agents, it is very challenging to balance the computational complexity and optimal efficiency in LS-MAS. To overcome this challenge, this project will provide several novel contributions, including i) A novel hierarchical game theory (HGT) that can maintain the LS-MAS optimal efficiency while simultaneously balancing computational complexity, ii) A new type of backward stochastic differential equation based actor-critic reinforcement learning to solve the high-dimensional HGT-based LS-MAS optimization problem, and iii) A quality-of-performance driven reliable, efficient, safe hybrid reinforcement learning framework that can balance learning efficiency and computational complexity with safety guaranteed and further pave the way to real-time learning-based LS-MAS optimization even with uncertainties from harsh environments. This project will lead a new direction in machine learning, optimal control, and game theory in real-time LS-MAS optimization, and also contribute to a variety of emerging LS-MAS, e.g. smart transportation, wide area power management systems, etc., which are of national priority.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.