Learning Methods for Decentralized Control in Multi-Agent Systems

多智能体系统（MAS）有望在军事和民用领域得到越来越广泛的应用。智能体的分散控制和决策是多智能体系统多样化应用的基本驱动力。期望代理在不依赖于集中的命令结构的情况下采取行动并做出决定。代理之间的通信和协调可能必须在稀疏、断断续续、不可靠、低数据率和/或嘈杂的通信网络上进行，这就排除了集中信息和决策的可能性。一个关键的设计挑战是为一个代理团队找到有效的计算分散控制和决策策略的方法。各种不确定性——环境的不确定性、有噪声的观测、不可靠的通信以及系统模型的不确定性，使问题进一步复杂化。在这个项目中，我们的目标是为多智能体系统中的分散控制开发基于学习的方法。智力优势：本研究发展了以下几点：(1)基于学习的多智能体系统近最优分散控制策略计算的实用方法。（ii）用于控制系统模型未知的多智能体系统的在线分散学习算法。我们的目标是开发去中心化算法，逐步为这样的系统找到最优的去中心化策略，并以最有效的方式学习。所提出的研究将为基于学习的分散最优控制奠定基础，这将在新兴的多智能体系统应用中变得越来越重要。更广泛的影响：该研究将对多智能体系统、自主机器人系统和强化学习的科学产生重大影响。它将引入一种系统的、实用的、基于学习的方法来设计多智能体系统，这是文献中长期缺乏的。拟议研究的教育影响将包括：(i)为研究生提供随机控制、在线学习和优化方面的多学科培训；（ii）让本科生在夏季参与计算和实验室实验；（iii）努力在我们的项目中招募女性和代表性不足的少数民族学生；（iv）研究成果将纳入由主要研究人员讲授的强化学习、随机系统和分散控制课程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Multi-agent systems (MAS) are expected to become increasingly prevalent in military and civilian domains. Decentralized control and decision-making by agents is a fundamental driver of the diverse applications of multi-agent systems. Agents are expected to act and make decisions without relying on a centralized command structure. Communication and coordination among agents may have to be carried out over sparse, intermittent, unreliable, low data rate and/or noisy communication networks that preclude the possibility of centralized information and decision-making. A key design challenge is to find efficient ways of computing decentralized control and decision strategies for a team of agents. The problem is further compounded by various kinds of uncertainties - uncertainties about the environment, noisy observations, unreliable communication as well as uncertainties in the system model. In this project, we aim to develop learning-based methods for decentralized control in multi-agent systems. Intellectual merit: The research develops the following: (i) learning-based practical methods for computing near-optimal decentralized control policies for multi-agent systems with known system model. (ii) online decentralized learning algorithms for control of multi-agent systems with unknown system model. We aim to develop decentralized algorithms that asymptotically find the optimal decentralized policy for such systems and learn in the most efficient way possible. The proposed research will lay the foundations for Learning-based Decentralized Optimal Control, which is expected to become increasingly important for emerging multi-agent system applications. Broader Impact: The research will significantly impact the science of multi-agent systems, autonomous robotic systems, and reinforcement learning. It will introduce a systematic and practical learning-based approach to design of multi-agent systems that has long been lacking in the literature. The educational impact of the proposed research will include: (i) providing graduate students with a multi-disciplinary training in stochastic control, online learning and optimization, (ii) involvement of undergraduate students during summer to perform computational and lab experiments (iii) efforts to recruit female and under-represented minority students in our projects; (iv) The research results will be incorporated in classes on reinforcement learning, stochastic systems, and decentralized control taught by the principal investigators.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.