ITR: A Formal Study of Coordination and Control of Collaborative Multi-Agent Systems Using Decentralized MDPs

本项目通过将问题形式化为分散的马尔可夫过程，为多智能体系统的规划和控制开发了一个决策理论框架。它适用于信息收集、分布式感知、多个机器人的协调以及复杂的人类组织的运作等需要多个协作代理执行决策的广泛应用领域。虽然在使用mdp计划和控制单一agent方面取得了实质性进展，但对多agent系统缺乏类似的正式处理。现有技术倾向于避免一个中心问题：代理通常具有关于整个系统的不同信息，并且它们不能始终共享所有这些信息。共享信息的成本必须考虑到整个决策过程中。本文研究了三种通信方法，基于(1)通信量的成本/收益分析，(2)在策略空间中搜索，以及(3)将更容易处理的集中式策略转换为分散策略。由此产生的技术在几个实际应用的背景下进行评估。这项研究有助于更好地理解现有的启发式协调方法的优势和局限性，更重要的是，它包括基于更正式基础的新方法。

This project develops a decision-theoretic framework for planning and control of multi-agent systems by formalizing the problem as decentralized Markov process. It applies to a wide range of application domains in which decision-making must be performed by multiple collaborating agents such as information gathering, distributed sensing, coordination of multiple robots, as well as the operation of complex human organizations. While substantial progress has been made in planning and control of single agents using MDPs, a similar formal treatment of multi-agent systems has been lacking. Existing techniques tend to avoid a central issue: agents typically have different information about the overall system and they cannot share all this information all the time. Sharing information has a cost that must be factored into the overall decision process. Three approaches to communication are studied based on (1) a cost/benefit analysis of the amount of communication, (2) search in policy space, and (3) transformations of the more tractable centralized policies into decentralized policies. The resulting techniques are evaluated in the context of several realistic applications. This research facilitates a better understanding of the strengths and limitations of existing heuristic approaches to coordination and, more importantly, it includes new approaches based on more formal underpinnings.