Learning to Efficiently Plan in Flexible Distributed Organizations

学习在灵活的分布式组织中进行有效规划

• 批准号: EP/R001227/2
• 负责人: Frans Oliehoek
• 金额: $ 5.2万
• 依托单位: Delft University of Technology
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FR001227%2F2
• 关键词: Learning; Efficiently; Plan; Flexible; Distributed

Teams of robots are expected to revolutionise industry and other other parts of society. However, decision making in such so-called multiagent systems (MASs) under uncertainty is computationally very complex. The decentralized partially observable Markov decision process (Dec-POMDP) framework facilitates principled formulation of such decision making problems, but currently there are no scalable solution methods that provide guarantees on task performance. To simplify coordination in MASs, agent organisations assign an abstracted, easier problem to each agent. Typically only the most rigid organisations, which completely decouple the agents, have led to clear computational benefits. However, these come at the expense of task performance: full decoupling means that agents can no longer collaborate to divide the workload. This project will focus on flexible distributed organisations (FDOs) for Dec-POMDPs, which restrict considered interactions to spatially nearby agents without imposing full decoupling. Currently no scalable decision making methods with guarantees on task performance exist for FDOs: the main goal of the project is to develop such methods along with the theory that supports their formalisation. To accomplish this goal, it will investigate the use of deep learning techniques to learn representations of 'influence' in FDOs and use those representations to develop novel planning methods. If successful, this will provide the proof-of-concept that learned influence representations can enable principled decision making in large-scale MASs. This will be the basis for a larger research program investigating such influence representations for different forms of abstraction and will spark applied research that investigates deployment of the developed algorithms in real robotic teams.

机器人团队有望彻底改变工业和社会其他领域。然而，在这种所谓的多智能体系统（MAS）的不确定性下的决策是非常复杂的计算。分散的部分可观察马尔可夫决策过程（Dec-POMDP）框架有利于原则制定这样的决策问题，但目前还没有可扩展的解决方案的方法，提供保证的任务性能。为了简化MAS中的协调，代理组织分配给每个代理一个抽象的，更容易的问题。通常，只有最严格的组织，完全解耦的代理，导致明确的计算效益。然而，这些都是以牺牲任务性能为代价的：完全解耦意味着代理不能再协作来划分工作负载。该项目将集中在灵活的分布式组织（FDO）的Dec-POMDPs，它限制了考虑到空间附近的代理人的相互作用，而不施加完全解耦。目前没有可扩展的决策方法，保证任务的性能存在FDO：该项目的主要目标是开发这种方法沿着的理论，支持他们的形式化。为了实现这一目标，它将研究使用深度学习技术来学习FDO中的“影响力”表示，并使用这些表示来开发新的规划方法。如果成功，这将提供概念验证，即学习的影响表示可以在大规模MAS中实现原则性决策。这将是一个更大的研究计划的基础，调查这种影响表示为不同形式的抽象，并将引发应用研究，调查部署的开发算法在真实的机器人团队。

项目成果

Analysing Factorizations of Action-Value Networks for Cooperative Multi-Agent Reinforcement Learning

分析协作多智能体强化学习的行动价值网络分解

• DOI: 10.48550/arxiv.1902.07497
• 发表时间: 2019
• 期刊: arXiv e-prints
• 作者: Castellini Jacopo

Beyond Local Nash Equilibria for Adversarial Networks

超越对抗性网络的局部纳什均衡

• DOI: 10.48550/arxiv.1806.07268
• 发表时间: 2018
• 期刊: arXiv e-prints
• 作者: Oliehoek Frans A.

Bayesian Reinforcement Learning in Factored POMDPs

• 发表时间: 2018-11
• 作者: Sammie Katt;F. Oliehoek;Chris Amato

Decentralized MCTS via Learned Teammate Models

• DOI: 10.24963/ijcai.2020/12
• 发表时间: 2020-03
• 作者: A. Czechowski;F. Oliehoek

Interactive Learning and Decision Making: Foundations, Insights & Challenges

• DOI: 10.24963/ijcai.2018/813
• 发表时间: 2018-07
• 作者: F. Oliehoek