Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions
Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions
复制标题
DOI：
10.1177/0278364917692864
复制
发表时间：
2017-02
期刊：
影响因子：
--
通讯作者：
Shayegan Omidshafiei;Ali-akbar Agha-mohammadi;Chris Amato;Shih‐Yuan Liu;J. How;J. Vian
中科院分区：
文献类型：
--
作者：
Shayegan Omidshafiei;Ali-akbar Agha-mohammadi;Chris Amato;Shih‐Yuan Liu;J. How;J. Vian
本文的研究重点是在给定高阶域描述的情况下，解决具有部分可观察性的连续空间中的一般多机器人规划问题。分散部分可观察马尔可夫决策过程（deco - pomdp）是求解多机器人协调问题的通用模型。然而，对于大型问题，表示和解决dec - pomdp通常是棘手的。这项工作将Dec-POMDP模型扩展到分散的部分可观察半马尔可夫决策过程（Dec-POSMDP），以利用多机器人问题的高级表示，并促进大型离散和连续问题的可扩展解决方案。Dec-POSMDP公式使用由低级局部动作创建的任务宏动作，允许机器人进行异步决策，这在多机器人领域至关重要。这种从dec - pomdp到dec - posmdp的转换具有有限的自动生成的宏动作集，允许使用有效的离散空间搜索算法来解决它们。本文提出了求解dec - posmdp的算法，该算法可以在规划中加入闭环的信念空间宏动作，具有较好的可扩展性。自动构造这些宏操作以生成健壮的解决方案。在一个复杂的多机器人不确定包裹递送问题上对所提出的算法进行了评估，表明我们的方法可以自然地表示现实问题，并为大规模问题提供高质量的解决方案。
This work focuses on solving general multi-robot planning problems in continuous spaces with partial observability given a high-level domain description. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems. However, representing and solving Dec-POMDPs is often intractable for large problems. This work extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP) to take advantage of the high-level representations that are natural for multi-robot problems and to facilitate scalable solutions to large discrete and continuous problems. The Dec-POSMDP formulation uses task macro-actions created from lower-level local actions that allow for asynchronous decision-making by the robots, which is crucial in multi-robot domains. This transformation from Dec-POMDPs to Dec-POSMDPs with a finite set of automatically-generated macro-actions allows use of efficient discrete-space search algorithms to solve them. The paper presents algorithms for solving Dec-POSMDPs, which are more scalable than previous methods since they can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed algorithms are then evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent realistic problems and provide high-quality solutions for large-scale problems.