Navigating Robots in Dynamic Environment With Deep Reinforcement Learning
Navigating Robots in Dynamic Environment With Deep Reinforcement Learning
复制标题
DOI：
10.1109/tits.2022.3213604
复制
发表时间：
2022-12
影响因子：
8.5
通讯作者：
Zhiqian Zhou;Zhiwen Zeng;Lin Lang;Weijia Yao;Huimin Lu;Zhiqiang Zheng;Zongtan Zhou
中科院分区：
文献类型：
--
作者：
Zhiqian Zhou;Zhiwen Zeng;Lin Lang;Weijia Yao;Huimin Lu;Zhiqiang Zheng;Zongtan Zhou
在抗击新冠疫情的过程中，许多机器人取代人类员工执行各类存在感染风险的任务。在这些任务里，机器人在人群中导航这一根本性问题，即机器人人群导航问题，仍未得到解决且颇具挑战性。因此，我们提出了HGAT - DRL，一种基于异构图注意力网络（GAT）的深度强化学习算法。该算法在由四种类型节点构成的异构图中对人 - 机器人共存的受限环境进行编码。它还为机器人周围的物体构建了交互式的智能体层面表征，并将来自非完整运动模型的运动学动力学约束融入深度强化学习（DRL）框架。仿真结果表明，我们提出的算法成功率达到92%，比四种基线算法至少高出6%。此外，在Fetch机器人上进行的硬件实验证明了我们的算法能够成功且便捷地迁移到真实机器人上。
In the fight against COVID-19, many robots replace human employees in various tasks that involve a risk of infection. Among these tasks, the fundamental problem of navigating robots among crowds, named robot crowd navigation, remains open and challenging. Therefore, we propose HGAT-DRL, a heterogeneous GAT-based deep reinforcement learning algorithm. This algorithm encodes the constrained human-robot-coexisting environment in a heterogeneous graph consisting of four types of nodes. It also constructs an interactive agent-level representation for objects surrounding the robot, and incorporates the kinodynamic constraints from the non-holonomic motion model into the deep reinforcement learning (DRL) framework. Simulation results show that our proposed algorithm achieves a success rate of 92%, at least 6% higher than four baseline algorithms. Furthermore, the hardware experiment on a Fetch robot demonstrates our algorithm’s successful and convenient migration to real robots.