PM-FSM: Policies Modulating Finite State Machine for Robust Quadrupedal Locomotion
PM-FSM: Policies Modulating Finite State Machine for Robust Quadrupedal Locomotion
复制标题
DOI：
10.1109/iros47612.2022.9982259
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Ren Liu;Nitish Sontakke;Sehoon Ha
中科院分区：
文献类型：
--
作者：
Ren Liu;Nitish Sontakke;Sehoon Ha
深度强化学习（Deep RL）已经成为开发腿式机器人控制器的有效工具。然而，香草深度RL通常需要大量的训练样本，并且对于实现健壮的行为不可行。相反，研究人员通过整合人类专家的知识，研究了一种新的策略架构，例如策略调制轨迹生成器（PMTG）。该架构通过结合参数轨迹生成器（TG）和反馈策略网络来构建循环控制回路，以实现更鲁棒的行为。在这项工作中，我们提出了政策调制有限状态机（PM-FSM），通过替换TG与接触感知有限状态机（FSM），它提供了更灵活的控制每个腿。本发明为策略提供了接触事件的明确概念，以协商意外的扰动。我们证明了所提出的架构可以实现更强大的行为在各种情况下，如具有挑战性的地形或外部扰动，在模拟和真实的机器人。
Deep reinforcement learning (deep RL) has emerged as an effective tool for developing controllers for legged robots. However, vanilla deep RL often requires a tremendous amount of training samples and is not feasible for achieving robust behaviors. Instead, researchers have investigated a novel policy architecture by incorporating human experts' knowledge, such as Policies Modulating Trajectory Generators (PMTG). This architecture builds a recurrent control loop by combining a parametric trajectory generator (TG) and a feedback policy network to achieve more robust behaviors. In this work, we propose Policies Modulating Finite State Machine (PM-FSM) by replacing TGs with contact-aware finite state machines (FSM), which offers more flexible control of each leg. This invention offers an explicit notion of contact events to the policy to negotiate unexpected perturbations. We demonstrated that the proposed architecture could achieve more robust behaviors in various scenarios, such as challenging terrains or external perturbations, on both simulated and real robots.