GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model
GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model
复制标题
DOI：
10.1007/978-3-031-21090-7_31
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Zhaoming Xie;Xingye Da;Buck Babich;Animesh Garg;M. V. D. Panne
中科院分区：
文献类型：
--
作者：
Zhaoming Xie;Xingye Da;Buck Babich;Animesh Garg;M. V. D. Panne
用于腿部运动的无模型强化学习(RL)通常依赖于一个物理模拟器，该模拟器能够准确地预测机器人的各个自由度的行为。相比之下，许多模型预测控制策略通常使用近似降阶模型。在这项工作中，我们放弃了在RL中使用高保真动力学模型的传统方法，而是试图了解当将RL与一个简单得多的质心模型应用于四足运动时，可以达到什么效果。我们表明，基于RL的质心模型的加速度控制是令人惊讶的有效的，当结合二次规划来实现通过地面接触力的命令动作。它允许简单的奖励结构、降低的计算成本和健壮的模拟到真实的传输。通过演示平地步态、踏石运动、两足原地平衡、平衡木运动和直接拟真转移等方法，展示了该方法的通用性。
Model-free reinforcement learning (RL) for legged locomotion commonly relies on a physics simulator that can accurately predict the behaviors of every degree of freedom of the robot. In contrast, approximate reduced-order models are commonly used for many model predictive control strategies. In this work we abandon the conventional use of high-fidelity dynamics models in RL and we instead seek to understand what can be achieved when using RL with a much simpler centroidal model when applied to quadrupedal locomotion. We show that RL-based control of the accelerations of a centroidal model is surprisingly effective, when combined with a quadratic program to realize the commanded actions via ground contact forces. It allows for a simple reward structure, reduced computational costs, and robust sim-to-real transfer. We show the generality of the method by demonstrating flat-terrain gaits, stepping-stone locomotion, two-legged in-place balance, balance beam locomotion, and direct sim-to-real transfer.