GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning
GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning
复制标题
DOI：
--
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
Jacky Liang;Viktor Makoviychuk;Ankur Handa;N. Chentanez;M. Macklin;D. Fox
中科院分区：
文献类型：
--
作者：
Jacky Liang;Viktor Makoviychuk;Ankur Handa;N. Chentanez;M. Macklin;D. Fox
大多数深度强化学习（Deep RL）算法需要大量的训练样本来学习复杂的任务。最近许多关于加速深度强化学习的工作都集中在分布式训练和模拟上。虽然分布式训练通常是在GPU上完成的，但模拟不是。在这项工作中，我们建议使用gpu加速的RL模拟作为CPU模拟的替代方案。使用NVIDIA Flex，一个基于gpu的物理引擎，我们展示了学习各种连续控制，运动任务的有希望的加速。使用一个GPU和CPU核心，我们能够在不到20分钟的时间内训练Humanoid运行任务，使用的CPU内核比以前的工作少10-1000倍。我们还展示了我们的模拟器的多gpu设置的可扩展性，以训练更具挑战性的运动任务。
Most Deep Reinforcement Learning (Deep RL) algorithms require a prohibitively large number of training samples for learning complex tasks. Many recent works on speeding up Deep RL have focused on distributed training and simulation. While distributed training is often done on the GPU, simulation is not. In this work, we propose using GPU-accelerated RL simulations as an alternative to CPU ones. Using NVIDIA Flex, a GPU-based physics engine, we show promising speed-ups of learning various continuous-control, locomotion tasks. With one GPU and CPU core, we are able to train the Humanoid running task in less than 20 minutes, using 10-1000x fewer CPU cores than previous works. We also demonstrate the scalability of our simulator to multi-GPU settings to train more challenging locomotion tasks.