SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
复制标题
DOI：
--
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Xingyu Lin;Yufei Wang;Jake Olkin;David Held
中科院分区：
文献类型：
--
作者：
Xingyu Lin;Yufei Wang;Jake Olkin;David Held
由于其高维状态表示和复杂的动力学，操作可变形物体长期以来一直是机器人技术中的一个挑战。深度强化学习最近的成功为使用数据驱动方法学习操作可变形物体提供了一个有前景的方向。然而，现有的强化学习基准测试仅涵盖具有直接状态可观测性和简单低维动力学的任务，或者是具有相对简单的基于图像的环境（例如那些有刚性物体的环境）的任务。在本文中，我们提出了SoftGym，这是一组用于操作可变形物体的开源模拟基准测试，它具有标准的OpenAI Gym API以及一个用于创建新环境的Python接口。我们的基准测试将使这一重要领域的研究具有可重复性。此外，我们在这些任务上评估了多种算法，并强调了强化学习算法面临的挑战，包括处理具有高内在维度且部分可观测的状态表示。实验和分析表明了现有方法在可变形物体操作背景下的优势和局限性，这有助于为未来方法的发展指明方向。学习到的策略的代码和视频可以在我们的项目网站上找到。
Manipulating deformable objects has long been a challenge in robotics due to its high dimensional state representation and complex dynamics. Recent success in deep reinforcement learning provides a promising direction for learning to manipulate deformable objects with data driven methods. However, existing reinforcement learning benchmarks only cover tasks with direct state observability and simple low-dimensional dynamics or with relatively simple image-based environments, such as those with rigid objects. In this paper, we present SoftGym, a set of open-source simulated benchmarks for manipulating deformable objects, with a standard OpenAI Gym API and a Python interface for creating new environments. Our benchmark will enable reproducible research in this important area. Further, we evaluate a variety of algorithms on these tasks and highlight challenges for reinforcement learning algorithms, including dealing with a state representation that has a high intrinsic dimensionality and is partially observable. The experiments and analysis indicate the strengths and limitations of existing methods in the context of deformable object manipulation that can help point the way forward for future methods development. Code and videos of the learned policies can be found on our project website.