Robot Learning With Crash Constraints
Robot Learning With Crash Constraints
复制标题
具有碰撞约束的机器人学习
DOI：
10.1109/lra.2021.3057055
复制
发表时间：
2020
影响因子：
5.2
通讯作者：
Sebastian Trimpe
中科院分区：
文献类型：
--
作者：
A. Marco;Dominik Baumann;M. Khadiv;Philipp Hennig;L. Righetti;Sebastian Trimpe
在过去的十年中，许多机器学习算法已经被证明可以成功地学习最优策略来控制真实的机器人系统。然而，随着学习循环的进行，遇到失败的行为是很常见的。具体来说，在机器人应用中，失败是不希望的，但不是灾难性的，许多算法都在努力利用从失败中获得的数据。这通常是由于（i）失败的实验过早结束，或（ii）所获得的数据稀缺或损坏。这两种方法都使惩罚失败的适当奖励函数的设计复杂化。在这封信中，我们提出了一个解决这些问题的框架。我们认为失败的行为是那些违反约束和解决问题的学习崩溃的约束，没有数据时，约束违反。无数据的情况下，解决了一种新的GP模型（GPCR）的约束，结合离散事件（失败/成功）与连续观测（仅获得成功后）。我们证明了我们的框架的有效性，在模拟的基准测试和一个真实的跳跃四足动物，约束阈值是未知的先验。实验数据收集，通过约束贝叶斯优化，直接在真实的机器人。我们的结果优于手动调整和GPCR证明有用的估计约束阈值。
In the past decade, numerous machine learning algorithms have been shown to successfully learn optimal policies to control real robotic systems. However, it is common to encounter failing behaviors as the learning loop progresses. Specifically, in robot applications where failing is undesired but not catastrophic, many algorithms struggle with leveraging data obtained from failures. This is usually caused by (i) the failed experiment ending prematurely, or (ii) the acquired data being scarce or corrupted. Both complicate the design of proper reward functions to penalize failures. In this letter, we propose a framework that addresses those issues. We consider failing behaviors as those that violate a constraint and address the problem of learning with crash constraints, where no data is obtained upon constraint violation. The no-data case is addressed by a novel GP model (GPCR) for the constraint that combines discrete events (failure/success) with continuous observations (only obtained upon success). We demonstrate the effectiveness of our framework on simulated benchmarks and on a real jumping quadruped, where the constraint threshold is unknown a priori. Experimental data is collected, by means of constrained Bayesian optimization, directly on the real robot. Our results outperform manual tuning and GPCR proves useful on estimating the constraint threshold.
影响因子：
5.2
作者：
Grimminger, Felix;Meduri, Avadesh;Righetti, Ludovic
通讯作者：
Righetti, Ludovic