A Composable Specification Language for Reinforcement Learning Tasks
A Composable Specification Language for Reinforcement Learning Tasks
复制标题
DOI：
--
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Kishor Jothimurugan;R. Alur;O. Bastani
中科院分区：
文献类型：
--
作者：
Kishor Jothimurugan;R. Alur;O. Bastani
强化学习是一种很有前途的学习机器人任务控制策略的方法。然而，指定复杂的任务（例如，具有多个目标和安全约束）可能具有挑战性，因为用户必须设计一个编码整个任务的奖励功能。此外，用户通常需要手动塑造奖励，以确保学习算法的收敛性。我们提出了一种用于指定复杂控制任务的语言，以及一种将我们语言中的规范编译成奖励函数并自动执行奖励塑造的算法。我们在一个名为SPECTRL的工具中实现了我们的方法，并表明它优于几个最先进的基线。
Reinforcement learning is a promising approach for learning control policies for robot tasks. However, specifying complex tasks (e.g., with multiple objectives and safety constraints) can be challenging, since the user must design a reward function that encodes the entire task. Furthermore, the user often needs to manually shape the reward to ensure convergence of the learning algorithm. We propose a language for specifying complex control tasks, along with an algorithm that compiles specifications in our language into a reward function and automatically performs reward shaping. We implement our approach in a tool called SPECTRL, and show that it outperforms several state-of-the-art baselines.