Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains
Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains
复制标题
DOI：
10.1109/lra.2021.3092676
复制
发表时间：
2021-10
影响因子：
5.2
通讯作者：
Aniruddh Gopinath Puranic;Jyotirmoy V. Deshmukh;S. Nikolaidis
中科院分区：
文献类型：
--
作者：
Aniruddh Gopinath Puranic;Jyotirmoy V. Deshmukh;S. Nikolaidis
安全、鲁棒和可解释的学习控制策略是开发机器人系统的突出挑战。用形式逻辑从演示中学习是强化学习中的一种新兴范式，用于估计奖励并提取寻求克服这些挑战的机器人控制策略。在这种方法中，我们假设机器人系统的任务级规范表示在一个合适的时间逻辑，如信号时序逻辑（STL）。其主要思想是通过对用户演示（可能是次优的或不完整的）进行评估和排名，自动推断出奖励。给定的STL规格。在现有的工作，侧重于确定性环境和离散状态空间，在这封信中，我们提出了重大的扩展，解决随机环境和连续状态空间。
Learning control policies that are safe, robust and interpretable are prominent challenges in developing robotic systems. Learning-from-demonstrations with formal logic is an arising paradigm in reinforcement learning to estimate rewards and extract robot control policies that seek to overcome these challenges. In this approach, we assume that mission-level specifications for the robotic system are expressed in a suitable temporal logic such as Signal Temporal Logic (STL). The main idea is to automatically infer rewards from user demonstrations (that could be suboptimal or incomplete) by evaluating and ranking them w.r.t. the given STL specifications. In contrast to existing work that focuses on deterministic environments and discrete state spaces, in this letter, we propose significant extensions that tackle stochastic environments and continuous state spaces.