Optimal Control of Discounted-Reward Markov Decision Processes Under Linear Temporal Logic Specifications
Optimal Control of Discounted-Reward Markov Decision Processes Under Linear Temporal Logic Specifications
复制标题
线性时序逻辑规范下贴现奖励马尔可夫决策过程的最优控制
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
P. Nuzzo
中科院分区：
文献类型：
--
作者：
K. C. Kalagarla;R. Jain;P. Nuzzo
我们提出了一种方法来找到一个最优的政策，关于一个折扣马尔可夫决策过程的奖励函数一般线性时序逻辑（LTL）规范。以前的工作要么集中在有限持续时间任务下最大化累积奖励目标，要么集中在最大化持续时间任务的平均奖励（例如，监视）任务。本文通过引入一对占用测度来分别表示LTL满意度目标和期望折扣奖励目标，扩展和推广了这些结果。这些占用措施，然后连接到一个单一的政策，通过一个新的减少导致在一个混合整数线性规划的解决方案提供了一个最佳的政策。我们的公式也可以扩展到包括额外的约束条件的二级奖励功能。我们说明了我们的方法的有效性，在机器人运动规划的背景下，复杂的任务不确定性和性能目标。
We present a method to find an optimal policy with respect to a reward function for a discounted Markov decision process under general linear temporal logic (LTL) specifications. Previous work has either focused on maximizing a cumulative reward objective under finite-duration tasks or maximizing an average reward for persistent (e.g., surveillance) tasks. This paper extends and generalizes these results by introducing a pair of occupancy measures to express the LTL satisfaction objective and the expected discounted reward objective, respectively. These occupancy measures are then connected to a single policy via a novel reduction resulting in a mixed integer linear program whose solution provides an optimal policy. Our formulation can also be extended to include additional constraints with respect to secondary reward functions. We illustrate the effectiveness of our approach in the context of robotic motion planning for complex missions under uncertainty and performance objectives.