Stochastic Optimal Control Using Gaussian Process Regression over Probability Distributions
Stochastic Optimal Control Using Gaussian Process Regression over Probability Distributions
复制标题
DOI：
10.23919/acc.2019.8814658
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Jana Mayer;Maxim Dolgov;Tobias Stickling;Selim Özgen;Florian Rosenthal;U. Hanebeck
中科院分区：
文献类型：
--
作者：
Jana Mayer;Maxim Dolgov;Tobias Stickling;Selim Özgen;Florian Rosenthal;U. Hanebeck
在本文中，我们讨论了有限的控制输入和测量空间下的运动和测量不确定性的非线性随机系统的最优控制。这样的问题可以被形式化为部分可观察的马尔可夫决策过程，其目标是通过动态规划来找到策略，该动态规划将控制器可用的信息映射到控制输入，同时优化性能标准。然而，他们遭受棘手的情况下，连续的状态空间和部分可观测性，使近似必要。基于点的值迭代方法是一类全局近似方法，它在给定参考点的值的情况下对值函数进行回归。在本文中，我们提出了一种新的基于点的连续状态空间的值迭代方法，使用高斯过程定义的概率分布的回归。所提出的方法的主要优点是，它是非参数的，因此可以通过选择概率分布空间中的参考点的数量和位置来调整近似质量。此外，它还提供了方差方面的近似质量的概念。
In this paper, we address optimal control of nonlinear stochastic systems under motion and measurement uncertainty with finite control input and measurement spaces. Such problems can be formalized as partially-observable Markov decision processes where the goal is to find policies via dynamic programming that map the information available to the controller to control inputs while optimizing a performance criterion. However, they suffer from intractability in scenarios with continuous state spaces and partial observability which makes approximations necessary. Point-based value iteration methods are a class of global approximate methods that regress the value function given the values at a set of reference points. In this paper, we present a novel point-based value iteration approach for continuous state spaces that uses Gaussian processes defined over probability distribution for the regression. The main advantages of the proposed approach is that it is nonparametric and therefore approximation quality can be adjusted by choosing the number and the position of reference points in the space of probability distributions. In addition, it provides a notion of approximation quality in terms of variance.