An Iterative Method for Nonlinear Stochastic Optimal Control Based on Path Integrals
An Iterative Method for Nonlinear Stochastic Optimal Control Based on Path Integrals
复制标题
DOI：
10.1109/tac.2016.2547979
复制
发表时间：
2017
影响因子：
6.8
通讯作者：
Satoshi Satoh;H. Kappen;M. Saeki
中科院分区：
文献类型：
--
作者：
Satoshi Satoh;H. Kappen;M. Saeki
提出一种基于路径积分分析的非线性随机最优控制问题的迭代求解新方法。首先，我们提供了求解与该问题相关的随机 Hamilton-Jacobi-Bellman (SHJB) 方程的迭代律，该方程是二阶非线性偏微分方程 (PDE)。该方法的每个迭代过程都由线性抛物型偏微分方程的柯西问题表示，其显式解由 Feynman-Kac 公式给出。其次，我们通过使用路径积分分析在每次迭代中导出次优反馈控制器。第三，研究了该方法的收敛性。这里，提供了一些条件，使得所提出的迭代的解序列收敛，并且满足 SHJB 方程。最后，数值模拟证明了该方法的有效性。
This paper proposes a new iterative solution method for nonlinear stochastic optimal control problems based on path integral analysis. First, we provide an iteration law for solving a stochastic Hamilton-Jacobi-Bellman (SHJB) equation associated to this problem, which is a nonlinear partial differential equation (PDE) of second order. Each iteration procedure of the proposed method is represented by a Cauchy problem for a linear parabolic PDE, and its explicit solution is given by the Feynman-Kac formula. Second, we derive a suboptimal feedback controller at each iteration by using the path integral analysis. Third, the convergence property of the proposed method is investigated. Here, some conditions are provided so that the sequence of solutions for the proposed iteration converges, and the SHJB equation is satisfied. Finally, numerical simulations demonstrate the effectiveness of the proposed method.