Value iteration with deep neural networks for optimal control of input-affine nonlinear systems
Value iteration with deep neural networks for optimal control of input-affine nonlinear systems
复制标题
DOI：
10.1080/18824889.2021.1936817
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Hirofumi Beppu;I. Maruta;K. Fujimoto
中科院分区：
文献类型：
--
作者：
Hirofumi Beppu;I. Maruta;K. Fujimoto
针对连续时间输入非线性系统的最优控制问题，提出了一种基于数值迭代算法的深度神经网络算法。该算法在迭代过程中利用神经网络逼近值函数和控制输入。因此，原算法中的偏微分方程组转化为网络参数的优化问题。传统算法虽然可以通过迭代计算得到最优控制，但每次计算都需要精确完成，在实际应用中很难达到足够的精度。相反，所提出的方法提供了一种使用深度神经网络的实用方法，并克服了基于网络性质的困难，在此情况下，我们的收敛性分析表明，所提出的算法可以实现值函数和相应的最优控制器的最小值。在两个数值模拟中证明了所提出的方法的有效性，即使有合理的计算资源。
This paper proposes a new algorithm with deep neural networks to solve optimal control problems for continuous-time input nonlinear systems based on a value iteration algorithm. The proposed algorithm applies the networks to approximating the value functions and control inputs in the iterations. Consequently, the partial differential equations of the original algorithm reduce to the optimization problems for the parameters of the networks. Although the conventional algorithm can obtain the optimal control with iterative computations, each of the computations needs to be completed precisely, and it is hard to achieve sufficient precision in practice. Instead, the proposed method provides a practical method using deep neural networks and overcomes the difficulty based on a property of the networks, under which our convergence analysis shows that the proposed algorithm can achieve the minimum of the value function and the corresponding optimal controller. The effectiveness of the proposed method even with reasonable computational resources is demonstrated in two numerical simulations.