Value iteration and adaptive optimal output regulation with assured convergence rate
Value iteration and adaptive optimal output regulation with assured convergence rate
复制标题
DOI：
10.1016/j.conengprac.2021.105042
复制
发表时间：
2022-01-10
影响因子：
4.9
通讯作者：
Lewis, Frank L.
中科院分区：
文献类型：
--
作者：
Jiang, Yi;Gao, Weinan;Lewis, Frank L.
在本文中，我们研究了受扰线性连续时间系统具有收敛速度要求的基于学习的自适应最优输出调节问题。提出了一种基于强化学习和自适应动态规划的自适应最优控制方法，以在保证收敛速度的情况下学习最优调节器。通过解决静态优化问题来寻找调节器方程的最优解，以及动态和约束优化问题来获得最优反馈控制增益，成功地解决了上述问题。在不需要精确的系统动力学或稳定的反馈控制增益的情况下，提出了一种新颖的在线值迭代算法，该算法可以使用可测量的数据来学习最优反馈控制增益和相应的前馈控制增益。此外，保证闭环系统的输出收敛得更快或等于用户设置的预定义收敛速率。最后，对基于 LCL 耦合逆变器的分布式发电系统的数值分析表明，所提出的方法可以实现预期的抗扰和跟踪性能。
In this paper, we investigate the learning-based adaptive optimal output regulation problem with convergence rate requirement for disturbed linear continuous-time systems. An adaptive optimal control approach is proposed based on reinforcement learning and adaptive dynamic programming to learn the optimal regulator with assured convergence rate. The above-mentioned problem is successfully solved by tackling a static optimization problem to find the optimal solution to the regulator equations, and a dynamic and constrained optimization problem to obtain the optimal feedback control gain. Without requiring on the accurate system dynamics or a stabilizing feedback control gain, a novel online value iteration algorithm is proposed, which can learn both the optimal feedback control gain and the corresponding feedforward control gain using measurable data. Moreover, the output of the closed-loop system is guaranteed to converge faster or equal to a predefined convergence rate set by user. Finally, the numerical analysis on a LCL coupled inverter-based distributed generation system shows that the proposed approach can achieve desired disturbance rejection and tracking performance.