Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approach
Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approach
复制标题
DOI：
10.1109/tpwrs.2022.3176525
复制
发表时间：
2020-09
影响因子：
6.6
通讯作者：
Wenqi Cui;Yan Jiang;Baosen Zhang
中科院分区：
文献类型：
--
作者：
Wenqi Cui;Yan Jiang;Baosen Zhang
随着越来越多的逆变器连接的可再生能源被整合到电网中，频率稳定性可能会因为机械惯性和阻尼的降低而降低。缓解这种性能下降的常见方法是使用可再生资源的电力电子接口进行主频率控制。由于逆变器连接的资源可以实现对频率变化的几乎任意响应，因此它们不限于再现线性下垂行为。为了充分利用它们的能力，强化学习（RL）已经成为设计非线性控制器以优化许多目标函数的流行方法。由于逆变器连接的资源和同步发电机将是电网的重要组成部分，在不久的将来和中期，前者的学习控制器应该稳定相对于后者的非线性动态。为了克服这一挑战，我们明确地设计了基于神经网络的控制器的结构，使得它们通过使用李雅普诺夫函数来保证系统的稳定性。一个递归神经网络架构是用来有效地训练控制器。所得到的控制器只使用本地信息和优于最佳线性下垂以及其他国家的最先进的学习方法。
As more inverter-connected renewable resources are integrated into the grid, frequency stability may degrade because of the reduction in mechanical inertia and damping. A common approach to mitigate this degradation in performance is to use the power electronic interfaces of the renewable resources for primary frequency control. Since inverter-connected resources can realize almost arbitrary responses to frequency changes, they are not limited to reproducing the linear droop behaviors. To fully leverage their capabilities, reinforcement learning (RL) has emerged as a popular method to design nonlinear controllers to optimize a host of objective functions. Because both inverter-connected resources and synchronous generators would be a significant part of the grid in the near and intermediate future, the learned controller of the former should be stabilizing with respect to the nonlinear dynamics of the latter. To overcome this challenge, we explicitly engineer the structure of neural network-based controllers such that they guarantee system stability by construction, through the use of a Lyapunov function. A recurrent neural network architecture is used to efficiently train the controllers. The resulting controllers only use local information and outperform optimal linear droop as well as other state-of-the-art learning approaches.