Stability Constrained Reinforcement Learning for Real-Time Voltage Control
Stability Constrained Reinforcement Learning for Real-Time Voltage Control
复制标题
DOI：
10.23919/acc53348.2022.9867476
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Yuanyuan Shi;Guannan Qu;S. Low;Anima Anandkumar;A. Wierman
中科院分区：
文献类型：
--
作者：
Yuanyuan Shi;Guannan Qu;S. Low;Anima Anandkumar;A. Wierman
深度强化学习（RL）已被视为应对电力系统实时控制挑战的一种有前途的工具。然而，由于缺乏正式的稳定性和安全性保证，其在现实世界电力系统中的应用受到了阻碍。在本文中，我们提出了一种用于配电网实时电压控制的稳定性约束强化学习方法，并证明了所提出的方法提供了一种正式的电压稳定性保证。我们方法的核心思想是一个明确构建的李雅普诺夫函数，它能确保稳定性。我们在案例研究中证明了该方法的有效性，与一种广泛使用的线性策略相比，所提出的方法可将暂态控制成本降低30%以上，并将响应时间缩短三分之一，同时始终能实现电压稳定。相比之下，标准的强化学习方法往往无法实现电压稳定。
Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems. However, its deployment in real-world power systems has been hindered by a lack of formal stability and safety guarantees. In this paper, we propose a stability constrained reinforcement learning method for real-time voltage control in distribution grids and we prove that the proposed approach provides a formal voltage stability guarantee. The key idea underlying our approach is an explicitly constructed Lyapunov function that certifies stability. We demonstrate the effectiveness of the approach in case studies, where the proposed method can reduce the transient control cost by more than 30% and shorten the response time by a third compared to a widely used linear policy, while always achieving voltage stability. In contrast, standard RL methods often fail to achieve voltage stability. 1