Decentralized safe reinforcement learning for inverter-based voltage control
Decentralized safe reinforcement learning for inverter-based voltage control
复制标题
DOI：
10.1016/j.epsr.2022.108609
复制
发表时间：
2022-10
影响因子：
3.9
通讯作者：
Wenqi Cui;Jiayi Li;Baosen Zhang
中科院分区：
文献类型：
--
作者：
Wenqi Cui;Jiayi Li;Baosen Zhang
基于逆变器的分布式能源通过快速调节其无功功率为快速时标电压控制提供了可能性。电力电子接口允许这些资源实现几乎任意的控制律，但设计这些分散控制器是不平凡的。强化学习（RL）方法越来越受欢迎，用于搜索由神经网络参数化的策略。这是困难的，但是，强制执行，学习控制器是安全的，在这个意义上，他们可能会引入不稳定性到system.This本文提出了一种安全的学习方法的电压控制。我们证明了系统是指数稳定的，如果每个控制器满足一定的Lipschitz约束。优化了Lipschitz界集，扩大了神经网络控制器的搜索空间。我们显式地设计神经网络控制器的结构，使它们满足Lipschitz约束。构建了一个分散的强化学习框架，在无模型设置中训练每个总线上的局部神经网络控制器。
Inverter-based distributed energy resources provide the possibility for fast time-scale voltage control by quickly adjusting their reactive power. The power-electronic interfaces allow these resources to realize almost arbitrary control law, but designing these decentralized controllers is nontrivial. Reinforcement learning (RL) approaches are becoming increasingly popular to search for policy parameterized by neural networks. It is difficult, however, to enforce that the learned controllers are safe, in the sense that they may introduce instabilities into the system.This paper proposes a safe learning approach for voltage control. We prove that the system is guaranteed to be exponentially stable if each controller satisfies certain Lipschitz constraints. The set of Lipschitz bound is optimized to enlarge the search space for neural network controllers. We explicitly engineer the structure of neural network controllers such that they satisfy the Lipschitz constraints by design. A decentralized RL framework is constructed to train local neural network controller at each bus in a model-free setting.