Reinforcement Learning using Physics Inspired Graph Convolutional Neural Networks
Reinforcement Learning using Physics Inspired Graph Convolutional Neural Networks
复制标题
DOI：
10.1109/allerton49937.2022.9929321
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Tong Wu;A. Scaglione;D. Arnold
中科院分区：
文献类型：
--
作者：
Tong Wu;A. Scaglione;D. Arnold
在这项工作中，我们提出了一个启发的图形卷积神经网络（GCN） - 增强架构（RL）体系结构来培训在线控制器政策，以最佳选择分布式能源（DER），而GCN的使用是兼容的。使用任何DRL方案，我们将其与流行的近端策略优化（PPO）算法结合进行测试，并且作为应用程序，我们考虑选择的设定点选择智能逆变器作为DER的案例研究的Volt/var/watt Control逻辑，我们能够在数值上表明GCN方案比确定由网络产生的电压和不可避免的电压动力学更有效除了探索给定网络的GCN的性能外，我们还研究了由于拓扑或线路参数而动态变化的网格GCN-RL政策对小扰动的鲁棒性，并评估所谓的“转移学习”能力的计划。
In this work, we propose a physics inspired Graph Convolutional Neural Network (GCN)-Reinforcement Learning (RL) architecture to train online controllers policies for the optimal selection of Distributed Energy Resources (DER) set-points. While the use of GCN is compatible with any DRL scheme, we test it in combination with the popular proximal policy optimization (PPO) algorithm and, as application, we consider the selection of set-points for Volt/Var and Volt/Watt control logic of smart inverters as the case study for DER control. We are able to show numerically that the GCN scheme is more effective than various benchmarks in regulating voltage and miti-gating undesirable voltage dynamics generated by cyber-attacks. In addition to exploring the performance of GCN for a given network, we investigate the case of grids that are dynamically changing due to topology or line parameters variations. We test the robustness of GCN-RL policies against small perturbations and evaluate the scheme so called “transfer learning” capabilities.