EAGER: Data-Driven Control of Power Systems using Structured Reinforcement Learning

在可预见的未来，北美电网预计将发展成为最复杂的物联网，从数千个数字传感器产生大量数据。在这个现代电网中，传统发电以及可再生能源、储能、智能负载和电力电子转换器形式的分布式能源（DER）被预见将作为主动端点，对这些数据驱动的命令做出主动反应和响应。为了实现这一愿景，如今的电网运营商必须开始规划基础设施，不仅要安装更多的传感器和执行器，还要弥合两者之间的差距，使它们能够从数据过渡到控制。然而，挑战在于避开数据量和控制器大小的维度灾难。如今，即使是最简单的控制设计也需要立方的数值复杂度，这使得实时学习它们几乎是不可能的。EAGER项目的目标是朝着应对这一挑战迈出一步。其目标是开发机器学习算法，将大量电力系统数据转化为实时控制动作，而不会出现不可接受的长收敛时间。结构化学习算法可以扫描，识别和提取大型数据集的最有用的属性，同时还可以识别需要使用该数据控制的网格中最脆弱的部分，以实现这一目标，从而将计算和控制时间减少显著数量级。在技术方面，该项目将研究两个关键问题。首先，给定来自数百个相量测量单元的同步相量数据流，这些数据的所有频谱分量对于采取给定的广域控制动作是必不可少的吗？或者是否存在足以实现控制目标的低秩数据结构？第二，是否需要在扰动之后触发网格中的每个致动器，或者在扰动源周围是否存在一定的邻域，控制这些邻域可能足以使网格稳定并具有相当好的性能？此外，该邻域的轮廓是否可以从来自能量管理系统的在线同步相量数据和离线存档数据的组合来估计？我们将整合结构化压缩感知，模型简化理论和自适应动态规划的思想，从核心机器学习和控制理论的角度回答这些问题。我们的研究将解决同步相量的两个关键控制应用，即（1）使用传统发电机和DER的分级频率控制，以及（2）在大规模风能和太阳能穿透的情况下，主要联络线流量的功率振荡阻尼。该研究将为未来机器学习应用的理论和实验研究带来许多新的方向。的电力系统运营。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Over the foreseeable future, the North American power grid is envisioned to evolve as the most complex Internet-of-Things, generating massive volumes of data from thousands of digital sensors. In this modern grid, conventional generation as well as distributed energy resources (DERs) in the form of renewables, energy storage, smart loads, and power electronic converters are foreseen to serve as active endpoints that react and respond proactively to commands driven by these data. To realize this vision, grid operators today must start planning for an infrastructure that not only seeks to install more sensors and actuators, but also bridges the gap between the two, and enables them to transition from data to control. The challenge, however, lies in dodging the curse of dimensionality in both data volume and controller size. Even the simplest control designs today demand cubic numerical complexity, making it almost impossible to learn them in real-time. The objective of this EAGER project is to take a step forward towards addressing this challenge. The goal is to develop machine learning algorithms that translate large volumes of power system data to real-time control actions without running into unacceptably long convergence times. Structured learning algorithms that can scan, recognize, and extract the most useful attributes of large datasets, and at the same time also identify the most vulnerable parts of the grid that need to controlled using that data will be developed to achieve this goal, reducing computation and control time by significant orders of magnitude. On the technical front, the project will study two key questions. First, given Synchrophasor data streaming from hundreds of Phasor Measurement Units, are all spectral components of these data essential for taking a given wide-area control action Or does there exist a low-rank structure of the data that may be enough for the control goal? Second, does every actuator in the grid need to be triggered after a disturbance, or does there exist a certain neighborhood around the source of the disturbance, controlling which may be enough to stabilize the grid with a reasonably good performance? Furthermore, can the outline of this neighborhood be estimated from combinations of online Synchrophasor data and offline archived data from the energy management system? We will integrate ideas from structured compressive sensing, model reduction theory, and adaptive dynamic programming to answer these questions from core machine learning and control-theoretic points of view. Our study will address two critical control applications of Synchrophasors, namely (1) hierarchical frequency control using both conventional generators and DERs, and (2) power oscillation damping of major tie-line flows in the presence of large-scale wind and solar penetrations. The study will promote many new directions of theoretical and experimental research in the application of machine learning for tomorrow?'s power system operations. Workshops and conference tutorials will be organized to train graduate students and power system professionals in Synchrophasor data analytics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.