EAGER: Data-Driven Control of Power Systems using Structured Reinforcement Learning

在可预见的未来，北美电网预计将演变为最复杂的物联网，从数千个数字传感器产生海量数据。在这个现代电网中，传统发电以及可再生能源、储能、智能负载和电力电子转换器形式的分布式能源(DER)预计将作为主动端点，对这些数据驱动的命令做出主动反应和响应。为了实现这一愿景，今天的电网运营商必须开始规划基础设施，不仅要寻求安装更多的传感器和执行器，还要弥合两者之间的差距，使它们能够从数据过渡到控制。然而，挑战在于躲避数据量和控制器大小的维度诅咒。即使是当今最简单的控制设计也需要立方的数值复杂性，这使得实时学习它们几乎是不可能的。这一迫切项目的目标是朝着应对这一挑战迈进一步。其目标是开发机器学习算法，将大量电力系统数据转换为实时控制动作，而不会遇到不可接受的长时间收敛。为了实现这一目标，将开发结构化学习算法，这些算法可以扫描、识别和提取大型数据集的最有用属性，同时还可以识别需要使用这些数据控制的网格中最脆弱的部分，从而将计算和控制时间减少显著的数量级。在技术方面，该项目将研究两个关键问题。首先，给定来自数百个相量测量单元的同步相量数据，这些数据的所有光谱分量对于采取给定的广域控制行动是必不可少的还是存在可能足以用于控制目标的数据的低阶结构？其次，电网中的每个执行器是否需要在扰动后触发，或者干扰源周围是否存在特定的邻域，控制可能足以稳定电网并具有合理良好性能的邻域？此外，从能量管理系统的在线同步相量数据和离线存档数据的组合中可以估计出这个社区的轮廓吗？我们将结合结构压缩感知、模型降阶理论和自适应动态规划的思想，从核心机器学习和控制理论的角度来回答这些问题。我们的研究将解决同步相量的两个关键控制应用，即(1)使用常规发电机和DER的分级频率控制；(2)在大规模风和太阳穿透的情况下，主要联络线潮流的功率振荡衰减。这项研究将促进未来机器学习应用的许多新的理论和实验研究方向？S电力系统操作。将组织研讨会和会议教程，培训研究生和电力系统专业人员进行同步数据分析。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Over the foreseeable future, the North American power grid is envisioned to evolve as the most complex Internet-of-Things, generating massive volumes of data from thousands of digital sensors. In this modern grid, conventional generation as well as distributed energy resources (DERs) in the form of renewables, energy storage, smart loads, and power electronic converters are foreseen to serve as active endpoints that react and respond proactively to commands driven by these data. To realize this vision, grid operators today must start planning for an infrastructure that not only seeks to install more sensors and actuators, but also bridges the gap between the two, and enables them to transition from data to control. The challenge, however, lies in dodging the curse of dimensionality in both data volume and controller size. Even the simplest control designs today demand cubic numerical complexity, making it almost impossible to learn them in real-time. The objective of this EAGER project is to take a step forward towards addressing this challenge. The goal is to develop machine learning algorithms that translate large volumes of power system data to real-time control actions without running into unacceptably long convergence times. Structured learning algorithms that can scan, recognize, and extract the most useful attributes of large datasets, and at the same time also identify the most vulnerable parts of the grid that need to controlled using that data will be developed to achieve this goal, reducing computation and control time by significant orders of magnitude. On the technical front, the project will study two key questions. First, given Synchrophasor data streaming from hundreds of Phasor Measurement Units, are all spectral components of these data essential for taking a given wide-area control action Or does there exist a low-rank structure of the data that may be enough for the control goal? Second, does every actuator in the grid need to be triggered after a disturbance, or does there exist a certain neighborhood around the source of the disturbance, controlling which may be enough to stabilize the grid with a reasonably good performance? Furthermore, can the outline of this neighborhood be estimated from combinations of online Synchrophasor data and offline archived data from the energy management system? We will integrate ideas from structured compressive sensing, model reduction theory, and adaptive dynamic programming to answer these questions from core machine learning and control-theoretic points of view. Our study will address two critical control applications of Synchrophasors, namely (1) hierarchical frequency control using both conventional generators and DERs, and (2) power oscillation damping of major tie-line flows in the presence of large-scale wind and solar penetrations. The study will promote many new directions of theoretical and experimental research in the application of machine learning for tomorrow?'s power system operations. Workshops and conference tutorials will be organized to train graduate students and power system professionals in Synchrophasor data analytics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.