Collaborative Research: Data-driven Power Systems Control with Stability Guarantees

本项目旨在设计一种具有稳定保障的数据驱动电力系统控制框架。由于可再生能源发电和分布式能源（包括太阳能、电动汽车和电池）的激增，电力系统正在经历一个快速变化的时期。这些新技术中的许多都是通过电力电子接口（即逆变器）与电网连接的，与传统机器相比，这些接口可以以更快的时间尺度进行控制。然而，由于底层电网的非线性、复杂性和不确定性，如何利用这种灵活性是非常重要的。该项目将通过开发新的强化学习（RL）算法来实现基于逆变器的频率和电压控制，并提供形式稳定性保证，从而带来革命性的变化。该项目的智力优势包括(i)一个连接李亚普诺夫控制理论和强化学习的新框架，因此为基于学习的控制器提供了稳定性保证；（ii）保证设计满足稳定性约束的神经网络结构设计。该项目的广泛影响包括为对能源系统和机器学习/人工智能感兴趣的学生提供各种新课程开发和研究机会。拟议的研究包括三个重点。Thrust 1专注于开发将RL与Lyapunov稳定性约束集成在一起的算法框架，为以后的推力奠定基础。具体来说，我们将利用分析模型来构建李雅普诺夫函数，并设计基于神经网络的控制器结构以满足稳定性约束。Thrust 2使用机器学习为现实电力系统模型发现新的Lyapunov函数并设计稳定的控制策略。推力3集成了推力1和推力2中开发的理论和算法，并对输配电电网中的建模误差和网络拓扑重新配置进行了鲁棒控制。这个项目的贡献是双重的。在理论方面，提出的研究将经典控制和学习联系起来，其中控制理论提供保证控制器稳定的结构约束，而RL使用神经网络在大参数空间中搜索以找到具有该结构的最佳性能控制器。在实践方面，我们的方法通过保证学习策略的稳定性，清除了将RL应用于电力系统的关键障碍。我们设想我们的框架将作为未来基于学习的智能电力系统控制架构的基础。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This NSF project aims to design a new data-driven power system control framework with stability guarantee. Power systems are experiencing a period of rapid changes due to the proliferation of renewable generation and distributed energy resources including solar, electric vehicles, and batteries. Many of these new technologies are interfaced with the grid through power electronic interfaces (i.e., inverters) that can be controlled at a much faster timescale compared to conventional machines. However, how to leverage such flexibility is nontrivial due to the nonlinearity, complexity, and uncertainty in the underlying power network. This project will bring transformative changes by developing new reinforcement learning (RL) algorithms for inverter-based frequency and voltage control with formal stability guarantees. The intellectual merits of the project include (i) a novel framework that bridges Lyapunov control theory and RL, therefore providing stability guarantee for learning-based controllers; (ii) neural network structure design that ensures stability constraint is met by design. The broader impacts of the project include various of new courses development and research opportunities for students interested in both energy systems and machine learning/AI.The proposed research consists of three thrusts. Thrust 1 focuses on developing the algorithmic framework that integrates RL with Lyapunov stability constraints, which serves as a foundation to later thrusts. Specifically, we will leverage analytical models to construct Lyapunov functions and engineer the structure of neural network-based controllers to meet the stability constraints. Thrust 2 uses machine learning to discover new Lyapunov functions for realistic power system models and design stable control policies. Thrust 3 integrates the theory and algorithms developed in Thrusts 1 and 2, and robustifies the controllers against modeling error, and network topology re-configurations in both transmission and distribution grids. The contributions of the project are two-folded. On the theoretical side, the proposed research bridges classic control and learning, where control theory provides the structural constraints that guarantee a controller is stable, and RL with neural networks searches over the large parametric spaces to find the best performing controllers that have this structure. On the practical side, our approach clears a critical hurdle in applying RL to power systems by guaranteeing the stability of the learned policy. We envision our framework will serve as the basis for future learning-based smart power system control architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.