Collaborative Research: Data-driven Power Systems Control with Stability Guarantee: A Lyapunov Approach

该项目旨在设计一种新的数据驱动的电力系统控制框架，具有稳定性保证。由于可再生能源发电和分布式能源（包括太阳能、电动汽车和电池）的激增，电力系统正在经历一个快速变化的时期。这些新技术中的许多通过电力电子接口（即，逆变器），与传统机器相比，可以在更快的时间尺度上进行控制。然而，由于底层电力网络的非线性、复杂性和不确定性，如何利用这种灵活性是不平凡的。该项目将通过为基于逆变器的频率和电压控制开发新的强化学习（RL）算法，并提供正式的稳定性保证，从而带来变革性的变化。该项目的智力优势包括：（i）一个新的框架，将李亚普诺夫控制理论和RL连接起来，从而为基于学习的控制器提供稳定性保证;（ii）神经网络结构设计，确保设计满足稳定性约束。该项目的更广泛影响包括为对能源系统和机器学习/AI感兴趣的学生提供各种新课程开发和研究机会。第一个重点是开发将RL与李雅普诺夫稳定性约束相结合的算法框架，这是后来的基础。具体来说，我们将利用分析模型来构造李雅普诺夫函数，并设计基于神经网络的控制器的结构，以满足稳定性约束。Thrust 2使用机器学习为现实的电力系统模型发现新的李雅普诺夫函数，并设计稳定的控制策略。Thrust 3集成了Thrust 1和2中开发的理论和算法，并使控制器对建模误差和输电网和配电网中的网络拓扑重新配置具有鲁棒性。该项目的贡献是双重的。在理论方面，所提出的研究桥接了经典控制和学习，其中控制理论提供了保证控制器稳定的结构约束，而RL和神经网络在大参数空间中搜索，以找到具有这种结构的最佳性能控制器。在实践方面，我们的方法通过保证学习策略的稳定性，清除了将RL应用于电力系统的关键障碍。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This NSF project aims to design a new data-driven power system control framework with stability guarantee. Power systems are experiencing a period of rapid changes due to the proliferation of renewable generation and distributed energy resources including solar, electric vehicles, and batteries. Many of these new technologies are interfaced with the grid through power electronic interfaces (i.e., inverters) that can be controlled at a much faster timescale compared to conventional machines. However, how to leverage such flexibility is nontrivial due to the nonlinearity, complexity, and uncertainty in the underlying power network. This project will bring transformative changes by developing new reinforcement learning (RL) algorithms for inverter-based frequency and voltage control with formal stability guarantees. The intellectual merits of the project include (i) a novel framework that bridges Lyapunov control theory and RL, therefore providing stability guarantee for learning-based controllers; (ii) neural network structure design that ensures stability constraint is met by design. The broader impacts of the project include various of new courses development and research opportunities for students interested in both energy systems and machine learning/AI.The proposed research consists of three thrusts. Thrust 1 focuses on developing the algorithmic framework that integrates RL with Lyapunov stability constraints, which serves as a foundation to later thrusts. Specifically, we will leverage analytical models to construct Lyapunov functions and engineer the structure of neural network-based controllers to meet the stability constraints. Thrust 2 uses machine learning to discover new Lyapunov functions for realistic power system models and design stable control policies. Thrust 3 integrates the theory and algorithms developed in Thrusts 1 and 2, and robustifies the controllers against modeling error, and network topology re-configurations in both transmission and distribution grids. The contributions of the project are two-folded. On the theoretical side, the proposed research bridges classic control and learning, where control theory provides the structural constraints that guarantee a controller is stable, and RL with neural networks searches over the large parametric spaces to find the best performing controllers that have this structure. On the practical side, our approach clears a critical hurdle in applying RL to power systems by guaranteeing the stability of the learned policy. We envision our framework will serve as the basis for future learning-based smart power system control architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.