Collaborative Research: Data-driven Power Systems Control with Stability Guarantee: A Lyapunov Approach

合作研究：具有稳定性保证的数据驱动电力系统控制：李亚普诺夫方法

• 批准号: 2200692
• 负责人: Yuanyuan Shi
• 金额: $ 20万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-03-15 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2200692&HistoricalAwards=false
• 关键词: Collaborative; Research; Data; driven; Power

This NSF project aims to design a new data-driven power system control framework with stability guarantee. Power systems are experiencing a period of rapid changes due to the proliferation of renewable generation and distributed energy resources including solar, electric vehicles, and batteries. Many of these new technologies are interfaced with the grid through power electronic interfaces (i.e., inverters) that can be controlled at a much faster timescale compared to conventional machines. However, how to leverage such flexibility is nontrivial due to the nonlinearity, complexity, and uncertainty in the underlying power network. This project will bring transformative changes by developing new reinforcement learning (RL) algorithms for inverter-based frequency and voltage control with formal stability guarantees. The intellectual merits of the project include (i) a novel framework that bridges Lyapunov control theory and RL, therefore providing stability guarantee for learning-based controllers; (ii) neural network structure design that ensures stability constraint is met by design. The broader impacts of the project include various of new courses development and research opportunities for students interested in both energy systems and machine learning/AI.The proposed research consists of three thrusts. Thrust 1 focuses on developing the algorithmic framework that integrates RL with Lyapunov stability constraints, which serves as a foundation to later thrusts. Specifically, we will leverage analytical models to construct Lyapunov functions and engineer the structure of neural network-based controllers to meet the stability constraints. Thrust 2 uses machine learning to discover new Lyapunov functions for realistic power system models and design stable control policies. Thrust 3 integrates the theory and algorithms developed in Thrusts 1 and 2, and robustifies the controllers against modeling error, and network topology re-configurations in both transmission and distribution grids. The contributions of the project are two-folded. On the theoretical side, the proposed research bridges classic control and learning, where control theory provides the structural constraints that guarantee a controller is stable, and RL with neural networks searches over the large parametric spaces to find the best performing controllers that have this structure. On the practical side, our approach clears a critical hurdle in applying RL to power systems by guaranteeing the stability of the learned policy. We envision our framework will serve as the basis for future learning-based smart power system control architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

NSF项目旨在设计一种新的具有稳定性保证的数据驱动的电力系统控制框架。由于太阳能、电动汽车和电池等可再生能源和分布式能源的激增，电力系统正在经历一段快速变化的时期。这些新技术中的许多都是通过电力电子接口(即逆变器)与电网连接的，与传统机器相比，这些接口可以更快的时间范围进行控制。然而，由于底层电力网络的非线性、复杂性和不确定性，如何利用这种灵活性并不是微不足道的。该项目将通过开发新的强化学习(RL)算法来为基于逆变器的频率和电压控制带来变革性的变化，并提供形式上的稳定性保证。该项目的智能优点包括：(I)一个新的框架，将Lyapunov控制理论与RL联系起来，从而为基于学习的控制器提供稳定性保证；(Ii)神经网络结构设计，确保通过设计满足稳定性约束。该项目的更广泛影响包括为对能源系统和机器学习/AI感兴趣的学生提供各种新课程开发和研究机会。推力1专注于开发将RL与Lyapunov稳定性约束相结合的算法框架，作为后续推力的基础。具体地说，我们将利用分析模型来构造Lyapunov函数，并设计基于神经网络的控制器的结构，以满足稳定性约束。推力2利用机器学习为现实电力系统模型发现新的李雅普诺夫函数，并设计稳定的控制策略。推力3集成了推力1和推力2中开发的理论和算法，并针对建模错误和输电网和配电网中的网络拓扑重新配置使控制器具有健壮性。该项目的贡献是双重的。在理论方面，提出的研究将经典控制和学习联系起来，其中控制理论提供了确保控制器稳定的结构约束，而神经网络RL在大的参数空间中搜索具有这种结构的最佳控制器。在实践方面，通过保证学习策略的稳定性，我们的方法清除了将RL应用于电力系统的一个关键障碍。我们设想我们的框架将作为未来基于学习的智能电力系统控制体系结构的基础。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Robust online voltage control with an unknown grid topology

具有未知电网拓扑的鲁棒在线电压控制

• DOI: 10.1145/3538637.3538853
• 发表时间: 2022
• 期刊: ACM International Conference on Future Energy Systems
• 作者: Yeh, Christopher;Yu, Jing;Shi, Yuanyuan;Wierman, Adam
• 通讯作者: Wierman, Adam

Structured Neural-PI Control with End-to-End Stability and Output Tracking Guarantees

• DOI: 10.48550/arxiv.2305.17777
• 发表时间: 2023-05
• 期刊: ArXiv
• 作者: Wenqi Cui;Yan Jiang;Baosen Zhang;Yuanyuan Shi

Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control

• DOI: 10.1109/tcns.2023.3338240
• 发表时间: 2022-09
• 期刊: IEEE Transactions on Control of Network Systems
• 影响因子: 4.2
• 作者: Jie Feng;Yuanyuan Shi;Guannan Qu;S. Low;Anima Anandkumar;A. Wierman

SustainGym: Reinforcement Learning Environments for Sustainable Energy Systems

• 发表时间: 2023
• 作者: Christopher Yeh;Victor Li;Rajeev Datta;Julio Arroyo;Nicolas H. Christianson;Chi Zhang;Yize Chen;Mohammad Mehdi Hosseini;A. Golmohammadi;Yuanyuan Shi;Yisong Yue;Adam Wierman

Leveraging Predictions in Power System Frequency Control: An Adaptive Approach

• DOI: 10.1109/cdc49753.2023.10383969
• 发表时间: 2023-05
• 期刊: 2023 62nd IEEE Conference on Decision and Control (CDC)
• 作者: Wenqi Cui;Guanya Shi;Yuanyuan Shi;Baosen Zhang