Enhanced Power System Resiliency through Adaptive Automatic Remedial Action Selection using Multi-Agent Reinforcement Learning

使用多智能体强化学习进行自适应自动补救措施选择，增强电力系统的弹性

• 批准号: 2231677
• 负责人: Luigi Vanfretti
• 金额: $ 38.3万
• 依托单位: Rensselaer Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2231677&HistoricalAwards=false
• 关键词: Enhanced; Power; System; Resiliency; through

Electrical power grids have been facing increased operational challenges, including those due to natural disasters, extreme weather conditions, and increasing complexity with the ongoing upgrades to achieve zero-carbon footprint goals. These factors challenge electrical power system operators in making decisions on which action to take in order to keep most of the households powered by the grid or even to avoid massive blackouts. Even though the control centers’ technology and their practices undergo upgrades over time, grid failures and blackouts still happen. When a severe event occurs on the grid, the power system operator has to follow specific procedures to resolve the unfavorable condition in order to bring the system back to normal operation. With the aforementioned challenges in mind, it becomes necessary to automate the process of choosing the most suitable remedial actions. In this case, the speed of decision-making is crucial to avoid massive disconnections and minimize the impact of unfavorable events on grid equipment. A significant improvement in response time and even full automation may become possible in practice with the help of modern computer science methods such as machine learning algorithms and, more specifically, its subgroup of learning methods that is known as reinforcement learning. Reinforcement learning acquires knowledge by choosing actions that provide the largest benefit and has been shown to solve complex problems, such as training robots to solve complicated tasks or even playing complex games such as chess and the ancient game of Go.Leveraging reinforcement learning methods, the proposed solution in this research allows us to prioritize an action according to its influence on consumers by choosing the action that reduces potential negative impacts. This is done by assigning higher priority to a potential action if the risk of disconnection of critical facilities is lower. To fully exploit the advantages of reinforcement learning methods, multiple “agents” (decision makers) are used. These agents can be distributed all across different grid facilities and perform local control actions. However, to assure grid resiliency, there is a degree of central coordination between them. Each agent is responsible for its own control area where it can operate according to the prioritized instructions that are provided in the form of actions during the agent’s learning phase. In such a way, we aim to increase the trustworthiness of the agents by incorporating prior knowledge about the power system that is learned via ultra-high-fidelity nonlinear simulations. Once the agents have learned enough, simulations are no longer needed and they can make their decisions "on the fly". The advantage of the proposed solution with respect to previously developed optimization methods in use in some grid control centers is their ultra-fast online computational performance. Thus, the flexibility of machine learning methods to perform exhaustive and fast analysis of a power system's security while considering a broader impact on the connected households makes them attractive to complement or even replace existing solutions. The work in this project aims not only to advance the development of the above mentioned methods but also to build proof-of-concept tools able to derive actionable information for power system operators to improve power system resiliency considering practical real-world constraints. This will be achieved by working together with two US utilities and using their grid models and measurements. If successful, the results of the project may lay the foundation of an entirely new approach for power system operation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

电网面临着越来越多的运营挑战，包括自然灾害、极端天气条件以及为实现零碳足迹目标而不断升级的复杂性。这些因素对电力系统运营商提出了挑战，他们需要决定采取何种行动，才能让大多数家庭用上电网，甚至避免大规模停电。尽管控制中心的技术和操作随着时间的推移不断升级，电网故障和停电仍然时有发生。当电网发生严重事件时，电力系统运行人员必须按照特定的程序解决不利条件，使系统恢复正常运行。考虑到上述挑战，有必要将选择最合适的补救措施的过程自动化。在这种情况下，决策的速度对于避免大规模断网和最小化不利事件对电网设备的影响至关重要。在现代计算机科学方法（如机器学习算法，更具体地说，是被称为强化学习的学习方法的子组）的帮助下，在实践中，响应时间甚至完全自动化的显著改进可能成为可能。强化学习通过选择提供最大利益的行为来获取知识，并已被证明可以解决复杂问题，例如训练机器人解决复杂任务，甚至玩复杂的游戏，如国际象棋和古老的围棋。利用强化学习方法，本研究中提出的解决方案允许我们根据对消费者的影响来选择减少潜在负面影响的行动，从而确定行动的优先级。这是通过在关键设施断开的风险较低的情况下为潜在操作分配更高的优先级来实现的。为了充分利用强化学习方法的优势，使用了多个“代理”（决策者）。这些代理可以分布在不同的网格设施中，并执行本地控制操作。然而，为了确保电网的弹性，它们之间存在一定程度的中心协调。每个智能体负责自己的控制区域，它可以根据在智能体学习阶段以动作形式提供的优先级指令进行操作。通过这种方式，我们的目标是通过结合通过超高保真非线性模拟学习到的关于电力系统的先验知识来增加智能体的可信度。一旦智能体掌握了足够的知识，就不再需要模拟，它们可以“在飞行中”做出决定。与一些电网控制中心使用的先前开发的优化方法相比，所提出的解决方案的优点是其超快的在线计算性能。因此，机器学习方法的灵活性可以对电力系统的安全性进行详尽和快速的分析，同时考虑到对联网家庭的更广泛影响，这使得它们具有补充甚至取代现有解决方案的吸引力。本项目的工作不仅旨在推进上述方法的发展，而且还旨在建立概念验证工具，这些工具能够为电力系统运营商提供可操作的信息，以考虑实际的现实世界限制，提高电力系统的弹性。这将通过与两家美国公用事业公司合作，并使用他们的网格模型和测量来实现。如果成功，该项目的成果可能为电力系统运行的全新方法奠定基础。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Developing a Campus Microgrid Model utilizing Modelica and the OpenIPSL Library

利用 Modelica 和 OpenIPSL 库开发校园微电网模型

• DOI: 10.1109/mscpes58582.2023.10123421
• 发表时间: 2023
• 期刊: Developing a Campus Microgrid Model utilizing Modelica and the OpenIPSL Library
• 作者: Fachini, Fernando;Pigott, Aisling;Laera, Giuseppe;Bogodorova, Tetiana;Vanfretti, Luigi;Baker, Kyri
• 通讯作者: Baker, Kyri