Enhanced Power System Resiliency through Adaptive Automatic Remedial Action Selection using Multi-Agent Reinforcement Learning

使用多智能体强化学习进行自适应自动补救措施选择，增强电力系统的弹性

• 批准号: 2231677
• 负责人: Luigi Vanfretti
• 金额: $ 38.3万
• 依托单位: Rensselaer Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2231677&HistoricalAwards=false
• 关键词: Enhanced; Power; System; Resiliency; through

Electrical power grids have been facing increased operational challenges, including those due to natural disasters, extreme weather conditions, and increasing complexity with the ongoing upgrades to achieve zero-carbon footprint goals. These factors challenge electrical power system operators in making decisions on which action to take in order to keep most of the households powered by the grid or even to avoid massive blackouts. Even though the control centers’ technology and their practices undergo upgrades over time, grid failures and blackouts still happen. When a severe event occurs on the grid, the power system operator has to follow specific procedures to resolve the unfavorable condition in order to bring the system back to normal operation. With the aforementioned challenges in mind, it becomes necessary to automate the process of choosing the most suitable remedial actions. In this case, the speed of decision-making is crucial to avoid massive disconnections and minimize the impact of unfavorable events on grid equipment. A significant improvement in response time and even full automation may become possible in practice with the help of modern computer science methods such as machine learning algorithms and, more specifically, its subgroup of learning methods that is known as reinforcement learning. Reinforcement learning acquires knowledge by choosing actions that provide the largest benefit and has been shown to solve complex problems, such as training robots to solve complicated tasks or even playing complex games such as chess and the ancient game of Go.Leveraging reinforcement learning methods, the proposed solution in this research allows us to prioritize an action according to its influence on consumers by choosing the action that reduces potential negative impacts. This is done by assigning higher priority to a potential action if the risk of disconnection of critical facilities is lower. To fully exploit the advantages of reinforcement learning methods, multiple “agents” (decision makers) are used. These agents can be distributed all across different grid facilities and perform local control actions. However, to assure grid resiliency, there is a degree of central coordination between them. Each agent is responsible for its own control area where it can operate according to the prioritized instructions that are provided in the form of actions during the agent’s learning phase. In such a way, we aim to increase the trustworthiness of the agents by incorporating prior knowledge about the power system that is learned via ultra-high-fidelity nonlinear simulations. Once the agents have learned enough, simulations are no longer needed and they can make their decisions "on the fly". The advantage of the proposed solution with respect to previously developed optimization methods in use in some grid control centers is their ultra-fast online computational performance. Thus, the flexibility of machine learning methods to perform exhaustive and fast analysis of a power system's security while considering a broader impact on the connected households makes them attractive to complement or even replace existing solutions. The work in this project aims not only to advance the development of the above mentioned methods but also to build proof-of-concept tools able to derive actionable information for power system operators to improve power system resiliency considering practical real-world constraints. This will be achieved by working together with two US utilities and using their grid models and measurements. If successful, the results of the project may lay the foundation of an entirely new approach for power system operation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

电力电网一直面临着增加的运营挑战，包括由于自然灾害，极端天气条件以及随着持续的升级而增加的复杂性，以实现零碳足迹目标。这些因素挑战了电力系统操作员，以决定要采取哪种行动，以使大多数家庭由电网提供动力，甚至避免大规模停电。即使控制中心的技术及其实践随着时间的推移而经历升级，但仍会发生网格故障和停电。当网格上发生严重的事件时，电力系统操作员必须遵循特定的程序来解决不利的状况，以使系统重新恢复正常运行。考虑到近似挑战，有必要自动选择最合适的补救措施的过程。在这种情况下，决策速度对于避免大规模断开连接至关重要，并最大程度地减少了不利事件对电网设备的影响。响应时间的重大改进，甚至完全自动化，在现代计算机科学方法（例如机器学习算法）的帮助下，更具体地说，更具体地说，是其被称为增强学习的学习方法的子组。强化学习通过选择提供最大收益的动作来获得知识，并已被证明可以解决复杂的问题，例如训练机器人解决复杂的任务，甚至玩复杂的游戏，例如国际象棋和古老的GO。掌握强化学习方法，这项研究中的拟议解决方案使我们能够根据其对消费者的影响来对消费者的影响进行优先考虑，从而对消费者进行潜在的影响，从而使潜在的影响重复影响。如果关键设施断开连接的风险较低，则可以通过将更高的优先级分配给潜在行动。为了充分利用强化学习方法的优势，使用了多种“代理”（决策者）。这些代理可以在各个不同的网格设施中分发并执行本地控制动作。但是，为了确保网格弹性，它们之间存在一定程度的中心协调。每个代理都负责其自己的控制区域，在该领域可以根据代理学习阶段以行动形式提供的优先说明进行操作。通过这种方式，我们旨在通过纳入有关通过超高限制非线性模拟学到的电力系统的先验知识来提高代理商的可信赖性。一旦代理人学到了足够多的知识，就不再需要模拟，他们可以“飞行”做出决定。在某些网格控制中心使用的先前开发的优化方法，该解决方案的优势是它们的超快速在线计算性能。这就是机器学习方法的灵活性，以对电源系统的安全性进行详尽而快速的分析，同时考虑对互联家庭产生更大影响，这使它们有吸引力，可以使它们有吸引力，甚至替代了现有的解决方案。该项目的工作不仅旨在推动上述方法的开发，还旨在构建能够为电力系统运营商提供可行信息的概念验证工具，以提高考虑实用现实世界约束的电力系统弹性。这将通过与两个美国公用事业一起使用并使用其网格模型和测量结果来实现。如果成功的话，该项目的结果可能奠定了全新的电力系统操作方法的基础。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响评估标准，被视为通过评估来获得珍贵的支持。

项目成果

Developing a Campus Microgrid Model utilizing Modelica and the OpenIPSL Library

利用 Modelica 和 OpenIPSL 库开发校园微电网模型

• DOI: 10.1109/mscpes58582.2023.10123421
• 发表时间: 2023
• 期刊: Developing a Campus Microgrid Model utilizing Modelica and the OpenIPSL Library
• 作者: Fachini, Fernando;Pigott, Aisling;Laera, Giuseppe;Bogodorova, Tetiana;Vanfretti, Luigi;Baker, Kyri
• 通讯作者: Baker, Kyri