Multi-level Reinforcement Learning for flow control

流量控制的多级强化学习

• 批准号: EP/V048899/1
• 负责人: Ahmed Elsheikh
• 金额: $ 25.79万
• 依托单位: Heriot-Watt University
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FV048899%2F1
• 关键词: Multi; level; Reinforcement; Learning; flow

Flow control is the process of targeted manipulation of fluid flow fields to accomplish a prescribed objective (e.g. reduce drag). Flow control uses information from the flow (provided by sensors) to adapt to incoming perturbations and adjust to changing flow conditions. General flow control is a largely unsolved mathematical problem appearing in many industries, including automotive, aerospace and environmental subsurface flow problems. The missing ingredient for turning flow control into a practical tool is the development of general flow control algorithms that can handle the following: (a) uncertainties in the system perturbations (e.g. the speed and direction of the perturbation), (b) uncertainties in the flow model parameters, (c) sparsity of the observations (i.e. partial and noisy observations) (d) modelling errors due to discretization and parameter upscaling.In this proposal, Reinforcement Learning (RL) algorithms will be utilized to learn general flow control polices using reliable simulated flow environments. From an application point of view, the developed mathematical techniques address flow control in two applications: (a) increasing energy efficiency in transportation trucks by flow control of incompressible Navier-Stokes flow past an obstacle and (b) safe and efficient storage of anthropogenic carbon dioxide (CO2) in deep geological formations using flow control in a Darcy-type subsurface flow. For the first application, road freight transportation accounts for approximately 5% of the UK's carbon footprint and flow control to reduce the aerodynamic drag could significantly improve the fuel efficiency, for example a 15% reduction in drag is equivalent to about 5% in fuel savings. For the CO2 storage application, the produced CO2 by human activities, for example from a power stations or an energy-intensive industries, could be injected into deep saline aquifers as a possible mitigation strategy to reduce anthropogenic emissions of carbon dioxide into the atmosphere. The control of injection strategies in the subsurface storage sites, given the inherent uncertainties in the subsurface properties, would minimize the risk of leakage while maximising the storage capacity.

流量控制是有针对性地操纵流体流场以实现规定目标（例如，减小阻力）的过程。流量控制使用来自流量的信息（由传感器提供）来适应传入的扰动并调整以改变流量条件。一般流动控制是出现在许多行业中的一个很大程度上未解决的数学问题，包括汽车、航空航天和环境地下流动问题。将流控制转变为实用工具所缺少的要素是开发可以处理以下内容的通用流控制算法：（a）系统扰动的不确定性（例如扰动的速度和方向），（B）流动模型参数的不确定性，（c）观察结果的稀疏性（即部分观测和噪声观测）（d）由于离散化和参数放大而产生的建模误差。在这一建议中，强化学习（RL）算法将被用来学习一般的流量控制策略，使用可靠的模拟流环境。从应用的角度来看，所开发的数学技术解决了两个应用中的流量控制：（a）通过对经过障碍物的不可压缩纳维尔-斯托克斯流进行流量控制来提高运输卡车的能量效率，以及（B）使用达西型地下流中的流量控制来安全有效地将人为二氧化碳（CO2）储存在深层地质构造中。对于第一种应用，公路货运占英国碳足迹的约5%，并且用于减少空气动力阻力的流量控制可以显著提高燃料效率，例如，阻力减少15%相当于燃料节省约5%。对于CO2储存应用，人类活动产生的CO2，例如发电站或能源密集型工业产生的CO2，可以注入深层盐水层，作为一种可能的缓解战略，以减少人为排放到大气中的二氧化碳。考虑到地下性质的固有不确定性，地下储存地点的注入策略控制将最大限度地减少泄漏风险，同时最大限度地提高储存能力。

项目成果

Stochastic optimal well control in subsurface reservoirs using reinforcement learning

• DOI: 10.1016/j.engappai.2022.105106
• 发表时间: 2022-07
• 期刊: ArXiv
• 作者: A. Dixit;A. Elsheikh

Robust Well-Production Control Using Surrogate Assisted Reinforcement Learning

使用替代辅助强化学习的鲁棒油井生产控制

• DOI: 10.3997/2214-4609.202244101
• 发表时间: 2022
• 作者: Dixit A

Robust Optimal Well Control using an Adaptive Multigrid Reinforcement Learning Framework

• DOI: 10.1007/s11004-022-10033-x
• 发表时间: 2022-11-04
• 期刊: MATHEMATICAL GEOSCIENCES
• 影响因子: 2.6
• 作者: Dixit, Atish;Elsheikh, Ahmed H.
• 通讯作者: Elsheikh, Ahmed H.

Gym-preCICE: Reinforcement learning environments for active flow control

Gym-preCICE：用于主动流量控制的强化学习环境

• DOI: 10.1016/j.softx.2023.101446
• 发表时间: 2023
• 期刊: SoftwareX
• 影响因子: 3.4
• 作者: Shams M