Active Gamma-Ray Log Pattern Localization With Distributionally Robust Reinforcement Learning
Active Gamma-Ray Log Pattern Localization With Distributionally Robust Reinforcement Learning
复制标题
DOI：
10.1109/tgrs.2023.3278491
复制
发表时间：
2023
影响因子：
8.2
通讯作者：
Yuan Zi;Lei Fan;Xuqing Wu;Jiefu Chen;Shirui Wang;Zhu Han
中科院分区：
文献类型：
--
作者：
Yuan Zi;Lei Fan;Xuqing Wu;Jiefu Chen;Shirui Wang;Zhu Han
准确定位一维信号模式（如伽马射线测井深度匹配）在油田服务行业中至关重要，因为它直接影响石油和天然气勘探的质量。然而，传统的测井曲线分析和井网手工匹配等方法劳动强度大，严重依赖于人类的专业知识，导致结果不一致。虽然已经尝试自动化这个过程，如低计算性能，非鲁棒性和非泛化的挑战仍然没有解决。为了应对这些挑战，我们开发了一个数据驱动的人工智能系统，该系统可以学习受人类注意力启发的主动信号模式定位策略。我们的人工智能系统使用离线强化学习（RL）框架作为其核心组件，通过对人类标记的历史数据进行离线训练来解决高度抽象的马尔可夫决策过程（MDP）问题。RL代理使用自上而下的推理，通过使用简单的变换动作使边界窗口变形来确定目标信号片段的位置。为了克服日志数据和真实的数据之间的分布偏移并确保泛化，我们提出了一个离散分布鲁棒软演员-评论家（SAC）RL框架（DRSAC-Discrete）来解决不确定性下的MDP问题。通过以限制性的方式探索不熟悉的环境，DRSAC-Discrete算法提供了一种安全的解决方案，可在此工业应用的早期阶段数据有限时使用。我们评估了基于RL的定位系统在增强场伽马射线测井数据集，结果表明有前途的定位能力。此外，DRSAC-Discrete算法在面临数据短缺时表现出相对稳健的性能保证。
Accurately localizing 1-D signal patterns, such as Gamma-ray well-log depth matching, is crucial in the oilfield service industry as it directly affects the quality of oil and gas exploration. However, traditional methods such as well-log curve analysis and pattern hand-picking matching are labor-intensive and heavily rely on human expertise, leading to inconsistent results. Although attempts have been made to automate this process, challenges such as low computational performance, nonrobustness, and nongeneralization remain unsolved. To address these challenges, we have developed a data-driven AI system that learns an active signal pattern localization strategy inspired by human attention. Our artificial intelligence system uses an offline reinforcement learning (RL) framework as its central component, which solves a highly abstracted Markov decision process (MDP) problem via offline training on human-labeled historical data. The RL agent uses top-down reasoning to determine the location of target signal fragments by deforming a bounding window using simple transformation actions. To overcome distribution shifts between logged data and real and ensure generalization, we propose a discrete distributionally robust soft actor-critic (SAC) RL framework (DRSAC-Discrete) to solve the MDP problem under uncertainty. By exploring unfamiliar environments in a restrictive manner, the DRSAC-Discrete algorithm provides a safe solution that can be used when data is limited during the early stage of this industrial application. We evaluated the RL-based localization system on augmented field Gamma-ray well-log datasets, and the results showed promising localization capability. Furthermore, the DRSAC-Discrete algorithm demonstrated relatively robust performance guarantees when facing data shortage.