Underwater Equipotential Line Tracking Based on Self-Attention Embedded Multiagent Reinforcement Learning Toward AUV-Based ITS
Underwater Equipotential Line Tracking Based on Self-Attention Embedded Multiagent Reinforcement Learning Toward AUV-Based ITS
复制标题
基于自注意嵌入式多智能体强化学习的水下等势线跟踪
DOI：
10.1109/tits.2022.3202225
复制
发表时间：
2023-08
影响因子：
8.5
通讯作者：
Chuan Lin;Guangjie Han;Qiuzi Tao;Li Liu;Syed Bilal Hussain Shah;Tongwei Zhang;Zhenglin Li
中科院分区：
文献类型：
--
作者：
Chuan Lin;Guangjie Han;Qiuzi Tao;Li Liu;Syed Bilal Hussain Shah;Tongwei Zhang;Zhenglin Li
智能水下设备的快速发展促进了海洋资源开发、海洋目标跟踪等海洋开发活动的开展。本文将介绍如何利用自主水下航行器（AUV）群或多AUV系统来跟踪水下扩散污染物，特别是特定浓度的等势线。与目前大多数研究不同的是，本文将AUV作为一个网络系统，利用软件定义网络（SDN）技术对网络结构进行优化，构建了一个支持SDN的AUV网络智能交通系统（ITS）。利用SDN技术的集中管理能力，提出了基于图的软演员-评论（SAC）算法的软件定义集中训练分散执行（CTDE）架构，以优化系统的控制和管理。为了提高计算和训练效率，我们将自注意机制嵌入到评论网络的构造中，从而得到一种基于自注意的SAC算法。评估结果表明，我们提出的方法是能够准确地跟踪一个特定的浓度在许多类别（具有不同类型的等位线（包括形状，噪声，和扩散值））的水下扩散场的等位线。同时，我们提出的方法优于一些经典的计划在系统奖励，跟踪误差等。
The rapid development of intelligent underwater devices promotes marine exploitation activities, including marine resource exploitation, marine target tracking, etc. This work will present how to utilize the Autonomous Underwater Vehicle (AUV) swarm or multi-AUVs system to track the underwater diffusion pollution, especially the equipotential line of particular concentration. Different from most of the current research, in this work, we take the AUV swam as a network system and utilize the Software-Defined Networking (SDN) technique to optimize the network architecture, constructing an SDN-enabled AUV network Intelligent Transportation Systems (ITS). With the centralized management ability of the SDN technique, we propose the software-defined Centralized Training Decentralized Execution (CTDE) architecture based on the graph-based Soft Actor-Critic (SAC) algorithm to optimize the system control and management. To improve the computing and training efficiency, we embed the self-attention mechanism into the critic network construction, leading to a self-attention-based SAC algorithm. Evaluation results demonstrate that our proposed approach is able to exactly track the equipotential lines of a particular concentration in many categories (with different types of equipotential lines (including the shape, noise, and diffusion value)) of underwater diffusion fields. Meanwhile, our proposed approaches outperform some classical schemes in system awards, tracking errors, etc.