Smart Underwater Pollution Detection Based on Graph-Based Multi-Agent Reinforcement Learning Towards AUV-Based Network ITS
Smart Underwater Pollution Detection Based on Graph-Based Multi-Agent Reinforcement Learning Towards AUV-Based Network ITS
复制标题
基于图多智能体强化学习的智能水下污染检测以及基于 AUV 的网络 ITS
DOI：
10.1109/tits.2022.3162850
复制
发表时间：
2022-04-05
影响因子：
8.5
通讯作者：
Peng, Yan
中科院分区：
文献类型：
--
作者：
Lin, Chuan;Han, Guangjie;Peng, Yan
海洋资源的开发利用和沿海城市城市化的快速发展，造成了严重的海洋污染，尤其是水下扩散污染。找出扩散污染源，减少污染的不利影响，是一项艰巨的任务。在6G框架的愿景下，我们采用了自主水下机器人(AUV)集群，并引入了基于AUV的网络概念。特别是，我们利用软件定义网络(SDN)技术来更新基于AUV的网络的可控性，从而产生了支持SDN的多AUV网络智能交通系统(sDNA-ITS)的范例。对于sDNA-ITS，我们利用人工势场理论对控制模型进行建模。为了优化系统的输出，我们引入了基于图的软行为者-批评者(SAC)算法，即一种多智能体强化学习(MARL)机制，其中每个AUV可以被视为图中的一个节点。特别是，我们在SDN控制器的辅助下改进了基于集中训练分散执行(CTDE)结构的优化模型，使每个AUV能够有效地调整其向扩散源的速度。此外，为了实现精确的路径规划以检测扩散源，提出了一种动态检测方案，输出联合控制策略对SDNA-ITS进行动态调度。仿真结果表明，在考虑实际场景的情况下，本文提出的方法能够有效地检测水下扩散源，且性能优于目前的一些研究成果。
The exploitation/utilization of marine resources and the rapid development of urbanization along coastal cities result in serious marine pollution, especially underwater diffusion pollution. It is a non-trivial task to detect the source of diffusion pollution, such that the disadvantageous effect of the pollution can be reduced. With the vision of 6G framework, we employ Autonomous Underwater Vehicle (AUV) flock and introduce the concept of AUV-based network. In particular, we utilize the Software-Defined Networking (SDN) technique to update the controllability of the AUV-based network, leading to the paradigm of SDN-enabled multi-AUVs network Intelligent Transportation Systems (SDNA-ITS). For SDNA-ITS, we utilize artificial potential field theories to model the control model. To optimize the system output, we introduce the graph-based Soft Actor-Critic (SAC) algorithm, i.e., a category of Multi-Agent Reinforcement Learning (MARL) mechanism where each AUV can be regarded as a node in a graph. In particular, we improve the optimization model based on Centralized Training Decentralized Execution (CTDE) architecture with the assistance of the SDN controller, by which each AUV can efficiently adjust its speed towards the diffusion source. Further, to achieve exact path planning for detecting the diffusion source, a dynamic detection scheme is proposed to output the united control policy to schedule the SDNA-ITS dynamically. Simulation results demonstrate that our approaches are available to detect the underwater diffusion source when the actual scenario is taken into account and perform better than some recent research products.