Reinforcement Learning-based Adaptive Trajectory Planning for AUVs in Under-ice Environments
Reinforcement Learning-based Adaptive Trajectory Planning for AUVs in Under-ice Environments
复制标题
DOI：
10.1109/oceans.2018.8604754
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
Chaofeng Wang;Li Wei;Zhaohui Wang;Min-je Song;N. Mahmoudian
中科院分区：
文献类型：
--
作者：
Chaofeng Wang;Li Wei;Zhaohui Wang;Min-je Song;N. Mahmoudian
本文研究了基于在线学习的多自主水下航行器（AUV）轨迹规划，以估计冰下环境中感兴趣的水参数场。被认为是一个集中式的系统，在冰层上的几个固定的接入点（AP）被引入AUV和远程数据融合中心（FC）之间的通信网关。我们将感兴趣的水参数场建模为具有未知超参数的高斯过程（GP）。AUV的采样轨迹是逐时期确定的。在每个历元结束时，AP将来自所有AUV的观测场样本中继到FC，FC基于高斯过程回归（GPR）计算场的后验分布并估计场超参数。所有AUV在下一个历元的最优轨迹被确定为最小化的长期成本是基于现场不确定性减少和AUV的移动性成本定义的，受到运动学约束，通信范围约束和传感区域约束。我们制定的自适应轨迹规划问题作为一个马尔可夫决策过程（MDP）。设计了一种基于强化学习（RL）的在线学习方法，用于确定受限连续空间中的最优AUV轨迹。仿真结果表明，所提出的基于学习的轨迹规划算法的性能类似于基准方法，假设完美的知识的领域超参数。
This work studies online learning-based trajectory planning for multiple autonomous underwater vehicles (AUVs) to estimate a water parameter field of interest in the under-ice environment. A centralized system is considered, where several fixed access points (APs) on the ice layer are introduced as gateways for communications between the AUVs and a remote data fusion center (FC). We model the water parameter field of interest as a Gaussian process (GP) with unknown hyper-parameters. The AUV trajectories for sampling are determined on an epoch-by-epoch basis. At the end of each epoch, the APs relay the observed field samples from all the AUVs to the FC which computes the posterior distribution of the field based on the Gaussian process regression (GPR) and estimates the field hyper-parameters. The optimal trajectories of all the AUVs in the next epoch are determined to minimize a long-term cost that is defined based on the field uncertainty reduction and the AUV mobility cost, subject to the kinematics constraint, the communication range constraint and the sensing area constraint. We formulate the adaptive trajectory planning problem as a Markov decision process (MDP). A reinforcement learning (RL)-based online learning method is designed to determine the optimal AUV trajectories in a constrained continuous space. Simulation results show that the proposed learning-based trajectory planning algorithm has performance similar to a benchmark method that assumes perfect knowledge of the field hyper-parameters.