Comparison of Deep Reinforcement Learning and Model Predictive Control for Real-Time Depth Optimization of a Lifting Surface Controlled Ocean Current Turbine
Comparison of Deep Reinforcement Learning and Model Predictive Control for Real-Time Depth Optimization of a Lifting Surface Controlled Ocean Current Turbine
复制标题
升力面控制洋流涡轮机实时深度优化的深度强化学习与模型预测控制的比较
DOI：
10.1109/ccta48906.2021.9659089
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Sultan, Cornel
中科院分区：
文献类型：
--
作者：
Hasankhani, Arezoo;Tang, Yufei;VanZwieten, James;Sultan, Cornel
本文评估了两种策略，深度强化学习（DRL）和模型预测控制（MPC），通过深度优化最大限度地利用升力面控制洋流涡轮机（OCT）的功率。为了解决海流的时空不确定性，应用在线高斯过程（GP），其中海流速度的预测误差也被建模。我们比较了基于MPC的优化与基于DRL的算法（即，深Q网络（DQN））使用一个多星期的现场收集的声学多普勒海流剖面仪（ADCP）数据。在海流速度预测较好的情况下，基于DRL的算法与基于MPC的算法在实时优化方面几乎等价。然而，当考虑海流预测误差时，基于DQN的算法的性能优于基于MPC的算法。通过比较结果，验证了使用DQN在提高所提出的时空优化的容错性的重要性。
This paper evaluates two strategies, deep reinforcement learning (DRL) and model predictive control (MPC), for maximizing harnessed power from a lifting surface controlled ocean current turbine (OCT) through depth optimization. To address spatiotemporal uncertainties in the ocean current, an online Gaussian Process (GP) is applied, where the prediction error of the ocean current speed is also modeled. We compare the performance of the MPC-based optimization with the DRL-based algorithm (i.e., deep Q-networks (DQN)) using over one week of field collected acoustic doppler current profiler (ADCP) data. The DRL-based algorithm is almost equivalent to the MPC-based algorithm in real-time optimization when the ocean current speed prediction is perfect. However, the performance of the DQN-based algorithm surpasses the MPC-based algorithm when ocean current prediction error is considered. The importance of using the DQN in improving the error-tolerance of the proposed spatiotemporal optimization is verified through the comparative results.