Deep Reinforcement Learning-Based Energy Storage Arbitrage With Accurate Lithium-Ion Battery Degradation Model
Deep Reinforcement Learning-Based Energy Storage Arbitrage With Accurate Lithium-Ion Battery Degradation Model
复制标题
DOI：
10.1109/tsg.2020.2986333
复制
发表时间：
2020-04
影响因子：
9.6
通讯作者：
Jun Cao;Daniel J. B. Harrold;Zhong Fan;Thomas Morstyn;David Healey;Kang Li
中科院分区：
文献类型：
--
作者：
Jun Cao;Daniel J. B. Harrold;Zhong Fan;Thomas Morstyn;David Healey;Kang Li
电池退化成本的准确估算是电池参与能源套利市场的主要障碍之一。本文通过使用无模型深度强化学习（DRL）方法来解决这个问题，以优化电池能量套利，同时考虑准确的电池退化模型。首先，将控制问题表示为马尔可夫决策过程（MDP）。然后提出了一种基于噪声网络的深度强化学习方法来学习存储充电/放电策略的优化控制策略。为了解决电价的不确定性，采用混合卷积神经网络（CNN）和长短期记忆（LSTM）模型来预测第二天的电价。最后，所提出的方法进行了测试的历史英国。电力批发市场价格。与基于模型的混合线性规划（MILP）的结果进行了比较，证明了所提出的框架的有效性和性能。
Accurate estimation of battery degradation cost is one of the main barriers for battery participating on the energy arbitrage market. This paper addresses this problem by using a model-free deep reinforcement learning (DRL) method to optimize the battery energy arbitrage considering an accurate battery degradation model. Firstly, the control problem is formulated as a Markov Decision Process (MDP). Then a noisy network based deep reinforcement learning approach is proposed to learn an optimized control policy for storage charging/discharging strategy. To address the uncertainty of electricity price, a hybrid Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) model is adopted to predict the price for the next day. Finally, the proposed approach is tested on the historical U.K. wholesale electricity market prices. The results compared with model based Mixed Integer Linear Programming (MILP) have demonstrated the effectiveness and performance of the proposed framework.