Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks.
Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks.
复制标题
工业园区用户端电池储能充放电策略的改进深度Q网络。
DOI：
10.3390/e23101311
复制
发表时间：
2021-10-06
期刊：
影响因子：
--
通讯作者：
Xiao W
中科院分区：
文献类型：
--
作者：
Chen S;Jiang C;Li J;Xiang J;Xiao W
蓄电池储能技术是工业园区保障稳定供电的重要环节，其粗糙的充放电模式难以满足节能减排、降本增效的应用要求。作为深度强化学习的经典方法，深度Q网络被广泛应用于解决用户侧电池储能充放电问题。在某些场景下，它的性能已经达到了人类专家的水平。然而，经验存储器中存储优先级的更新往往滞后于Q网络参数的更新。针对蓄电池充放电精益管理的需求，本文提出了一种改进的深度Q-网络，用于更新序列样本的优先级和深度神经网络的训练性能，降低了充放电动作成本和园区能耗。该方法考虑了实时电价、电池状态和时间等因素。设计了能量消耗状态、充放电行为、奖励函数、神经网络结构，满足充放电策略的灵活调度，最终能够实现电池储能效益的优化。该方法可以解决优先级更新滞后的问题，提高经验池样本的利用效率和学习性能。本文选取了美国和我国部分地区的电价数据进行仿真实验。实验结果表明，与传统算法相比，该方法在两种电价体系下均能取得更好的性能，从而大大降低了电池储能成本，为工业园区电池储能系统的安全稳定运行提供了更强有力的保障。
Battery energy storage technology is an important part of the industrial parks to ensure the stable power supply, and its rough charging and discharging mode is difficult to meet the application requirements of energy saving, emission reduction, cost reduction, and efficiency increase. As a classic method of deep reinforcement learning, the deep Q-network is widely used to solve the problem of user-side battery energy storage charging and discharging. In some scenarios, its performance has reached the level of human expert. However, the updating of storage priority in experience memory often lags behind updating of Q-network parameters. In response to the need for lean management of battery charging and discharging, this paper proposes an improved deep Q-network to update the priority of sequence samples and the training performance of deep neural network, which reduces the cost of charging and discharging action and energy consumption in the park. The proposed method considers factors such as real-time electricity price, battery status, and time. The energy consumption state, charging and discharging behavior, reward function, and neural network structure are designed to meet the flexible scheduling of charging and discharging strategies, and can finally realize the optimization of battery energy storage benefits. The proposed method can solve the problem of priority update lag, and improve the utilization efficiency and learning performance of the experience pool samples. The paper selects electricity price data from the United States and some regions of China for simulation experiments. Experimental results show that compared with the traditional algorithm, the proposed approach can achieve better performance in both electricity price systems, thereby greatly reducing the cost of battery energy storage and providing a stronger guarantee for the safe and stable operation of battery energy storage systems in industrial parks.
登录
查看更多内容
行人防撞与人机协同驾驶的深度强化学习
影响因子：
8.1
作者：
Li, Junxiang;Yao, Liang;Ren, Junkai
通讯作者：
Ren, Junkai
影响因子：
11.2
作者：
Ng, Kong Soon;Moo, Chin-Sien;Hsieh, Yao-Ching
通讯作者：
Hsieh, Yao-Ching
影响因子：
7.1
作者：
Jin, Hongyang;Teng, Yun;Chen, Zhe
通讯作者：
Chen, Zhe
DOI：
10.1109/tpds.2014.2372781
发表时间：
2015-12-01
影响因子：
5.3
作者：
Hung Khanh Nguyen;Song, Ju Bin;Han, Zhu
通讯作者：
Han, Zhu
影响因子：
--
作者：
Ji, Wei;Liu, Xiaoqing;Li, Tongchun
通讯作者：
Li, Tongchun