Reinforcement Learning-based Fast Charging Control Strategy for Li-ion Batteries
Reinforcement Learning-based Fast Charging Control Strategy for Li-ion Batteries
复制标题
DOI：
10.1109/ccta41146.2020.9206314
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Saehong Park;Andrea Pozzi;Michael Whitmeyer;Won Tae Joe;D. Raimondo;S. Moura
中科院分区：
文献类型：
--
作者：
Saehong Park;Andrea Pozzi;Michael Whitmeyer;Won Tae Joe;D. Raimondo;S. Moura
锂离子电池社区面临的最关键挑战之一是如何在不损坏电池的情况下寻找最短的充电时间。这可以归结为根据电池模型求解大规模非线性最优控制问题。在此背景下，文献中提出了几种基于模型的技术。然而，这种策略的有效性受到模型复杂性和不确定性的极大限制。此外，难以跟踪与老化相关的参数并重新调整基于模型的控制策略。为了克服这些限制，本文提出了一种基于安全约束的快速充电策略，该策略依赖于无模型强化学习框架。特别地，我们重点研究了基于策略梯度的actor-critic算法，即深度确定性策略梯度（deep deterministic policy gradient， DDPG），以处理连续的动作集和集合。将简化后的电化学模型作为实际工厂进行仿真，验证了该方法的有效性。最后，通过考虑状态约简，突出了所提策略对环境参数变化的在线适应性。
One of the most crucial challenges faced by the Li-ion battery community concerns the search for the minimum time charging without damaging the cells. This can fall into solving large-scale nonlinear optimal control problems according to a battery model. Within this context, several model-based techniques have been proposed in the literature. However, the effectiveness of such strategies is significantly limited by model complexity and uncertainty. Additionally, it is difficult to track parameters related to aging and re-tune the model-based control policy. With the aim of overcoming these limitations, in this paper we propose a fast-charging strategy subject to safety constraints which relies on a model-free reinforcement learning framework. In particular, we focus on the policy gradient-based actor-critic algorithm, i.e., deep deterministic policy gradient (DDPG), in order to deal with continuous sets of actions and sets. The validity of the proposal is assessed in simulation when a reduced electrochemical model is considered as the real plant. Finally, the online adaptability of the proposed strategy in response to variations of the environment parameters is highlighted with consideration of state reduction.