CAREER: Active Learning of Second-Life Battery Systems by Combining Reinforcement Learning Principle and Device Physics

这一学院早期职业发展(CALEAR)项目旨在开发一种变革性的主动学习框架，这对实现二次生命电池和其他能源系统应用至关重要。废弃电动汽车(EV)电池的再利用/再利用一直被认为是促进交通电气化和可再生能源发电的关键途径，这两个方面是新兴的清洁能源革命的两个基石，对我们的国家繁荣至关重要。电动汽车电池被要求在剩余容量为80%的情况下退役，直接处置将意味着对剩余价值的大量浪费。将这些电池重新用于要求较低的固定存储，例如存储可再生但间歇性的太阳能/风能，可以显著提高电动汽车和可再生发电行业的成本和可持续性。重新调整用途面临的一个关键挑战是，由于第一次使用造成损坏和降级，电池运行存在不安全/不健康的风险。因此，当前的再利用实践需要对每个退役电池模块进行繁琐且昂贵的手动测试和分级。尽管如此，这种一次性测试只能在开始时保证安全和性能，而不能在随后的操作中保证。电池在更大范围内重新利用的一个关键因素是对电池状态和参数进行准确、快速、自动和连续估计的先进技术。该项目的研究部分将与一个主题为“将新兴的ML/AI与传统控制相结合”的教育计划相结合，以实现PI的总体教育目标，即在工科学生和专业人士，特别是代表不足的少数群体中促进机器学习(ML)和人工智能(AI)教育。研究将探索第二次生命电池的主动学习，通过调节输入电流来优化响应电池电压的信息含量，以提高估计的速度和精度。关键的创新是一个通用的主动学习框架，它将强化学习的原理与设备物理相结合，以克服当前主动学习实践中的根本局限性。这是将强化学习用于估计的首次尝试之一，它在信息状态计算、奖励和学习结构设计以及收敛分析方面提出了一系列基础研究问题。研究的第二部分侧重于通过模块协作来提升多模块系统的学习绩效。目标是通过实现模块之间的数据共享和合作搜索来解决强化学习的根本挑战，例如需要大量的训练数据和缓慢的收敛。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This Faculty Early Career Development (CAREER) project aims at developing a transformative active learning framework, which is critical for enabling second-life battery and other energy system applications. Reuse/repurposing of retired electric vehicle (EV) batteries has been considered as a critical approach for facilitating transportation electrification and renewable power generation, which are two cornerstones of the emerging clean energy revolution essential to our national prosperity. EV batteries are mandated to retire with 80% remaining capacity, and direct disposal would mean a substantial waste of the remaining value. Repurposing these batteries for the less-demanding stationary storage, e.g. to store the renewable but intermittent solar/wind energy, could significantly improve the cost and sustainability of both EV and renewable power generation industries. A key challenge facing repurposing is the risk of unsafe/unhealthy battery operation due to the damage and degradation suffered from the first use. Therefore, current repurposing practice requires tedious and costly manual testing and grading of each retired battery module. Still, such one-time testing can only guarantee safety and performance at the beginning but not during subsequent operation. A key enabler for battery repurposing at a much larger scale is the advanced technique for accurate, fast, automatic, and continuous estimation of battery states and parameters. The research component of this project will be integrated with an education plan with the theme “Connecting Emerging ML/AI with Traditional Control” to achieve the PI’s overarching education goal of promoting machine learning (ML) and artificial intelligence (AI) education among engineering students and professionals, especially underrepresented minorities.The research will explore active learning of second-life batteries, where the input current is regulated to optimize the information content of the response battery voltage to improve the speed and accuracy of estimation. The key innovation is a generic active learning framework, which combines the principle of reinforcement learning with device physics to overcome the fundamental limitations in the current practice of active learning. This is among the first attempts to use reinforcement learning for estimation, which presents a series of fundamental research problems in information state computation, reward and learning architecture design, and convergence analysis. The second part of the research focuses on promoting the learning performance for multi-modular systems through module collaboration. The goal is to address fundamental challenges of reinforcement learning, e.g. the need for massive training data and slow convergence, by enabling data sharing and cooperative search among modules.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.