CAREER: Active Learning of Second-Life Battery Systems by Combining Reinforcement Learning Principle and Device Physics

这个教师早期职业发展（CAREER）项目旨在开发一个变革性的主动学习框架，这对于实现二次电池和其他能源系统应用至关重要。退役电动汽车电池的再利用/再利用被认为是促进交通电气化和可再生能源发电的关键方法，这是新兴清洁能源革命的两个基石，对我们的国家繁荣至关重要。电动汽车电池必须以80%的剩余容量退役，直接处置将意味着剩余价值的大量浪费。将这些电池重新用于要求较低的固定存储，例如存储可再生但间歇性的太阳能/风能，可以显着提高电动汽车和可再生发电行业的成本和可持续性。重新利用面临的一个关键挑战是由于首次使用造成的损坏和退化而导致电池运行不安全/不健康的风险。因此，当前的再利用实践需要对每个退役电池模块进行繁琐且昂贵的手动测试和分级。然而，这种一次性测试只能在开始时保证安全和性能，而不能在随后的操作中保证。大规模电池再利用的关键推动因素是准确、快速、自动和连续估计电池状态和参数的先进技术。该项目的研究部分将与主题为“连接新兴ML/AI与传统控制”的教育计划相结合，以实现PI的总体教育目标，即在工程专业学生和专业人士，特别是代表性不足的少数民族中促进机器学习（ML）和人工智能（AI）教育。该研究将探索二次电池的主动学习，其中调节输入电流以优化响应电池电压的信息内容，从而提高估计的速度和准确度。关键的创新是一个通用的主动学习框架，它将强化学习的原理与设备物理学相结合，以克服当前主动学习实践中的根本局限性。这是第一次尝试使用强化学习进行估计，这在信息状态计算，奖励和学习架构设计以及收敛性分析方面提出了一系列基础研究问题。第二部分的研究重点是通过模块协作来提高多模块系统的学习性能。该奖项旨在通过实现模块间的数据共享和合作搜索，解决强化学习的基本挑战，例如需要大量训练数据和收敛缓慢。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This Faculty Early Career Development (CAREER) project aims at developing a transformative active learning framework, which is critical for enabling second-life battery and other energy system applications. Reuse/repurposing of retired electric vehicle (EV) batteries has been considered as a critical approach for facilitating transportation electrification and renewable power generation, which are two cornerstones of the emerging clean energy revolution essential to our national prosperity. EV batteries are mandated to retire with 80% remaining capacity, and direct disposal would mean a substantial waste of the remaining value. Repurposing these batteries for the less-demanding stationary storage, e.g. to store the renewable but intermittent solar/wind energy, could significantly improve the cost and sustainability of both EV and renewable power generation industries. A key challenge facing repurposing is the risk of unsafe/unhealthy battery operation due to the damage and degradation suffered from the first use. Therefore, current repurposing practice requires tedious and costly manual testing and grading of each retired battery module. Still, such one-time testing can only guarantee safety and performance at the beginning but not during subsequent operation. A key enabler for battery repurposing at a much larger scale is the advanced technique for accurate, fast, automatic, and continuous estimation of battery states and parameters. The research component of this project will be integrated with an education plan with the theme “Connecting Emerging ML/AI with Traditional Control” to achieve the PI’s overarching education goal of promoting machine learning (ML) and artificial intelligence (AI) education among engineering students and professionals, especially underrepresented minorities.The research will explore active learning of second-life batteries, where the input current is regulated to optimize the information content of the response battery voltage to improve the speed and accuracy of estimation. The key innovation is a generic active learning framework, which combines the principle of reinforcement learning with device physics to overcome the fundamental limitations in the current practice of active learning. This is among the first attempts to use reinforcement learning for estimation, which presents a series of fundamental research problems in information state computation, reward and learning architecture design, and convergence analysis. The second part of the research focuses on promoting the learning performance for multi-modular systems through module collaboration. The goal is to address fundamental challenges of reinforcement learning, e.g. the need for massive training data and slow convergence, by enabling data sharing and cooperative search among modules.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.