Machine learning based electric vehicle charging management system for smart grid applications

电动汽车（EV）的需求正在迅速增长，预计到2030年电动汽车将占新车销量的60%。增长受到政府政策的推动，符合净零目标，增加消费者选择和支持基础设施的可用性。然而，在全球范围内，需要大规模的充电基础设施来支持电动汽车需求的增长，预计到2040年将需要高达1万亿美元的投资。随着净零目标也推动了可再生基本负荷电力的采用，电网面临着管理可再生能源供应带来的不稳定性的挑战。电动汽车车队作为需求（电网到车辆（G2 V））和供应（车辆到电网（V2 G））的来源，提供了帮助电网平衡的潜力，潜在地减少了对电网规模储能的需求。为了在这种情况下管理电网平衡，准确预测需求和供应的能力至关重要。人工智能和机器学习技术可以帮助从变量之间具有复杂关系的大规模数据集中获得模式。此外，人类行为模型越来越多地被整合，以帮助理解和预测电动汽车车主的行为。该项目将试图证明这样一个假设，即如果所有者得到适当的激励，电动汽车有可能有意义地帮助电网平衡。该研究的一个关键目标是建立一个基于人工智能的模型，在整体电网供应、需求和电网平衡策略的背景下模拟电动汽车充电和放电活动-该模型将允许用户模拟不同的场景，并衡量电动汽车车主对不断变化的条件的反应，包括整合合适的人类行为模型。该项目旨在建立在迄今为止的研究基础上，并了解电动汽车大规模参与电网平衡的任何障碍以及如何克服这些障碍。该研究可能会引起一系列利益相关者的兴趣，包括电网运营商，能源公司，电动汽车公司和充电站运营商。这项工作也有可能向其他领域研究人工智能技术和人类行为建模相结合的人学习，并引起他们的兴趣。迄今为止的研究表明，基于强化学习（RL）的方法越来越多地应用于电动汽车充电场景。与传统的基于模型的优化相比，RL具有许多优点，更适合电力网络建模的复杂性，动态性和随机性。所提出的方法是获得电网供需数据，沿着电动汽车使用和充电数据，并开发多车辆深度强化学习模型，电动汽车所有者寻求在其环境中最大化其效用，并使用认知模型（例如期望效用理论和前景理论）被构建到在RL奖励函数中感知奖励（例如与范围、成本和时间相关的效用）的方式中。随着道路上电动汽车数量的增加和数据可用性的提高，有可能完成对充电行为的真实世界研究，将结果整合到RL奖励功能的工作中，以增加现实性。如果模型的开发具有一定的现实性，这将使用户能够测试对不同情景的反应，并能够调整一系列变量。迄今为止的研究表明，在提高集成到RL方法中的行为建模水平方面可以有新奇，例如通过将累积前景理论和更先进的认知模型集成到RL奖励函数中。此外，未来的电网预计将包括飞行器和其他形式的交通工具，以及地面电动汽车;到目前为止，对飞行器充电行为方面的研究有限。

Demand for electric vehicles (EVs) is growing rapidly, with EVs forecast to account for as much as 60% of new car sales by 2030. Growth is being driven by government policy, in line with net zero targets, increasing consumer choice and availability of supporting infrastructure. Large scale charging infrastructure scale up is required however, globally, to support EV demand growth and is forecast to require up to $1 trillion of investment by 2040. With net zero targets also driving increased renewable baseload power adoption, grids have the challenge of managing the intermittency that comes with renewable energy supply. Electric vehicle fleets provide the potential to assist grid balancing as sources of both demand (grid to vehicle (G2V)) and supply (vehicle to grid (V2G)), potentially reducing the need for grid scale energy storage. In order to manage grid balancing in this context, the ability to accurately forecast demand and supply will be crucial. AI and machine learning techniques can assist in this area to derive patterns from large scale datasets with complex relationships between variables. Also, human behavioural models are increasingly being integrated, to assist the understanding and prediction of EV owner behaviours. This project will seek to prove the hypothesis that EVs have the potential to meaningfully assist grid balancing if owners are appropriately incentivised. A key objective of the research will be to produce an AI based model that simulates EV charging and discharging activity in the context of overall electricity grid supply, demand and grid balancing strategies - the model will allow users to simulate different scenarios and gauge EV owner responses to changing conditions, including the integration of suitable human behavioural models. The project will aim to build on research to date and to understand any barriers to the large-scale involvement of EVs in grid balancing and how these may be overcome. The research has the potential to be of interest to a range of stakeholders, including grid operators, energy companies, EV companies and charging station operators. There is also the potential for the work to both learn from and be of interest to those in other fields studying the combination of AI techniques and human behavioural modelling. Research to date shows that reinforcement learning (RL) based methods are increasingly being applied to EV charging scenarios. RL has a number of benefits over conventional model-based optimisation, being more suited to the complexity, dynamism and randomness of power network modelling. The proposed approach is to obtain grid supply and demand data, along with EV usage and charging data and to develop a multi-vehicle, deep reinforcement learning model, where EV owners seek to maximise their utility within their environment, with cognitive models (such as expected utility theory and prospect theory) built into the way rewards (such as utility relating to range, cost and time) are perceived in the RL reward function. With increasing numbers of EVs on the road and increasing availability of data, there is the potential to complete a real-world study into charging behaviours, integrating the results into the workings of the RL reward function, to add realism. If the model can be developed with some realism, this will allow users to test responses to differing scenarios, with the ability to adjust a range of variables. Research to date shows that there can be novelty in advancing the level of behavioural modelling integrated into RL methods, for example by integrating cumulative prospect theory and more advanced cognitive models into the RL reward function. Also, the future grid is expected to include aerial vehicles and other forms of transport, as well as ground-based electric vehicles; research into the behavioural aspects of aerial vehicle charging has so far been limited.