Optimal charging control of electrified public transportation based on deep reinforcement learning

加拿大各城市公共交通的电气化减少了温室气体(GHG)排放，改善了空气质量，减少了噪音污染，降低了维护和燃料成本，并为公众提供了更舒适的乘车体验。近年来，随着加拿大最大的城市正在努力到2050年实现车队完全净零，电动公交车的采用一直在迅速增长，一些城市的目标是最早在2036年和2040年实现。然而，城市公交车队的电气化对当前的电网容量提出了巨大的挑战，包括电动汽车S的充电需求与电网供应之间的电力不平衡，电压波动更大，以及更多的电力损失。适当协调的充电控制策略不仅可以缓解上述影响，还可以通过平坦化直接负荷、减少可再生能源削减、增加系统灵活性来造福智能电网。为此，电动公交车应该能够在非高峰时间充电，甚至在高峰时间向电网放电，以应对时变的电价。然而，由于电动公交车在发车前必须充分充电，由于随机交通条件导致的电价和到达/发车时间的不确定性，实时优化充电控制是具有挑战性的。近年来，深度强化学习(DRL)作为一种在不确定环境下进行连续决策的高效框架得到了长足的发展。DRL可以直接从真实世界的数据中通过试错学习最优策略，而不需要对随机性的分布进行建模。这一合作伙伴关系的目标是开发用于电动公交车最佳充电控制的DRL算法，该算法将(1)最大限度地减少温室气体排放；(2)最大限度地降低充电成本，同时确保公共交通公交车的可靠运行；(3)平坦化充电负荷并增强电网的可再生能源调度能力。这一合作伙伴关系的预期结果将为加拿大平稳高效地过渡到电气化公共交通网络提供创新的解决方案和有用的见解。

Electrification of public transport across Canadian cities reduces Greenhouse Gas (GHG) emissions, improves air quality and decreases noise pollution, lowers maintenance and fuel costs, and provides the public with a more comfortable riding experience. In recent years, the adoption of electric buses has been growing rapidly as Canada's largest cities are working to have fully net-zero fleets by 2050, with some aiming for as early as 2036 and 2040. However, the electrification of urban bus fleets presents significant challenges to current grid capacity, which include a power imbalance between Electric Vehicle (EV)s' charging demand and the power grid's supply, larger voltage fluctuation, and more power loss. Properly coordinated charging control strategies can not only alleviate the above impacts, but also benefit the smart grid by flattening direct load, reducing renewable generation curtailment, and increasing system flexibility. For this purpose, electric buses should be able to charge during off-peak hours and even discharge energy to the grid during peak hours in response to time-varying electricity prices. However, as the electric buses must be sufficiently charged before departure, it is challenging to optimize their charging control in real-time due to the uncertainty in electricity prices and in their arrival/departure time resulting from random traffic conditions. In recent years, Deep Reinforcement Learning (DRL) has made great progress as an efficient and effective framework to make sequential decisions under uncertainty. DRL can directly learn an optimal policy by trial and error from real-world data, and there is no need to model the distribution of the randomness. The goal of this partnership is to develop DRL algorithms for optimal charging control of electric buses that will (1) minimize GHG emissions; (2) minimize charging cost while ensuring reliable operation of buses for public transportation; and (3) flatten charging load and enhance renewable generation dispatchability for power grid. The expected outcome of this partnership will provide innovative solutions and useful insights for a smooth and efficient transition toward an electrified public transit network in Canada.