Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment
Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment
复制标题
互联自动化环境下基于强化学习的协同公路工区合并控制
DOI：
10.1177/0361198120935873
复制
发表时间：
2020-07-17
影响因子：
1.7
通讯作者：
Jiang, Liming
中科院分区：
文献类型：
--
作者：
Ren, Tianzhu;Xie, Yuanchang;Jiang, Liming
鉴于美国日益老化的基础设施和预计将增加的高速公路工区数量，研究工区合并控制非常重要，这对提高工区安全和通行能力至关重要。提出并评价了一种基于人工智能协同驾驶行为的高速公路工作区合并控制策略。提出的方法假设所有车辆都是完全自动化的、连接的和协作的。它在开放车道上插入两个计价区，为封闭车道上的车辆汇合腾出空间。此外，封闭车道上的每一辆车都学习如何根据周围的交通状况，使用非策略软行动者批评者强化学习(RL)算法来最优地调整其纵向位置，以在开放车道上找到安全间隙。学习结果在卷积神经网络中被捕获，并在测试阶段用于控制单个车辆。通过增加计量区并考虑周围车辆的位置、速度和加速度，隐含地考虑了车辆之间的合作。使用微观交通模拟器对基于RL的模型进行训练和评估。结果表明，这种基于RL的协作合并控制在移动性和安全性方面都明显优于延迟合并和提前合并等流行的合并控制策略。它的表现也好于假设所有车辆都配备了协作自适应巡航控制的策略。
Given the aging infrastructure and the anticipated growing number of highway work zones in the U.S.A., it is important to investigate work zone merge control, which is critical for improving work zone safety and capacity. This paper proposes and evaluates a novel highway work zone merge control strategy based on cooperative driving behavior enabled by artificial intelligence. The proposed method assumes that all vehicles are fully automated, connected, and cooperative. It inserts two metering zones in the open lane to make space for merging vehicles in the closed lane. In addition, each vehicle in the closed lane learns how to adjust its longitudinal position optimally to find a safe gap in the open lane using an off-policy soft actor critic reinforcement learning (RL) algorithm, considering its surrounding traffic conditions. The learning results are captured in convolutional neural networks and used to control individual vehicles in the testing phase. By adding the metering zones and taking the locations, speeds, and accelerations of surrounding vehicles into account, cooperation among vehicles is implicitly considered. This RL-based model is trained and evaluated using a microscopic traffic simulator. The results show that this cooperative RL-based merge control significantly outperforms popular strategies such as late merge and early merge in terms of both mobility and safety measures. It also performs better than a strategy assuming all vehicles are equipped with cooperative adaptive cruise control.