A Deep Reinforcement Learning Network for Traffic Light Cycle Control
A Deep Reinforcement Learning Network for Traffic Light Cycle Control
复制标题
DOI：
10.1109/tvt.2018.2890726
复制
发表时间：
2019-02-01
影响因子：
6.8
通讯作者：
Han, Zhu
中科院分区：
文献类型：
--
作者：
Liang, Xiaoyuan;Du, Xunsheng;Han, Zhu
现有的交通信号灯周期控制效率低下，造成了大量的问题，如长时间的延误和浪费能源。为了提高效率，必须将实时交通信息作为输入，并根据实时交通信息动态调整交通灯的持续时间。现有的工程要么将交通信号分成相等的持续时间，要么只利用有限的交通信息。本文研究了如何根据不同传感器采集的数据来确定交通信号灯的持续时间。我们提出了一个深度强化学习模型来控制交通灯周期。在该模型中，我们通过收集交通数据并将整个交叉口划分为小网格，将复杂的交通场景量化为状态。交通灯的持续时间变化是行为，这是建模为一个高维马尔可夫决策过程。奖励是两个周期之间的累积等待时间差。为了求解该模型，采用卷积神经网络将状态映射到奖励。该模型结合了多个优化元素，如决斗网络，目标网络，双Q学习网络，优先经验重放，以提高性能。我们评估我们的模型，通过模拟城市流动性模拟器的模拟。仿真结果表明，我们的模型在控制交通灯的有效性。
Existing inefficient traffic light cycle control causes numerous problems, such as long delay and waste of energy. To improve efficiency, taking real-time traffic information as an input and dynamically adjusting the traffic light duration accordingly is a must. Existing works either split the traffic signal into equal duration or only leverage limited traffic information. In this paper, we study how to decide the traffic signal duration based on the collected data from different sensors. We propose a deep reinforcement learning model to control the traffic light cycle. In the model, we quantify the complex traffic scenario as states by collecting traffic data and dividing the whole intersection into small grids. The duration changes of a traffic light are the actions, which are modeled as a high-dimension Markov decision process. The reward is the cumulative waiting time difference between two cycles. To solve the model, a convolutional neural network is employed to map states to rewards. The proposed model incorporates multiple optimization elements to improve the performance, such as dueling network, target network, double Q-learning network, and prioritized experience replay. We evaluate our model via simulation on a Simulation of Urban MObility simulator. Simulation results show the efficiency of our model in controlling traffic lights.