基于深度强化学习的交通信号控制方法

交通信号的智能控制是智能交通研究中的热点问题。为更加及时有效地自适应协调交通,文中提出了一种基于分布式深度强化学习的交通信号控制模型,采用深度神经网络框架,利用目标网络、双Q网络、价值分布提升模型表现。将交叉路口的高维实时交通信息离散化建模并与相应车道上的等待时间、队列长度、延迟时间、相位信息等整合作为状态输入,在对相位序列及动作、奖励做出恰当定义的基础上,在线学习交通信号的控制策略,实现交通信号Agent的自适应控制。为验证所提算法,在SUMO(Simulation of Urban Mobility)中相同设置下,将其与3种典型的深度强化学习算法进行对比。实验结果表明,基于分布式的深度强化学习算法在交通信号Agent的控制中具有更好的效率和鲁棒性,且在交叉路口车辆的平均延迟、行驶时间、队列长度、等待时间等方面具有更好的性能表现。

Traffic Signal Control Method Based on Deep Reinforcement Learning

The control of traffic signals is always a hotspot in intelligent transportation systems research.In order to adapt and coordinate traffic more timely and effectively,a novel traffic signal control algorithm based on distributional deep reinforcement learning was proposed.The model utilizes a deep neural network framework composed of target network,double Q network and value distribution to improve the performance.After integrating the discretization of the high-dimensional real-time traffic information at intersections with waiting time,queue length,delay time and phase information as states and making appropriate definitions of actions,rewards in the algorithm,it can learn the control strategy of traffic signals online and realize the adaptive control of traffic signals.It was compared with three typical deep reinforcement learning algorithms,and the experiments were performed in SUMO(Simulation of Urban Mobility)with the same setting.The results show that the distributional deep reinforcement learning algorithm is more efficient and robust,and has better performance on average delay,travel time,queue length,and wai-ting time of vehicles.