基于深度强化学习的无信号灯交叉路口车辆控制

利用深度强化学习技术实现无信号灯交叉路口车辆控制是智能交通领域的研究热点。现有研究存在无法适应自动驾驶车辆数量动态变化、训练收敛慢、训练结果只能达到局部最优等问题。文中研究在无信号灯交叉路口,自动驾驶车辆如何利用分布式深度强化方法来提升路口的通行效率。首先,提出了一种高效的奖励函数,将分布式强化学习算法应用到无信号灯交叉路口场景中,使得车辆即使无法获取整个交叉路口的状态信息,只依赖局部信息也能有效提升交叉路口的通行效率。然后,针对开放交叉路口场景中强化学习方法训练效率低的问题,使用了迁移学习的方法,将封闭的8字型场景中训练好的策略作为暖启动,在无信号灯交叉路口场景继续训练,提升了训练效率。最后,提出了一种可以适应所有自动驾驶车辆比例的策略,此策略在任意比例自动驾驶车辆的场景中均可提升交叉路口的通行效率。在仿真平台Flow上对TD3强化学习算法进行了验证,实验结果表明,改进后的算法训练收敛快,能适应自动驾驶车辆比例的动态变化,能有效提升路口的通行效率。     

自动驾驶车辆在无信号交叉口右转驾驶决策技术研究

利用深度强化学习（deep reinforcement learning,DRL）技术实现自动驾驶决策已成为国内外研究热点,现有研究中的车辆交通流缺乏随机性与真实性,同时自动驾驶车辆在环境中的有效探索具有局限性。因此利用TD3算法进行自动驾驶车辆在无信号交叉口下的右转驾驶决策研究,首先在Carla仿真平台中开发无信号交叉口的训练与测试场景,并添加交通流管理功能,提高系统训练和测试随机性。其次,为了提高自动驾驶车辆的探索性,对TD3算法中的Actor网络进行改进,为目标动作添加OU噪声。最后使用通行成功率和平均通行时间评估指标评价自动驾驶行为决策。结果表明,在不同交通流场景下,改进后的TD3算法通行成功率与基于DDPG算法控制的车辆相比平均提升6.2%,与基于规则的AEB模型相比平均提升23%。改进后的TD3算法不仅能够探索更多可能,而且其通行决策表现更加突出。     

自动驾驶交叉路口测试场景建模及验证方法

自动驾驶汽车在缓解交通拥堵和消除交通事故方面发挥着重要作用.为了保证自动驾驶系统的安全性和可靠性,在自动驾驶汽车部署到公共道路之前,必须进行全面的测试.现有的测试场景数据大多来源于交通事故和交通违法场景,而且自动驾驶系统最基本的安全需求就是遵守交通法规,这充分体现了自动驾驶汽车遵守交通规则的重要性.然而,目前严重缺少针对交通法规构建的自动驾驶测试场景.因此,本文从交通法规出发,根据自动驾驶系统安全需求,提出交叉路口测试场景的Petri网建模及形式化验证方法.首先,依据自动驾驶测试场景对交规进行分类,提取适合自动驾驶汽车的文本交规,并进行半形式化表征.其次,以覆盖道路交通安全法规以及测试场景功能测试规程为目标,融合交叉路口场景要素的交互行为,合理选择并组合测试场景要素,布设交叉路口测试场景.然后,基于交规的测试场景被建模为一个Petri网,其中,库所描述自动驾驶汽车的状态,变迁表示状态的触发条件,并选择时钟约束规范语言（CCSL）作为中间语义语言,将Petri网转换为一个可进行形式化验证的中间语义模型,提出具体的转换方法.最后,通过Tina软件分析验证交规场景模型的活性、有界性和可达性,结果表明所建模型的正确性,并基于SMT的分析工具MyCCSL来分析CCSL约束,采用LTL公式以形式化方法验证交规场景模型的一致性.     

Modeling of a multilane-multiple intersection based on queue theory and standard approach techniques

The currently available queue model for multilane traffic control is based on priority discipline. This priority discipline based technique is not suitable to be applied for multilane-multiple intersections, since all vehicles in each lane move simultaneously according to its respective signal phase. In view of this scenario, a new general traffic model for multilane-multiple intersections based on queue theory and standard techniques has been developed. The model framework used in this study is M/M/1 single server networks with arbitrarily-linked topology structure. A virtual server for each lane in multiple intersections is introduced in order to control the outgoing vehicles in each lane by their own server. In this study, the decomposition algorithm has been applied to deal with the feed-forward flows and finite buffer between intersections in an open queuing network model. This is achieved by decomposing the network into intersections, where each intersection was analyzed independently in order to obtain the overall model. A real case study has been conducted in one of the busiest streets in Kuala Lumpur. The study shows that the inter-arrival and inter-departure traffic flow follow the exponential distribution which also validates the chosen model. Simulation results show good correlation between the proposed models and real case studies.     

交叉口混合交通流微观控制模型

为构建智能网联汽车(CAV)和有人驾驶汽车(HDV)混合通行情况下的交叉口通行机制与控制方法, 本文提出CAV专用道条件下交叉口协同通行模型. 首先, 设计CAV专用道条件下的交叉口布置, 对交叉口进行网格化处理,将CAV通行时隙和HDV绿灯相位对交叉口某部分网格某时段的占用统一到交叉口时空资源描述框架下; 其次, 建立兼顾CAV与HDV的交叉口时空网格资源分配模型, 构建自适应信号灯控制算法和CAV轨迹规划算法; 再次, 以车辆最小延误为目标进行自适应信号灯配时优化和CAV轨迹优化; 最后, 选取广州某典型交叉口建立仿真实验对所提方法的有效性进行了验证.     

Behaviour based on decision matrices for a coordination between agents in a urban traffic simulation

This paper describes a multi-agent coordination mechanism applied to intersection simulation situations. In a goal of urban traffic simulation, we must consider the dynamic interactions between autonomous vehicles. The field of multi-agent systems provides us some studies for such systems, in particular on the coordination mechanisms. Conflicts between vehicles (i.e. agents) are very frequent in such applications, and they may cause deadlocks, particularly at intersections such as crossroads. Our approach is based on the solving of two player games/decision matrices which characterize three basic situations. An aggregation method generalizes to n-player games for complex crossroads. The objective of this approach consists in searching basic two-player matrices for solving n-agent problems. To explain the principle, we describe our approach for a particular case of crossroad with three agents. Finally, the obtained results have been examined via a tool of road traffic simulation, ARCHISIM. We assume also that the global traffic replicates the behavior of agents in different situations.     

自动驾驶环境下交叉口车辆路径规划与最优控制模型

自动驾驶环境下的交叉口基于车车/车路之间的双向信息交互, 能保障自动驾驶车辆相互穿插与协作地通过交叉口, 而无需信号灯控制. 因此, 如何设计高效的面向自动驾驶车辆通行的交叉口管控模型, 已成为研究的热点. 已有研究在建模时, 均基于自动驾驶车辆在交叉口内部的行驶路径已知并作为模型输入, 且大多对交叉口内部的冲突点进行简化. 本文首先将交叉口空间离散化处理, 考虑车辆的实际尺寸并面向非常规交叉口, 使用椭圆曲线建立转弯车辆行驶路径的精确轨迹方程, 再通过外边界投影降维法建立轨迹方程和交叉口空间的映射关系. 建立了基于混合整数线性规划(Mixed integer linear programming, MILP)的自动驾驶交叉口管控模型, 以交叉口总延误最小为控制目标, 同时优化车辆在交叉口的最佳行驶路径和驶入时刻, 使用AMPL (A mathematical programming language)对模型进行编译并使用CPLEX求解器求解. 与经典感应控制和先到先服务模型进行对比, 结果表明, 本文所提出模型能对车辆进入交叉口的时刻和行驶路径进行双重优化, 显著降低自动驾驶车辆通过交叉口的车均延误, 提高交叉口空间的利用效率.     

Towards load-balanced de-congested multi-robotic agent traffic control by coordinated control at intersections

This paper presents a methodology for the coordination of multiple robotic agents moving from one location to another in an environment embedded with a network of agents, placed at strategic locations such as intersections. These intersection agents, communicate with robotic agents and also with each other to route robots in a way as to minimize the congestion, thus resulting in the continuous flow of robot traffic. A robot’s path to its destination is computed by the network (in this paper, ‘Network’ refers to the collection of ‘Network agents’ operating at the intersections) in terms of the next waypoints to reach. The intersection agents are capable of identifying robots in their proximity based on signal strength. An intersection agent controls the flow of agent traffic around it with the help of the data it collects from the messages received from the robots and other surrounding intersection agents. The congestion of traffic is reduced using a two-layered hierarchical strategy. The primary layer operates at the intersection to reduce the time delay of robots crossing them. The secondary layer maintains coordination between intersection agents and routes traffic such that delay is reduced through effective load balancing. The objective at the primary level, to reduce congestion at the intersection, is achieved through assigning priorities to pathways leading to the intersection based on the robot traffic density. At the secondary level, the load balancing of robots over multiple intersections is achieved through coordination between intersection agents by communication of robot densities in different pathways. Extensive comparisons show the performance gain of the current method over existing ones. Theoretical analysis apart from simulation show the advantages of load-balanced traffic flow over uncoordinated allotment of robotic agents to pathways. Transferring the burden of coordination to the network releases more computational power for the robots to engage in critical assistive activities.     

基于机器视觉的交叉口特种车辆快速通行技术研究

交叉口是道路网络中重要的交通节点,容易产生交通堵塞问题,为了在保证通行安全的情况下提高特种车辆的通行效率,研究基于机器视觉的交叉口特种车辆快速通行技术。优化通行基础采用图像采集及预处理、检测识别和通行控制作为技术框架结构,利用机器视觉技术采集交叉口实时交通图像,通过图像滤波、图像增强等步骤,实现初始图像的预处理。利用Car-YOLO网络识别交叉口通行能力,规划快速通行路线,考虑前车行驶状态,求解特种车辆通行速度,针对车辆所占车道,通过绿灯早启、绿灯周期时间延长等方式控制交叉口信号灯,实现交叉口特种车辆快速通行。实验结果表明:在拥堵和正常通行场景下,优化设计技术的特种车辆通过时间的平均值分别为18.2s、10.1s,事故发生概率分别低于2%、1.4%,具有较好的应用效果。     

车路协同环境下智能交叉口车速控制

传统信号控制交叉口通过相位禁行方式来解决交通冲突,导致交叉口时空资源利用率低,延误增加.为提高车路协同环境下的交叉口通行效率,本文提出了一种基于时空间隙动态分配的智能交叉口车速控制方法,对交叉口车辆通行时间进行归一处理,建立了车辆跟驰控制模型和冲突避碰模型,使车辆能够根据交叉口控制区域内的实时路况信息预先进行速度调整,达到不停车通过交叉口的最优安全速度.运用VISSIM和MATLAB联合搭建仿真运行环境,分别在600 pcu·h^-1, 1200 pcu·h^-1和1800 pcu·h^-1交通流量条件下,对智能控制和传统控制交叉口的控制效益进行评价对比,结果表明:该智能控制方法均能够有效缩短交叉口车辆延误,减少车辆油耗和各类污染物的排放,且交通流量越高,改善效果越明显.     

结合状态机和动态目标路径的无人驾驶决策仿真

目的 决策系统是无人驾驶技术的核心研究之一。已有决策系统存在逻辑不合理、计算效率低、应用场景局限等问题,因此提出一种动态环境下无人驾驶路径决策仿真。方法 首先,基于规则模型构建适于无人驾驶决策系统的交通有限状态机;其次,针对交通动态特征,提出基于统计模型的动态目标路径算法计算状态迁移风险;最后,将交通状态机和动态目标路径算法有机结合,设计出一种基于有限状态机的无人驾驶动态目标路径模型,适用于交叉口冲突避免和三车道换道行为。将全速度差连续跟驰模型运用到换道规则中,并基于冲突时间提出动态临界跟车距离。结果 为验证模型的有效性和高效性,对交通环境进行虚拟现实建模,模拟交叉口通行和三车道换道行为,分析文中模型对车流量和换道率的影响。实验结果显示,在交叉口通行时,自主车辆不仅可以检测冲突还可以根据风险评估结果执行安全合理的决策。三车道换道时,自主车辆既可以实现紧急让道,也可以通过执行换道达成自身驾驶期望。通过将实测数据和其他两种方法对比,当车流密度在0.20.5时,本文模型的平均速度最高分别提高32 km/h和22 km/h。当车流密度不超过0.65时,本文模型的换道成功率最高分别提升37%和25%。结论 实验结果说明本文方法不仅可以在动态城区环境下提高决策安全性和正确性,还可以提高车流量饱和度,缓解交通堵塞。     

交通强度优先的交叉口模糊控制研究

为了降低交叉口车辆延误,提高通行能力,研究了一个四相位交叉口交通信号的模糊控制方法。用交通强度刻画各相位交通流通行需求的紧急程度,根据各相位的交通强度由模糊推理得到当前相位的绿灯延长时间,并选取后续绿灯相位。以交叉口车辆平均延误作为交叉口信号控制的性能评价指标,在相同交通条件下对几种控制方式进行了仿真试验。结果表明,该文的控制方法相对于感应控制方法和直接采用车辆排队长度作为输入的模糊控制方法,更能有效减小交叉口的车辆平均延误。     

Multi-criteria evolutionary optimization of a traffic light using genetics algorithm and teaching-learning based optimization

Today, the development of urbanization and increasing the number of vehicles has resulted in displeased consequences like traffic congestion and vehicle queuing. The vast majority of countries in the world encounter the challenge of the explosive rise in traffic demand. In this regard, it is necessary to meet traffic demand in transport networks, especially in metropolitans. In traffic management and shortening the trip duration, traffic lights on the signalized intersections play an essential role in urban pathways. This work provides a multi-criteria decision-making method for optimum traffic light control in an isolated corner. The main idea involves establishing a set of sub-optimal solutions for traffic light timing and selecting the best one among the diverse solutions. We have mathematically modelled the problem as an optimization problem to achieve an optimal solution with less waiting time for vehicles in intersections and the lowest cost. Genetic algorithm (GA) and Teaching-Learning-based Optimization (TLBO) are utilized for each phase to create a set of suitable timing scenarios. The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method is used to identify the best scenario, considering both waiting vehicles and traffic capacity as decision criteria. Its efficiency has been demonstrated over three different traffic volumes. Also, in a real-world implementation, its practical capability has been approved at a crossroads in Mashhad, Iran. The simulations indicate the improvement in the number of vehicles waiting behind the crossroad and the traffic capacity by 10% and 6.76% compared to the existing signal timing of the studied intersection, respectively.     

车路协同下的无信号交叉口车辆通行调度策略

无信号交叉口车辆通行调度问题是智能交通领域的研究重点,由于车辆通行顺序决策问题的解空间随着车辆数增加而指数级增长,在保证实时性的同时找到较优通行顺序成为无信号交叉口通行调度的一大问题。针对该问题提出一种基于自适应蒙特卡罗树搜索算法的无信号交叉口车辆通行调度方法,采用分层式框架,上层集中式顺序决策,下层分布式轨迹规划。首先,建立基于冲突点的交叉口模型,将网联车加入到待搜索队列中,根据交叉口中的车辆通行特点设计通行顺序的蒙特卡罗树搜索流程,以总通行时间为指标建立评价函数,然后针对不同交通情景设计自适应探索系数及其他超参数,使算法在求解不同车辆数时以及搜索的不同时期保持最佳状态。轨迹规划环节以加速度二范数为目标函数,以速度、加速度以及始终点位置等为约束,建立最优控制命题求解车辆轨迹。最后,进行实验,结果表明该算法相较于其他算法在数值仿真和微缩平台实验中最大优化幅度分别达到33.42%和38.04%,为无信号交叉口车辆通行调度提供了一个有效解决方案。     

CATS: An adaptive traffic signal system based on car-to-car communication

Traffic signal controls play an important role in regulating vehicular flow at road intersections. Traditional systems are not capable of adjusting the timing pattern in accordance with vehicular demand. This results in excessive delays for road users. Hence it is necessary to develop dynamic systems that can adjust the timing patterns according to traffic demand. In this paper, the design and implementation of an adaptive traffic signal control system based on car-to-car communication is presented. Also, a clustering algorithm is defined which will assist in estimating the density of vehicles approaching an intersection. The cycle time, which is calculated using the estimated density of vehicular traffic, helps in reducing both the waiting time for vehicles at intersections and queue length. It is also shown that the proposed solution is collision free at intersections. The proposed system is compared with a classic pre-timed system and an adaptive fuzzy logic system. The simulations also show that the data convergence time and the communication delay between vehicles and traffic signals do not compromise the efficiency of the system.     

Predictive car-following scheme for improving traffic flows on urban road networks

Driving behavior is one of the main reasons that causes bottleneck on the freeway or restricts the capacity of signalized intersections. This paper proposes a car-following scheme in a model predictive control (MPC) framework to improve the traffic flow behavior, particularly in stopping and speeding up of individual vehicles in dense urban traffic under a connected vehicle (CV) environment. Using information received through vehicle-to-vehicle (V2V) communication, the scheme predicts the future states of the preceding vehicle and computes the control input by solving a constrained optimization problem considering a finite future horizon. The objective function is to minimize the weighted costs due to speed deviation, control input, and unsafe gaps. The scheme shares the planned driving information with the following vehicles so that they can make better cooperative driving decision. The proposed car-following scheme is simulated in a typical driving scenario with multiple vehicles in dense traffic that has to stop at red signals in multiple intersections. The speeding up or queue clearing and stopping characteristics of the traffic using the proposed scheme is compared with the existing car-following scheme through numerical simulation.     

A Distributed Approach for Coordination of Traffic Signal Agents

Innovative control strategies are needed to cope with the increasing urban traffic chaos. In most cases, the currently used strategies are based on a central traffic-responsive control system which can be demanding to implement and maintain. Therefore, a functional and spatial decentralization is desired. For this purpose, distributed artificial intelligence and multi-agent systems have come out with a series of techniques which allow coordination and cooperation. However, in many cases these are reached by means of communication and centrally controlled coordination processes, giving little room for decentralized management. Consequently, there is a lack of decision-support tools at managerial level (traffic control centers) capable of dealing with decentralized policies of control and actually profiting from them. In the present work a coordination concept is used, which overcomes some disadvantages of the existing methods. This concept makes use of techniques of evolutionary game theory: intersections in an arterial are modeled as individually-motivated agents or players taking part in a dynamic process in which not only their own local goals but also a global one has to be taken into account. The role of the traffic manager is facilitated since s/he has to deal only with tactical ones, leaving the operational issues to the agents. Thus the system ultimately provides support for the traffic manager to decide on traffic control policies. Some application in traffic scenarios are discussed in order to evaluate the feasibility of transferring the responsibility of traffic signal coordination to agents. The results show different performances of the decentralized coordination process in different scenarios (e.g. the flow of vehicles is nearly equal in both opposing directions, one direction has a clearly higher flow, etc.). Therefore, the task of the manager is facilitate once s/he recognizes the scenario and acts accordingly.This revised version was published online in August 2005 with a corrected cover date.     

A multiagent systems with Petri Net approach for simulation of urban traffic networks

This paper presents a novel model framework for complex urban traffic systems based on the interconnection of a dynamical multi-agent system in a macroscopic level. The agents describe all the types of street segments, intersections, sources and sinks of cars, modelling the behavior of the flow of vehicles through them as simple differential equations. These agents include the phenomena of changes in the flow rate due to congestions, traffic signals and the density of the vehicles. Traffic signal changes are obtained by the evolution of Petri Nets, in order to represent a more real behavior. Therefore, a complex network can be constructed by the interconnection of the agents, in continuous time, and the Petri Nets, in a discrete-event behavior, becoming a hybrid and scalable system. In order to analyze the performance of the approach, a real set of streets and intersections in Montevideo City is studied. Also, the approach is compared with a simulation realized in the software TSIS-CORSIM, which contains real data of density of vehicles. The multi-agent system achieves comparable results, taking into account the differences in the level of details respect to TSIS-CORSIM. Thus, the results can represent the most important issues of vehicular traffic with less computational resources.     

A Distributed Approach for Coordination of Traffic Signal Agents

Innovative control strategies are needed to cope with the increasing urban traffic chaos. In most cases, the currently used strategies are based on a central traffic-responsive control system which can be demanding to implement and maintain. Therefore, a functional and spatial decentralization is desired. For this purpose, distributed artificial intelligence and multi-agent systems have come out with a series of techniques which allow coordination and cooperation. However, in many cases these are reached by means of communication and centrally controlled coordination processes, giving little room for decentralized management. Consequently, there is a lack of decision-support tools at managerial level (traffic control centers) capable of dealing with decentralized policies of control and actually profiting from them. In the present work a coordination concept is used, which overcomes some disadvantages of the existing methods. This concept makes use of techniques of evolutionary game theory: intersections in an arterial are modeled as individually-motivated agents or players taking part in a dynamic process in which not only their own local goals but also a global one has to be taken into account. The role of the traffic manager is facilitated since s/he has to deal only with tactical ones, leaving the operational issues to the agents. Thus the system ultimately provides support for the traffic manager to decide on traffic control policies. Some application in traffic scenarios are discussed in order to evaluate the feasibility of transferring the responsibility of traffic signal coordination to agents. The results show different performances of the decentralized coordination process in different scenarios (e.g. the flow of vehicles is nearly equal in both opposing directions, one direction has a clearly higher flow, etc.). Therefore, the task of the manager is facilitate once s/he recognizes the scenario and acts accordingly.     

Diagnosing resource usage failures in multi-agent systems

In the not-so-far future, autonomous vehicles will be ubiquitous and, consequently, need to be coordinated to avoid traffic jams and car accidents. A failure in one or more autonomous vehicles may break this coordination, resulting in reduced efficiency (due to traffic load) or even bodily harm (due to accidents). The challenge we address in this paper is to identify the root cause of such failures. Identifying the faulty vehicles in such cases is crucial in order to know which vehicles to repair to avoid future failures as well as for determining accountability (e.g., for legal purposes). More generally, this paper discusses multi-agent systems (MAS) in which the agents use a shared pool of resources and they coordinate to avoid resource contention by agreeing on a temporal resource allocation. The problem we address, called the Temporal Multi-Agent Resource Allocation (TMARA) diagnosis problem (TMARA-Diag), is to find the root cause of failures in such MAS that are caused by malfunctioning agents that use resources not allocated to them. As in the autonomous vehicles example, such failures may cause the MAS to perform suboptimally or even fail, potentially causing a chain reaction of failures, and we aim to identify the root cause of such failures, i.e., which agents did not follow the planned resource allocation. We show how to formalize TMARA-Diag as a model-based diagnosis problem and how to compile it to a set of logical constraints that can be compiled to Boolean satisfiability (SAT) and solved efficiently with modern SAT solvers. Importantly, the proposed solution does not require the agents to share their actual plans, only the agreed upon temporal resource allocation and the resources used at the time of failure. Such solutions are key in the development and success of intelligent, large, and security-aware MAS.