Opportunities for multiagent systems and multiagent reinforcement learning in traffic control

The increasing demand for mobility in our society poses various challenges to traffic engineering, computer science in general, and artificial intelligence and multiagent systems in particular. As it is often the case, it is not possible to provide additional capacity, so that a more efficient use of the available transportation infrastructure is necessary. This relates closely to multiagent systems as many problems in traffic management and control are inherently distributed. Also, many actors in a transportation system fit very well the concept of autonomous agents: the driver, the pedestrian, the traffic expert; in some cases, also the intersection and the traffic signal controller can be regarded as an autonomous agent. However, the “agentification” of a transportation system is associated with some challenging issues: the number of agents is high, typically agents are highly adaptive, they react to changes in the environment at individual level but cause an unpredictable collective pattern, and act in a highly coupled environment. Therefore, this domain poses many challenges for standard techniques from multiagent systems such as coordination and learning. This paper has two main objectives: (i) to present problems, methods, approaches and practices in traffic engineering (especially regarding traffic signal control); and (ii) to highlight open problems and challenges so that future research in multiagent systems can address them.     

A multiagent approach to managing air traffic flow

Intelligent air traffic flow management is one of the fundamental challenges facing the Federal Aviation Administration (FAA) today. FAA estimates put weather, routing decisions and airport condition induced delays at 1,682,700 h in 2007 (FAA OPSNET Data, US Department of Transportation website, ), resulting in a staggering economic loss of over $41 billion (Joint Economic Commission Majority Staff, Your flight has been delayed again, 2008). New solutions to the flow management are needed to accommodate the threefold increase in air traffic anticipated over the next two decades. Indeed, this is a complex problem where the interactions of changing conditions (e.g., weather), conflicting priorities (e.g., different airlines), limited resources (e.g., air traffic controllers) and heavy volume (e.g., over 40,000 flights over the US airspace) demand an adaptive and robust solution. In this paper we explore a multiagent algorithm where agents use reinforcement learning (RL) to reduce congestion through local actions. Each agent is associated with a fix (a specific location in 2D space) and has one of three actions: setting separation between airplanes, ordering ground delays or performing reroutes. We simulate air traffic using FACET which is an air traffic flow simulator developed at NASA and used extensively by the FAA and industry. Our FACET simulations on both artificial and real historical data from the Chicago and New York airspaces show that agents receiving personalized rewards reduce congestion by up to 80% over agents receiving a global reward and by up to 90% over a current industry approach (Monte Carlo estimation).     

A multiagent systems with Petri Net approach for simulation of urban traffic networks

This paper presents a novel model framework for complex urban traffic systems based on the interconnection of a dynamical multi-agent system in a macroscopic level. The agents describe all the types of street segments, intersections, sources and sinks of cars, modelling the behavior of the flow of vehicles through them as simple differential equations. These agents include the phenomena of changes in the flow rate due to congestions, traffic signals and the density of the vehicles. Traffic signal changes are obtained by the evolution of Petri Nets, in order to represent a more real behavior. Therefore, a complex network can be constructed by the interconnection of the agents, in continuous time, and the Petri Nets, in a discrete-event behavior, becoming a hybrid and scalable system. In order to analyze the performance of the approach, a real set of streets and intersections in Montevideo City is studied. Also, the approach is compared with a simulation realized in the software TSIS-CORSIM, which contains real data of density of vehicles. The multi-agent system achieves comparable results, taking into account the differences in the level of details respect to TSIS-CORSIM. Thus, the results can represent the most important issues of vehicular traffic with less computational resources.     

A verifiable simulation model for real-world microscopic traffic simulations

This article presents a scientific discussion about the ongoing progress in the development of traffic simulation system platforms. As part of the discussion, the presentation introduces a simulation model that is based on the fully functional, real-world online traffic information system OLSIMv4 which is the updated version of the traffic information platform for the highway network of North Rhine-Westphalia (Germany). The simulation model consists of a simulation engine and a combination of several subject-specific model families such as vehicle models, microscopic traffic models, detector models, and tuning element models. Additionally, it provides a data model for arbitrary road traffic networks in highway and urban environments.The presentation includes a demonstration of how to form and initialize all relevant system components by providing an example for each component. The demonstrations use the declarative programming language Maude to form and initialize the components due to its simplicity and expressive power. The components facilitate their enhancement by a verification and validation management approach. The goal of the enhancement effort is to optimize the further development of the underlying OLSIMv4 system. In addition, the presented methodology stands exemplarily for the design and implementation of a whole class of systems. Additionally, the definitions of the simulation model can be used as a specification for an implementation with sequential, parallel, and distributed operation. Therein, independent entities can be inferred automatically by the simulation engine as part of an automatic domain decomposition. It has been implemented as a sequential and a parallel simulation that exploits CPU thread-level parallelism.     

Two methods for computing bounds for the distribution of cumulative reward for large Markov models

Degradable fault-tolerant systems can be evaluated using rewarded continuous-time Markov chain (CTMC) models. In that context, a useful measure to consider is the distribution of the cumulative reward over a time interval [0,t]. All currently available numerical methods for computing that measure tend to be very expensive when the product of the maximum output rate of the CTMC model and t is large and, in that case, their application is limited to CTMC models of moderate size. In this paper, we develop two methods for computing bounds for the cumulative reward distribution of CTMC models with reward rates associated with states: BT/RT (Bounding Transformation/Regenerative Transformation) and BT/BRT (Bounding Transformation/Bounding Regenerative Transformation). The methods require the selection of a regenerative state, are numerically stable and compute the bounds with well-controlled error. For a class of rewarded CTMC models, class , and a particular, natural selection for the regenerative state the BT/BRT method allows us to trade off bound tightness with computational cost and will provide bounds at a moderate computational cost in many cases of interest. For a class of models, class , slightly wider than class , and a particular, natural selection for the regenerative state, the BT/RT method will yield tighter bounds at a higher computational cost. Under additional conditions, the bounds obtained using the less expensive version of BT/BRT and BT/RT seem to be tight for any value of t or not small values of t, depending on the initial probability distribution of the model. Class and class models with these additional conditions include both exact and bounding typical failure/repair performability models of fault-tolerant systems with exponential failure and repair time distributions and repair in every state with failed components and a reward rate structure which is a non-increasing function of the collection of failed components. We illustrate both the applicability and the performance of the methods using a large CTMC performability example of a fault-tolerant multiprocessor system.     

Scalability of pseudospectral methods for geodynamo simulations

The problem of understanding how Earth's magnetic field is generated is one of the foremost challenges in modern science. It is believed to be generated by a dynamo process, where the complex motions of an electrically conducting fluid provide the inductive action to sustain the field against the effects of dissipation. Current dynamo simulations, based on the numerical approximation to the governing equations of magnetohydrodynamics, cannot reach the very rapid rotation rates and low viscosities (i.e. low Ekman number) of Earth due to limitations in available computing power. Using a pseudospectral method, the most widely used method for simulating the geodynamo, computational requirements needed to run simulations in an ‘Earth‐like’ parameter regime are explored theoretically by approximating operation counts, memory requirements and communication costs in the asymptotic limit of large problem size. Theoretical scalings are tested using numerical calculations. For asymptotically large problems the spherical transform is shown to be the limiting step within the pseudospectral method; memory requirements and communication costs are asymptotically negligible. Another limitation comes from the parallel implementation, however, this is unlikely to be threatened soon and we conclude that the pseudospectral method will remain competitive for the next decade. Extrapolating numerical results based upon the code analysis shows that simulating a problem characterizing the Earth with Ekman number E = 10⁻⁹ would require at least 13 000 days per magnetic diffusion time with 54 000 available processors, a formidable computational challenge. At E = 10⁻⁸ an allocation of around 350 million CPU hours would compute a single diffusion time, many more CPU hours than are available in current supercomputing allocations but potentially reachable in the next decade. Exploration of the 10⁻⁶⩽E⩽10⁻⁷ regime could be performed at the present time using a substantial share of national supercomputing facilities or a dedicated cluster. Copyright © 2010 John Wiley & Sons, Ltd.     

面向诱导的交通状态信息提取方法

为实现可变信息板（Variable Message Signs,VMS）的交通状态信息提取,选取速度、饱和度与交叉口平均延误作为交通参数,提出了基于模糊推理的关联路段交通状态信息提取方法。针对驾驶员的VMS信息响应特性,设计了面向拥挤状态的VMS诱导策略,在关联路段均处于拥挤状态时基于Vague集对交通状态进行排序,以判别交通状态相对最优的关联路段进行发布。以实际算例验证了该方法的有效性,所提出的VMS诱导策略能够逐步引导拥挤交通流的分流,并为驾驶员在拥挤状态下进行路径选择提供决策支持。     

Spectral methods for simulations of transition and turbulence

The basic elements of spectral methods for unsteady, three-dimensional simulations of transition and turbulence are reviewed with a special emphasis on recent developments. Illustrations of large-scale applications are provided for both incompressible and compressible flow.     

Curvilinear grids for WENO methods in astrophysical simulations

We investigate the applicability of curvilinear grids in the context of astrophysical simulations and WENO schemes. With the non-smooth mapping functions from Calhoun et al. (2008), we can tackle many astrophysical problems which were out of scope with the standard grids in numerical astrophysics. We describe the difficulties occurring when implementing curvilinear coordinates into our WENO code, and how we overcome them. We illustrate the theoretical results with numerical data. The WENO finite difference scheme works only for high Mach number flows and smooth mapping functions, whereas the finite volume scheme gives accurate results even for low Mach number flows and on non-smooth grids.     

RAO logic for multiagent framework

In this paper,we deal with how agents reason about knowledge of others in multiagent system.We first present a knowledge representation framework called reasoning about others(RAO) which is designed specifically to represent concepts and rules used in reasoning about knowledge of others.From a class of sentences usually taken by people in daily life to reason about others,a rule called position exchange principle(PEP)is abstracted.PEP is described as an axiom scheme in RAO and regarded as a basic rule for agents to reason about others,and further it has the similar form and role to modus ponens and(K) axion of knowledge logic.The relationship between speech acts and common sense is also discussed which is necessary for RAO.Based on ideas from situation calculus,this relationship is characterized by an axiom schema in RAO.Our theories are also demonstrated by an example.     

Denoising methods for thermomechanical decomposition for quasi-equilibrium molecular dynamics simulations

In this work, we applied robust denoising methods well established in the signal processing field for the thermomechanical decomposition of velocity data obtained from molecular dynamics (MD) simulations. In the decomposition, the atomic velocity was assumed to be the sum of the mechanical velocity and the thermal velocity, which can be linked to the stress and temperature field at the continuum scale, respectively. For the quasi-equilibrium process, with the thermal velocity treated as the Gaussian distributed stationary noise with zero mean and a positive variance that is linearly proportional to temperature and mechanical velocity as the clean signal, the velocity decomposition can be recasted into a denoising problem, for which powerful denoising methods have been developed to estimate a clean signal from noisy data. We investigated the widely-used linear parametric real-domain, linear nonparametric Fourier-based, and nonlinear nonparametric wavelet-based denoising methods, first on their theoretical properties and then made comparsion among them for denosing some synthetic noisy 1-dimensional (1D) data generated from MD simulations. The nonlinear wavelet-based thresholding estimator possessed better optimality properties than the other estimators, and also outperformed the other estimators in the synthetic data test. A further test comparing the various denoising methods for an adiabatic shear crack nucleation and propagation process simulated using MD simulations showed better performance by the wavelet-based denoisng method. Results from this work reveal good potential of applying wavelet-based denoising method to the study of thermomechanical processes simulated using MD simulations.     

多代理系统下的主动配电网故障恢复博弈策略

在保障主动配电网可靠恢复前提下,为实现供电侧与用户侧利益均衡,本文提出了多代理系统(multiagent system,MAS)下的主动配电网故障恢复博弈策略.为充分考虑电网和电力用户对故障恢复的决策影响,设计了由电网代理、用户总代理和协调代理构成的MAS,建立了MAS信息传输模型.然后,构建了供电侧和用户侧在故障恢复中的利益函数、恢复策略空间,以及以电网代理和用户总代理作为参与人的合作博弈恢复模型.电网代理和用户总代理分别以改进的蚁群算法和统计方法进行分布并行计算,协调代理以双方共同的利益函数作为寻优目标通过迭代算法求得满足纳什均衡的恢复策略.本文以IEEE69节点模型为例,分别对单故障和连锁故障恢复情景求解,验证了本文所提策略的有效性.     

无线局域网业务流预报方法比较

介绍了基于时间序列、神经网络和小波的多种网络业务的预报方法,应用真实的无线局域网业务流序列检验了这些模型的预报性能,结果表明,和其他预报模型相比,基于神经网络的模型能够比较精确地捕获无线局域网业务流自身的特性,对业务流具有良好的预报性能,而基于ARIMA模型的预报性能最差。     

基于MPLS流量工程的多路径负载均衡方法

对多协议标签交换（MPLS）流量工程负载均衡问题，提出了两种多路径基于约束的负载均衡方法，在LSP建立初期就融入负载均衡思想。在通常的CSPF算法中，对于一个大带宽约束很可能无法找到可行路径，文中所提方法在没有单一路径满足带宽约束时，能将带宽约束划分为两个或多个子约束，并为每一子约束找到约束路径。实验结果表明，所提方法能增加路径建立的成功率，提高网络资源利用率，达到流量均衡。     

Vortex methods for incompressible flow simulations on the GPU

We present a remeshed vortex particle method for incompressible flow simulations on GPUs. The particles are convected in a Lagrangian frame and are periodically reinitialized on a regular grid. The grid is used in addition to solve for the velocity–vorticity Poisson equation and for the computation of the diffusion operators. In the present GPU implementation of particle methods, the remeshing and the solution of the Poisson equation rely on fast and efficient mesh-particle interpolations. We demonstrate that particle remeshing introduces minimal artificial dissipation, enables a faster computation of differential operators on particles over grid-free techniques and can be efficiently implemented on GPUs. The results demonstrate that, contrary to common practice in particle simulations, it is necessary to remesh the (vortex) particle locations in order to solve accurately the equations they discretize, without compromising the speed of the method. The present method leads to simulations of incompressible vortical flows on GPUs with unprecedented accuracy and efficiency.     

A comparative study of partitioning methods for crowd simulations

The simulation of large crowds of autonomous agents with realistic behavior is still a challenge for several computer research communities. In order to handle large crowds, some scalable architectures have been proposed. Nevertheless, the effective use of distributed systems requires the use of partitioning methods that can properly distribute the workload generated by agents among the existing distributed resources.In this paper, we analyze the use of irregular shape regions (convex hulls) for solving the partitioning problem. We have compared a partitioning method based on convex hulls with two techniques that use rectangular regions. The performance evaluation results show that the convex hull method outperforms the rest of the considered methods in terms of both fitness function values and execution times, regardless of the movement pattern followed by the agents. These results show that the shape of the regions in the partition can improve the performance of the partitioning method, rather than the heuristic method used.     

Preconditioned methods for simulations of low speed compressible flows

A time-derivative preconditioned system of equations suitable for the numerical simulation of inviscid compressible flow at low speeds is formulated. The preconditioned system of equations are hyperbolic in time and remain well-conditioned in the incompressible limit. The preconditioning formulation is easily generalized to multicomponent/multiphase mixtures. When applying conservative methods to multicomponent flows with sharp fluid interfaces, nonphysical solution behavior is observed. This stimulated the authors to develop an alternative solution method based on the nonconservative form of the equations which does not generate the aforementioned nonphysical behavior. Before the results of the application of the nonconservative method to multicomponent flow problems is reported, the accuracy of the method on single component flows will be demonstrated. In this report a series of steady and unsteady inviscid flow problems are simulated using the nonconservative method and a well-known conservative scheme. It is demonstrated that the nonconservative method is both accurate and robust for smooth low speed flows, in comparison to its conservative counterpart.     

网络通信流的Gamma分布多重分形模型

网络通信流具有的多重分形特性,使得对其建模成为一个极具挑战性的问题。以普通的多重分形小波模型为基础,提出一种基于Gamma分布的多重分形小波模型。该模型充分考虑最“粗”尺度系数概率分布特性,运用Gamma分布拟合最“粗”尺度的尺度系数,对于乘子Aj,k采用[－1,1]区间的对称Beta分布建模。通过仿真和数学分析的方法,分别从数据的概率分布特性、自相似特性以及多重分形特性等方面证实了该模型的优越性能。