Neural Networks for Real-Time Traffic Signal Control

Real-time traffic signal control is an integral part of the urban traffic control system, and providing effective real-time traffic signal control for a large complex traffic network is an extremely challenging distributed control problem. This paper adopts the multiagent system approach to develop distributed unsupervised traffic responsive signal control models, where each agent in the system is a local traffic signal controller for one intersection in the traffic network. The first multiagent system is developed using hybrid computational intelligent techniques. Each agent employs a multistage online learning process to update and adapt its knowledge base and decision-making mechanism. The second multiagent system is developed by integrating the simultaneous perturbation stochastic approximation theorem in fuzzy neural networks (NN). The problem of real-time traffic signal control is especially challenging if the agents are used for an infinite horizon problem, where online learning has to take place continuously once the agent-based traffic signal controllers are implemented into the traffic network. A comprehensive simulation model of a section of the Central Business District of Singapore has been developed using PARAMICS microscopic simulation program. Simulation results show that the hybrid multiagent system provides significant improvement in traffic conditions when evaluated against an existing traffic signal control algorithm as well as the SPSA-NN-based multiagent system as the complexity of the simulation scenario increases. Using the hybrid NN-based multiagent system, the mean delay of each vehicle was reduced by 78% and the mean stoppage time, by 85% compared to the existing traffic signal control algorithm. The promising results demonstrate the efficacy of the hybrid NN-based multiagent system in solving large-scale traffic signal control problems in a distributed manner.     

Model reference adaptive iterative learning control for linear systems

In this paper, we propose a model reference adaptive control (MRAC) strategy for continuous‐time single‐input single‐output (SISO) linear time‐invariant (LTI) systems with unknown parameters, performing repetitive tasks. This is achieved through the introduction of a discrete‐type parametric adaptation law in the ‘iteration domain’, which is directly obtained from the continuous‐time parametric adaptation law used in standard MRAC schemes. In fact, at the first iteration, we apply a standard MRAC to the system under consideration, while for the subsequent iterations, the parameters are appropriately updated along the iteration‐axis, in order to enhance the tracking performance from iteration to iteration. This approach is referred to as the model reference adaptive iterative learning control (MRAILC). In the case of systems with relative degree one, we obtain a pointwise convergence of the tracking error to zero, over the whole finite time interval, when the number of iterations tends to infinity. In the general case, i.e. systems with arbitrary relative degree, we show that the tracking error converges to a prescribed small domain around zero, over the whole finite time interval, when the number of iterations tends to infinity. It is worth noting that this approach allows: (1) to extend existing MRAC schemes, in a straightforward manner, to repetitive systems; (2) to avoid the use of the output time derivatives, which are generally required in traditional iterative learning control (ILC) strategies dealing with systems with high relative degree; (3) to handle systems with multiple tracking objectives (i.e. the desired trajectory can be iteration‐varying). Finally, simulation results are carried out to support the theoretical development. Copyright © 2006 John Wiley & Sons, Ltd.     

Global optimization with the PSO coupling‐type discrete gradient chaos model

In this paper, we propose a new multiagent discrete gradient chaos model using a coupling structure which PSO has. Concretely, first, we introduce a multiagent‐type optimization model whose agents search autonomously with the discrete gradient chaos model which is the simplest dynamical global search model, and they are coupled by convective coupling. Convective coupling in this model is used to aim at overcoming of emergence of boundary crisis which is a problem of the original discrete gradient chaos model. Second, we introduce PSO coupling structure, where population drifts to the “gbest” and the “pbest”, into discrete gradient chaos model. Then, we propose “PSO coupling‐type discrete gradient chaos model” with the search strategy based on objective function's value. In this paper, our proposed models are applied to several benchmark problems. The results show that our proposed models have better global optimization ability than the original discrete gradient chaos model and PSO model. © 2008 Wiley Periodicals, Inc. Electr Eng Jpn, 165(4): 67–75, 2008; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/eej.20563     

Adaptive iterative learning control of discrete‐time varying systems with unknown control direction

Without using Nussbaum gain, a novel method is presented to solve the unknown control direction problem for discrete‐time systems. The underlying idea is to fully exploit the convergence property of parameter estimates in well‐known adaptive algorithms. By incorporating two modifications into the control and the parameter update laws, respectively, we present an adaptive iterative learning control scheme for discrete‐time varying systems without the prior knowledge of the sign of control gain. It is shown that the proposed adaptive iterative learning control can achieve perfect tracking over the finite time interval while all the closed‐loop signals remain bounded. An illustrative example is presented to verify effectiveness of the proposed scheme. Copyright © 2012 John Wiley & Sons, Ltd.     

Fault estimation and accommodation for networked control systems with nonuniform sampling periods

This paper deals with the problem of fault estimation and accommodation for a class of networked control systems with nonuniform uncertain sampling periods. Firstly, the reason why the adaptive fault diagnosis observer cannot be applied to networked control systems is analyzed. Based on this analysis, a novel robust fault estimation observer is constructed to estimate both continuous‐time fault and system states by using nonuniformly discrete‐time sampled outputs. Furthermore, using the obtained states and fault information, a nonuniformly sampled‐data fault tolerant control law is designed to preserve the stability of the closed‐loop system. The proposed scheme can not only guarantee the impact of continuous‐time uncertainties and discrete‐time sampled estimation errors on the faulty system to satisfy a H _∞ performance index but also repress the negative effect of the unknown intersample behavior of continuous‐time fault by use of an inequality technique. Finally, simulation results are included to demonstrate the feasibility of the proposed method. Copyright © 2014 John Wiley & Sons, Ltd.     

A new command governor architecture for transient response shaping

In this paper, we develop a control framework for stabilization and command following of nonlinear uncertain dynamical systems. The proposed methodology consists of a new command governor architecture and an adaptive controller. The command governor is a dynamical system that adjusts the trajectory of a given command to follow an ideal reference system capturing a desired closed‐loop dynamical system behavior in transient time. Specifically, we show that the controlled nonlinear uncertain dynamical system can approach the ideal reference system by choosing the design parameter of the command governor. In addition, an adaptive element is used to asymptotically assure that the error between the controlled nonlinear uncertain dynamical system and the ideal reference system is reduced in long term. Therefore, the proposed methodology not only has closed‐loop transient and steady‐state performance guarantees but can also shape the transient response by adjusting the trajectory of the given command with the command governor. We highlight that there exists a trade‐off between the adaptive controller's learning rate and the command governor's design parameter. This key feature of our framework allows rapid suppression of system uncertainties without resorting to a high learning rate in the adaptive controller. Furthermore, we discuss the robustness properties of the proposed approach with respect to high‐frequency dynamical system content such as measurement noise and ∕ or unmodeled dynamics. A numerical example is provided to demonstrate the efficacy of the proposed architecture. Copyright © 2012 John Wiley & Sons, Ltd.     

Stability and synchronization of directed complex dynamical networks with random packet loss: the continuous‐time case and the discrete‐time case

This paper presents models for both the continuous‐time and discrete‐time directed complex dynamical networks (CDNs) with random packet loss, respectively. The packet loss may take place in the communication between every two neighbor nodes. The exponential mean‐square stability and the exponential mean‐square synchronization for both the continuous‐time and discrete‐time CDNs with random packet loss are investigated, respectively, by defining the packet loss probability matrix. In addition, two special cases, the CDN with synchronous packet loss and the CDN with node‐dependent packet loss, are discussed. Numerical examples are provided for illustrations. Copyright © 2012 John Wiley & Sons, Ltd.     

Adaptive dynamic surface control of stochastic nonstrict-feedback constrained nonlinear systems with input and state unmodeled dynamics

In this paper, the issue of adaptive neural control is discussed for a class of stochastic nonstrict-feedback constrained nonlinear systems with input and state unmodeled dynamics. A dynamic signal produced by the first-order auxiliary system is employed to deal with the dynamical uncertain terms. Radial basis function neural networks are used to reconstruct unknown nonlinear continuous functions. With the help of the mean value theorem and Young's inequality, only one learning parameter is adjusted online at recursive each step. Using the hyperbolic tangent function as nonlinear mapping, the output constrained stochastic nonstrict-feedback system in the presence of unmodeled dynamics is transformed into a novel unconstrained stochastic nonstrict-feedback system. Based on dynamic surface control technology and the property of Gaussian function, adaptive neural control is developed for the transformed stochastic nonstrict-feedback system. The output abides by stochastic constraints in probability. By the Lyapunov method, all signals of the closed-loop control system are proved to be semi-global uniform ultimate bounded (SGUUB) in probability. The obtained theoretical findings are verified by two numerical examples.     

A switching periodic adaptive control approach for time‐varying parameters with unknown periodicity

Periodic variations are encountered in many real systems, which can exist in the system parameters, as a disturbance or as the tracking objective. However, there exist a great number of situations where the periodicity is not known in advance. Hence, how to compensate for the effects of time‐varying parameters with unknown periodicity remains a challenge for the controller design. In this paper, we proposed a switching periodic adaptive control approach for continuous‐time nonlinear parametric systems with periodic uncertainties in which the period and bound are not known in advance. We utilized a fully saturated periodic adaptation law to identify the unknown periodic parameters in a pointwise manner. In addition, we provided a logic‐based switching scheme to estimate the unknown period and bound online simultaneously. By virtue of Lyapunov stability analysis, we show that the asymptotic convergence can be guaranteed irrespective of the initial conditions. Finally, we carried out numerical simulations to demonstrate the efficacy of the switching periodic adaptive control algorithm. The proposed approach can be applied to parametric nonlinear systems with time‐varying parameters of unknown periodicity irrespective of the types of periodic uncertainties. Copyright © 2015 John Wiley & Sons, Ltd.     

A multi‐agent approach to unit commitment

This paper proposes a multiagent system to power system unit commitment problems. Multiagent is a new paradigm for developing software applications. Coordinating the behavior of autonomous agents is a key issue in agent‐oriented programming techniques today. Recently, agents are being used in an increasingly wide variety of applications, ranging from comparatively small systems such as E‐mail filters to large, open, complex systems such as air traffic control. Though some agent frameworks have been proposed in the power system field, the number of studies is limited. In this paper, we developed a power system unit commitment application by multiagent architecture. Our multiagent system has the following characteristics: (1) The system consists of a single facilitator agent, two mobile agents, and one or more generator agents which are elements of power system network. (2) The facilitator agent is developed to act as a manager for the process by using the singleton design pattern. The mobile agents migrate to generator‐agents to increase or decrease their power generations. The generator agents have their operational data. (3) Message object is developed to communicate between the agents using KQML‐like object. The proposed approach is applied to a simple model system, and the results show that the multiagent system is an efficient decentralized approach for solving power system unit commitment problems. © 2002 Wiley Periodicals, Inc. Electr Eng Jpn, 141(2): 41–47, 2002; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/eej.10057     

An approach to digital implementation of continuous backstepping adaptive control for linear systems

This paper presents a solution to the problem of digitally implementing backstepping adaptive control for linear systems. The continuous‐time system to be controlled is given a discrete‐time representation in the δ‐operator. A discrete adaptive backstepping controller is then designed for such a discrete‐time model. The effect of the modelling error, generated by the sampling process, is accounted for in the parameter update law by a σ‐modification. It is shown that all the signals (discrete and continuous) of the closed loop are uniformly bounded, with a region of attraction which is a K_∞ function of the sampling rate. An upper bound on the asymptotic tracking error is then given, and shown to be proportional to the sampling period. Copyright © 1999 John Wiley & Sons, Ltd.     

Iterative learning control for a class of non‐affine‐in‐input processes in Hilbert space

In this paper, iterative learning control (ILC) of a class of non‐affine‐in‐input processes is considered in Hilbert space, where the plant operators are quite general in the sense that they could be static or dynamic, differentiable or non‐differentiable, continuous‐time or discrete‐time, and so forth. The control problem is first transformed to a problem of solving global implicit function to ensure the uniqueness of desired control input. Then, two contraction mapping‐based ILC schemes are proposed in terms of the continuous differentiability of process model, where the learning convergence condition is derived through rigorous analysis. The proposed ILC schemes make full use of the process repetition, deal with system uncertainties easily, and are effective to infinite‐dimensional or distributed parameter systems. In the end, the learning controller is applied to the boundary output control of a class of anaerobic digestion process for wastewater treatment. The control efficacy is verified by simulation. Copyright © 2013 John Wiley & Sons, Ltd.     

A convex optimization approach to adaptive stabilization of discrete‐time LTI systems with polytopic uncertainties

This paper suggests a simple convex optimization approach to state‐feedback adaptive stabilization problem for a class of discrete‐time LTI systems subject to polytopic uncertainties. The proposed method relies on estimating the uncertain parameters by solving an online optimization at each time step, such as a linear or quadratic programming, and then, on tuning the control law with that information, which can be conceptually viewed as a kind of gain‐scheduling or indirect adaptive control. Specifically, an admissible domain of stabilizing state‐feedback gain matrices is designed offline by means of linear matrix inequality problems, and based on the online estimation of the uncertain parameters, the state‐feedback gain matrix is calculated over the set of stabilizing feedback gains. The proposed stabilization algorithm guarantees the asymptotic stability of the overall closed‐loop control system. An example is given to show the effectiveness of the proposed approach. Copyright © 2014 John Wiley & Sons, Ltd.     

Discrete time filters for doubly stochastic poisson processes and other exponential noise models

The well‐known Kalman filter is the optimal filter for a linear Gaussian state‐space model. Furthermore, the Kalman filter is one of the few known finite‐dimensional filters. In search of other discrete‐time finite‐dimensional filters, this paper derives filters for general linear exponential state‐space models, of which the Kalman filter is a special case. One particularly interesting model for which a finite‐dimensional filter is found to exist is a doubly stochastic discrete‐time Poisson process whose rate evolves as the square of the state of a linear Gaussian dynamical system. Such a model has wide applications in communications systems and queueing theory. Another filter, also with applications in communications systems, is derived for estimating the arrival times of a Poisson process based on negative exponentially delayed observations. Copyright © 1999 John Wiley & Sons, Ltd.     

Sampled‐data adaptive control of a class of nonlinear systems

The paper considers the discrete‐time implementation of a class of backstepping adaptive controllers for uncertain nonlinear systems in the parametric strict‐feedback form. The stability of the resultant sampled‐data system cannot be guaranteed when the direct discretization of the continuous‐time backstepping controller is applied to the same system via a sampling and hold device. Therefore, the paper has presented a sampled‐data control scheme which involves first modifying the existing continuous‐time controller and then discretizing it using the forward Euler method. It has been shown that the proposed control guarantees in a semi‐global sense the boundedness of all the variables of the overall sampled‐data system under some conditions. Particularly, the state of the nonlinear system to be controlled can converge to any arbitrarily small neighbourhood of the origin. Copyright © 2011 John Wiley & Sons, Ltd.     

Decentralized guaranteed cost control for uncertain large‐scale systems using additive control gain

This paper investigates an application of additive control gain to the guaranteed cost control (GCC) problem of decentralized robust control for a class of discrete‐time uncertain large‐scale systems. Based on the Linear Matrix Inequality (LMI) design approach, a class of decentralized local fixed state feedback controllers with additive control gain is established. The novel contribution of this paper is that multiobjective control is attained by using the additive control gain. Although the additive control gain is included in the uncertain large‐scale systems, the closed‐loop system is asymptotically stable. In order to demonstrate the efficiency of our proposed controller, using the fuzzy logic control as the additive control gain, the simple numerical example is given. © 2009 Wiley Periodicals, Inc. Electr Eng Jpn, 169(3): 18–32, 2009; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/eej.20726     

Extended fuzzy function model with stable learning methods for online system identification

The aim of the online nonlinear system identification is the accurate modeling of the current local input‐output behavior of the plant without using any prior knowledge and offline modeling phase. It is a challenging task for many intelligent systems when used for real‐time control applications. In this paper, we propose a novel computationally efficient extended fuzzy functions (EFF) model for system identification of unknown nonlinear discrete‐time systems. The main contributions are to introduce an effective quasi‐nonlinear model (EFF) and propose adaptive learning rates (ALR) for recursive least squares (RLS) and gradient‐descent (GD) methods. The asymptotic convergence of the modeling errors and boundedness of the parameters are proved by using the input‐to‐state stability (ISS) approach. Numerical simulations are performed for Box–Jenkins gas furnace system and a nonlinear dynamic system. The benefits of its accuracy, stability and simple implementation in practice indicate that EFF model is a promising technique for online identification of nonlinear systems. Copyright © 2010 John Wiley & Sons, Ltd.     

Matching a system behavior with a known set of models: A quadratic optimization‐based adaptive solution

The matching process between a time‐domain external behavior of a lumped single‐input single‐output dynamical system and a known set of linear continuous time‐invariant models is tackled in this paper. The proposed online solution is based on an adaptive structure detector, which in finite time locates in the known set of models the one corresponding to the observed external behavior; the detector results from the solution of a constrained quadratic optimization problem. The problem is expressed in terms of the time‐domain activity of a family of discriminating filters and is solved via a normalized gradient algorithm, which avoids mismatching due to the presence of structural zeros in the filters and can take into account band‐limited high‐frequency measurement noise. A failure detection problem concerning a simulated servomechanism is included in order to illustrate the proposed solution. Copyright © 2008 John Wiley & Sons, Ltd.     

Robust output tracking of uncertain nonlinear systems with completely unknown dead‐zone input: A self‐tuning design approach

The main purpose of this paper is to propose a direct and simple approach, called a self‐tuning design approach, to dealing with any nonsymmetric dead‐zone input nonlinearity where its information is completely unknown. In order to describe the approach, the output tracking problem is considered for a class of uncertain nonlinear systems with any nonsymmetric dead‐zone input. First, a dead‐zone input is represented as a time‐varying input‐dependent function such that the considered dynamical system with dead‐zone input can be transfered into an uncertain nonlinear dynamical system subject to a linear input with time‐varying input coefficient. Then, by making use of the self‐tuning design approach, a class of adaptive robust output tracking control schemes with a rather simple structure is synthesized. Thus, the proposed direct and simple self‐tuning design approach can be easily understood by the engineering designers, and the resulting simple adaptive robust control schemes can be well implemented in most practical engineering control problems. By combining the proposed self‐tuning design approach with other control methods, one may expect to obtain a number of interesting results for a rather large class of uncertain nonlinear dynamical systems with dead‐zone in the actuators. Finally,the simulations of some numerical examples are provided to demonstrate the validity of the theoretical results. Copyright © 2016 John Wiley & Sons, Ltd.     

Evolutionary reinforcement learning system with time‐varying parameters

In this paper, an evolutionary reinforcement learning system with time‐varying parameters that can learn appropriate policy in dynamical POMDPs is proposed. The proposed system has time‐varying parameters that can be adjusted by using reinforcement learning. Hence, the system can adapt to the time variation of the dynamical environment even if its variation cannot be observed. In addition, the state space of the environment is divided evolutionarily. Thus, one need not divide the state space in advance. The efficacy of the proposed system is shown by mobile robot control simulation under the environment belonging to dynamical POMDPs. The environment is the passage that has gates iterate opening and closing. © 2006 Wiley Periodicals, Inc. Electr Eng Jpn, 156(1): 54–60, 2006; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/eej.20170