Infinite time horizon nonzero-sum linear quadratic stochastic differential games with state and control-dependent noise

This paper discusses the infinite time horizon nonzero-sum linear quadratic （LQ） differential games of stochastic systems governed by Itoe＇s equation with state and control-dependent noise. First, the nonzero-sum LQ differential games are formulated by applying the results of stochastic LQ problems. Second, under the assumption of mean-square stabilizability of stochastic systems, necessary and sufficient conditions for the existence of the Nash strategy are presented by means of four coupled stochastic algebraic Riccati equations. Moreover, in order to demonstrate the usefulness of the obtained results, the stochastic H-two/H-infinity control with state, control and external disturbance-dependent noise is discussed as an immediate application.     

Broadcast stochastic receding horizon control of multi-agent systems

Optimal regulation of stochastically behaving agents is essential to achieve a robust aggregate behavior in a swarm of agents. How optimally these behaviors are controlled leads to the problem of designing optimal control architectures. In this paper, we propose a novel broadcast stochastic receding horizon control architecture as an optimal strategy for stabilizing a swarm of stochastically behaving agents. The goal is to design, at each time step, an optimal control law in the receding horizon control framework using collective system behavior as the only available feedback information and broadcast it to all agents to achieve the desired system behavior. Using probabilistic tools, a conditional expectation based predictive model is derived to represent the ensemble behavior of a swarm of independently behaving agents with multi-state transitions. A stochastic finite receding horizon control problem is formulated to stabilize the aggregate behavior of agents. Analytical and simulation results are presented for a two-state multi-agent system. Stability of the closed-loop system is guaranteed using the supermartingale theory. Almost sure (with probability 1) convergence of the closed-loop system to the desired target is ensured. Finally, conclusions are presented.     

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.     

Robustness of exponential stability of a class of stochastic functional differential equations with infinite delay

We regard the stochastic functional differential equation with infinite delay as the result of the effects of stochastic perturbation to the deterministic functional differential equation , where is defined by x_t(θ)=x(t+θ),θ(−∞,0]. We assume that the deterministic system with infinite delay is exponentially stable. In this paper, we shall characterize how much the stochastic perturbation can bear such that the corresponding stochastic functional differential system still remains exponentially stable.     

On the exponential stability in mean square of neutral stochastic functional differential equations

In [1 and 2], some efforts have been devoted to the investigation of exponential stability in mean square of neutral stochastic functional differential equations. However, the results derived there are either difficult to demonstrate in a straightforward way for practical situations or somewhat too restricted to be applied to general neutral stochastic functional differential equations, for instance, nonautonomous cases. In this paper, we shall establish some results which are more effective and relatively easy to verify to obtain the required stability.     

Estimation of the variances of TCP/RED using stochastic differential equation

By using the stochastic differential equation (SDE), a model is built to describe the system behavior, especially the queue length behavior, of TCP/RED (transmission control protocol with random early detection) on a single bottleneck network. Then, not only the static equilibrium point of the system is calculated, but also its variance is estimated. After calculating the variance of the queue length in the router, the authors show how such random oscillation affects the maximum connection number that the link admits while keeping low dropping probability, and how this relation is influenced by other network parameters. This model can be applied to the case where only homogeneous TCP flows exist, as well as where TCP and UDP (user data protocol) flows coexist. The simulations by NS‐2 show that our deductive method is effective. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Stabilization and destabilization of hybrid systems of stochastic differential equations

This paper aims to determine whether or not a stochastic feedback control can stabilize or destabilize a given nonlinear hybrid system. New methods are developed and sufficient conditions on the stability and instability for hybrid stochastic differential equations are provided. These results are then used to examine stochastic stabilization and destabilization.     

Composition schemes for the stochastic differential equation describing collisional pitch-angle diffusion

Two new second order accurate Monte Carlo integration schemes are derived for the stochastic differential equation describing pitch-angle scattering by Coulomb collisions in magnetized plasmas. Here the pitch-angle is the angle between the magnetic field and the particle velocity vectors. Mathematically this collision process corresponds to diffusion in the polar angle of a spherical coordinate system. The schemes are simple to implement, they are naturally bounded to the solution domain and their convergences are shown to compare favourably against commonly used alternative integration schemes.     

Linear−quadratic optimal control and nonzero‐sum differential game of forward−backward stochastic system

An existence and uniqueness result for one kind of forward–backward stochastic differential equations with double dimensions was obtained under some monotonicity conditions. Then this result was applied to the linear‐quadratic stochastic optimal control and nonzero‐sum differential game of forward–backward stochastic system. The explicit forms of the optimal control and the Nash equilibrium point are obtained respectively. We note that our method is effective in studying the uniqueness of Nash equilibrium point. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

On the stability of receding horizon control for continuous-time stochastic systems

We study the stability of receding horizon control for continuous-time non-linear stochastic differential equations. We illustrate the results with a simulation example in which we employ receding horizon control to design an investment strategy to repay a debt.     

Stochastic maximum principle for delayed doubly stochastic control systems and their applications

ABSTRACT

In this paper, we investigate the optimal control problems for delayed doubly stochastic control systems. We first discuss the existence and uniqueness of the delayed doubly stochastic differential equation by martingale representation theorem and contraction mapping principle. As a necessary condition of the optimal control, we deduce a stochastic maximum principle under some assumption. At the same time, a sufficient condition of optimality is obtained by using the duality method. At the end of the paper, we apply our stochastic maximum principle to a class of linear quadratic optimal control problem and obtain the explicit expression of the optimal control.     

Mean‐field backward stochastic differential equation with non‐Lipschitz coefficient

This paper establishes a new existence and uniqueness result of a solution for one dimensional mean‐field backward stochastic differential equation (MFBSDE), where its coefficient is weaker than the classical Lipschitz case. An example is given to illustrate its applicability. This new solution will provide a key tool for studying mean‐field control problems.     

Delay-dependent exponential stability of the backward Euler method for nonlinear stochastic delay differential equations

Recently, several scholars discussed the question of under what conditions numerical solutions can reproduce exponential stability of exact solutions to stochastic delay differential equations, and some delay-independent stability criteria were obtained. This paper is concerned with delay-dependent stability of numerical solutions. Under a delay-dependent condition for the stability of the exact solution, it is proved that the backward Euler method is mean-square exponentially stable for all positive stepsizes. Numerical experiments are given to confirm the theoretical results.     

On exponential convergence and robustness of least-squares identification

Robustness and convergence properties of exponentially weighted least-squares identification are studied. It is shown that exponential convergence in the noiseless case can be obtained for a class of increasing or decreasing regression vectors. The rate of change of the limits in the regressors affect the convergence rates, which are explicitly given. It is demonstrated that for a sub-class of regressors decreasing-in-the-norm exponential convergence without the noise does not guarantee robustness subject to a bounded noise. Instead, exponential divergence of the estimate is shown in a specific case.     

Infinite horizon linear quadratic stochastic Nash differential games of Markov jump linear systems with its application

In this paper, we deal with the problem of stochastic Nash differential games of Markov jump linear systems governed by Itô-type equation. Combining the stochastic stabilizability with the stochastic systems, a necessary and sufficient condition for the existence of the Nash strategy is presented by means of a set of cross-coupled stochastic algebraic Riccati equations. Moreover, the stochastic H₂/H_∞ control for stochastic Markov jump linear systems is discussed as an immediate application and an illustrative example is presented.     

Split-step double balanced approximation methods for stiff stochastic differential equations

In the modelling of many important problems in science and engineering we face stiff stochastic differential equations (SDEs). In this paper, a new class of split-step double balanced (SSDB) approximation methods is constructed for numerically solving systems of stiff Itô SDEs with multi-dimensional noise. In these methods, an appropriate control function has been used twice to improve the stability properties. Under global Lipschitz conditions, convergence with order one in the mean-square sense is established. Also, the mean-square stability (MS-stability) properties of the SSDB methods have been analysed for a one-dimensional linear SDE with multiplicative noise. Therefore, the MS-stability functions of SSDB methods are determined and in some special cases, their regions of MS-stability have been compared to the stability region of the original equation. Finally, simulation results confirm that the proposed methods are efficient with respect to accuracy and computational cost.     

Stability of stochastic interval systems with time delays

Consider a given exponentially stable system undergoing a random perturbation which is dependent on a past state of the solution of the system. Suppose this stochastically perturbed system is described by a stochastic differential-functional equation. In this paper, we establish a sufficient condition that the perturbed system remains exponentially stable. Using a specific example, we show how this condition may be used, and we extend it to deal with multiple time delays.     

Infinite horizon optimal control of linear discrete time systems with stochastic parameters

The infinite horizon optimal control problem is considered in the general case of linear discrete time systems and quadratic criteria, both with stochastic parameters which are independent with respect to time. A stronger stabilizability property and a weaker observability property than usual for deterministic systems are introduced. It is shown that the infinite horizon problem has a solution if the system has the first property. If in addition the problem has the second property the solution is unique and the control system is stable in the mean square sense. A simple necessary and sufficient condition, explicit in the system matrices, is given for the system to have the stronger stabilizability property. This condition also holds for deterministic systems to be stabilizable in the usual sense. The stronger stabilizability and weaker observability properties coincide with the usual ones if the parameters are deterministic.     

Second-order Taylor expansion for backward doubly stochastic control system

We consider the second-order Taylor expansion for backward doubly stochastic control system. The results are obtained under no restriction on the convexity of control domain. Moreover, the control variable is allowed in the drift coefficient and the diffusion coefficient.