Linear quadratic regulation for discrete-time systems with state delays and multiplicative noise

In this paper, the linear quadratic regulation problem for discrete-time systems with state delays and multiplicative noise is considered. The necessary and sufficient condition for the problem admitting a unique solution is given. Under this condition, the optimal feedback control and the optimal cost are presented via a set of coupled difference equations. Our approach is based on the maximum principle. The key technique is to establish relations between the costate and the state.     

Linear minimum mean square filter for discrete-time linear systems with Markov jumps and multiplicative noises

In this paper we obtain the linear minimum mean square estimator (LMMSE) for discrete-time linear systems subject to state and measurement multiplicative noises and Markov jumps on the parameters. It is assumed that the Markov chain is not available. By using geometric arguments we obtain a Kalman type filter conveniently implementable in a recurrence form. The stationary case is also studied and a proof for the convergence of the error covariance matrix of the LMMSE to a stationary value under the assumption of mean square stability of the system and ergodicity of the associated Markov chain is obtained. It is shown that there exists a unique positive semi-definite solution for the stationary Riccati-like filter equation and, moreover, this solution is the limit of the error covariance matrix of the LMMSE. The advantage of this scheme is that it is very easy to implement and all calculations can be performed offline.     

Indefinite quadratic with linear costs optimal control of Markov jump with multiplicative noise systems

In this paper we consider the stochastic optimal control problem of discrete-time Markov jump with multiplicative noise linear systems. The performance criterion is assumed to be formed by a linear combination of a quadratic part and a linear part in the state and control variables. The weighting matrices of the state and control for the quadratic part are allowed to be indefinite. We present a necessary and sufficient condition under which the problem is well posed and a state feedback solution can be derived from a set of coupled generalized Riccati difference equations interconnected with a set of coupled linear recursive equations. For the case in which the quadratic-term matrices are non-negative, this necessary and sufficient condition can be written in a more explicit way. The results are applied to a problem of portfolio optimization.     

Linear quadratic regulation for linear time-varying systems with multiple input delays

This paper studies the classic linear quadratic regulation (LQR) problem for both continuous-time and discrete-time systems with multiple input delays. For discrete-time systems, the LQR problem for systems with single input delay has been studied in existing literature, whereas a solution to the multiple input delay case is not known to our knowledge. For continuous-time systems with multiple input delays, the LQR problem has been tackled via an infinite dimensional system theory approach and a frequency/time domain approach. The objective of the present paper is to give an explicit solution to the LQR problem via a simple and intuitive approach. The main contributions of the paper include a fundamental result of duality between the LQR problem for systems with multiple input delays and a smoothing problem for an associated backward stochastic system. The duality allows us to obtain a solution to the LQR problem via standard projection in linear space. The LQR controller is simply constructed by the solution of one backward Riccati difference (for the discrete-time case) or differential (for the continuous-time case) equation of the same order as the plant (ignoring the delays).     

Indefinite linear quadratic optimal control problem for singular discrete-time system with multiple input delays

An indefinite linear quadratic (ILQ) optimal control problem is discussed for singular discrete-time-varying linear systems with multiple input delays. The problem is transformed to the one for standard systems by normalizability decomposition. An explicit controller is obtained by computing the gain of the smoothing estimation of dual systems. Necessary and sufficient conditions guaranteeing the existence of unique solution are given simultaneously. A numerical example illustrates the presented method.     

Linear quadratic regulation for systems with time‐varying delay

In this paper we study the linear quadratic regulation (LQR) problem for discrete‐time systems with time‐varying delay in the control input channel. We assume that the time‐varying delay is of a known upper bound, then the LQR problem is transformed into the optimal control problem for systems with multiple input channels, each of which has single constant delay. The optimal controller is derived by establishing a duality between the LQR and a smoothing estimation for an associated system with a multiple delayed measurement. Copyright © 2009 John Wiley & Sons, Ltd.     

Linear quadratic regulation of systems with stochastic parameter uncertainties

In this paper, we develop a theoretical framework for linear quadratic regulator design for linear systems with probabilistic uncertainty in the parameters. The framework is built on the generalized polynomial chaos theory. In this framework, the stochastic dynamics is transformed into deterministic dynamics in higher dimensional state space, and the controller is designed in the expanded state space. The proposed design framework results in a family of controllers, parameterized by the associated random variables. The theoretical results are applied to a controller design problem based on stochastic linear, longitudinal F16 model. The performance of the stochastic design shows excellent consistency, in a statistical sense, with the results obtained from Monte-Carlo based designs.     

Stochastic linear quadratic regulation for discrete-time linear systems with input delay

This paper considers the stochastic LQR problem for systems with input delay and stochastic parameter uncertainties in the state and input matrices. The problem is known to be difficult due to the presence of interactions among the delayed input channels and the stochastic parameter uncertainties in the channels. The key to our approach is to convert the LQR control problem into an optimization one in a Hilbert space for an associated backward stochastic model and then obtain the optimal solution to the stochastic LQR problem by exploiting the dynamic programming approach. Our solution is given in terms of two generalized Riccati difference equations (RDEs) of the same dimension as that of the plant.     

State estimation for multi‐channel stochastic singular systems with multiplicative noise

The state estimation problem for multi‐channel singular systems with multiplicative noise is considered based on singular value decomposition. First, two equivalent reduced order subsystems are obtained via the decomposition. Then, in order to solve the estimation problem, the subsystems are rewritten into a new form. It is noted that the measurement noise here becomes colored noise, which contains the dynamic noise, measurement noise, and multiplicative noise of the original system. In this situation, existing filtering methods cannot be directly applied, so a modified filtering method is given. The recursive algorithm for the state estimation is obtained by the filtering method. In addition, the estimation of dynamic noise is derived via the algorithm. A simulation example is given to show the effectiveness of the proposed algorithm. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Optimal control of discrete-time linear fractional-order systems with multiplicative noise

A finite horizon linear quadratic (LQ) optimal control problem is studied for a class of discrete-time linear fractional systems (LFSs) affected by multiplicative, independent random perturbations. Based on the dynamic programming technique, two methods are proposed for solving this problem. The first one seems to be new and uses a linear, expanded-state model of the LFS. The LQ optimal control problem reduces to a similar one for stochastic linear systems and the solution is obtained by solving Riccati equations. The second method appeals to the principle of optimality and provides an algorithm for the computation of the optimal control and cost by using directly the fractional system. As expected, in both cases, the optimal control is a linear function in the state and can be computed by a computer program. A numerical example and comparative simulations of the optimal trajectory prove the effectiveness of the two methods. Some other simulations are obtained for different values of the fractional order.     

Linear quadratic tracking problem for discrete‐time systems with multiple delays in single input channel

The linear quadratic tracking problem for discrete‐time systems with multiple delays in single input channel is considered. In this paper, we provide an approach without resorting to system state augmentation. The optimal tracking control is given in terms of the current state, the previous inputs, and the output of an auxiliary backward deterministic delay system which is formulated for the first time in this paper. The solution relies on a Riccati difference equation of the same dimension as the plant (ignoring the delays). The key to our development is the establishment of a duality between the optimal tracking control and the optimal smoothing estimation of an associated stochastic backward system as well as the introduction of the auxiliary backward deterministic delay system. An analysis of the computational complexity of the proposed approach and its comparison with that of the augmentation method, which is to incorporate the delayed inputs into the augmented state, are provided. An example is given to demonstrate the effectiveness of the results. Copyright © 2009 John Wiley & Sons, Ltd.     

Robust state estimation for uncertain discrete-time stochastic systems with missing measurements

In this paper, results of robust estimation of Zhou (2010a) are extended to state estimation with missing measurements. A new procedure is derived which inherits the main properties of that of Zhou (2010a). In this extension, a covariance matrix used in the recursions is replaced by its estimate which makes its asymptotic property investigation mathematically difficult. Though introducing a monotonic function and using the so-called squeeze rule, this new robust estimator is proved to converge to a stable system. Numerical simulation results indicate that the proposed estimator may have an estimation accuracy better than the estimator of Wang, Yang, Daniel, and Liu (2005).     

On state-constrained porous-media systems with gradient-type multiplicative noise

The aim of the present paper is to provide necessary and sufficient conditions to maintain a stochastic coupled system with porous media components and gradient-type noise in a prescribed set of constraints by using internal controls. This work is a complementary contribution to the results obtained by the same authors, also on the viability problem associated to the porous media equation, but with Lipschitz noise. Second, the present paper provides a different framework in which the quasi-tangency condition can be obtained with optimal speed. In comparison with the aforementioned result, and from a technical point of view, here, we transform the stochastic system into a random-PDE one, via the rescaling approach, and then we study the viability of random sets. As an application, (stronger) conditions for the stabilization of the stochastic porous media equations are obtained. These are illustrated on a simple example.     

Linear quadratic Nash game-based tracker for multiparameter singularly perturbed sampled-data systems: digital redesign approach

In this paper, a linear quadratic Nash game-based tracker for multiparameter singularly perturbed sample-data systems is developed. A generalized cross-coupled multiparameter algebraic Riccati equation (GCMARE) with two quadratic cost functions is solved by applying the LQR design methodology for the optimal tracker design. Firstly, the asymptotic expansions of the GCMARE are newly established, and the proposed algorithm is able to effectively solve the GCMARE with the quadratic convergence rate. Then, the low-gain digital controller with a high design performance is realized through the prediction-based digital redesign method. Finally, for further improving the tracking performance, the chaos-evolutionary-programming algorithm (CEPA) is utilized to optimally tune the parameters of the tracker. An example is presented to demonstrate the effectiveness of the proposed methodology.     

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.     

Stability analysis of 2-D switched systems with multiplicative noise under arbitrary and restricted switching signals

This paper introduces a new class of discrete-time two-dimensional (2-D) switched systems with multiplicative noise. Firstly, we extend the definition of the asymptotic stability and establish a sufficient stochastically stability condition for this new model under arbitrary switching signal. Then, by introducing the average dwell time into this new model and combining with the Lyapunov function, we investigate the extended stochastic exponential stability of the 2-D switched systems with multiplicative noise for the restricted switching case. Moreover, some remarks and discussions are given to illustrate the significance of obtained results, which generalise and comprise some previous results the literature. Finally, two examples are provided to show the effectiveness of the theoretical results.     

Linear estimation for random delay systems

This paper is concerned with the linear estimation problems for discrete-time systems with random delayed observations. When the random delay is known online, i.e., time-stamped, the random delayed system is reconstructed as an equivalent delay-free one by using measurement reorganization technique, and then an optimal linear filter is presented based on the Kalman filtering technique. However, the optimal filter is time-varying, stochastic, and does not converge to a steady state in general. Then an alternative suboptimal filter with deterministic gains is developed under a new criteria. The estimator performance in terms of their error covariances is provided, and its mean square stability is established. Finally, a numerical example is presented to illustrate the efficiency of proposed estimators.     

H2 control of discrete-time periodic systems with Markovian jumps and multiplicative noise

This paper addresses the problem of optimal and robust H₂ control for discrete-time periodic systems with Markov jump parameters and multiplicative noise. To analyse the system performance in the presence of exogenous random disturbance, an H₂ norm is firstly established on the basis of Gramian matrices. Further, under the condition of exact observability, a necessary and sufficient condition is presented for the solvability of H₂ optimal control problem by means of a generalised Riccati equation. When the transition probabilities of jump parameter are incompletely measurable, an H₂-guaranteed cost norm is exploited and the robust H₂ controller is designed through a linear matrix inequality (LMI) optimisation approach. An example of a networked control system is supplied to illustrate the proposed results.     

Null controllability with vanishing energy for discrete-time systems

In this paper null controllability with vanishing energy for discrete-time systems is considered. As in the case of continuous time systems necessary and sufficient conditions in terms of an algebraic Riccati equation are given. Then necessary and sufficient conditions involving the eigenvalues of the state matrix are given. Reachability and controllability with vanishing energy are also considered and necessary and sufficient conditions for them are given. Finally applications to sampled-data systems, systems with impulse control and periodic systems are discussed.