<Emphasis Type="BoldItalic">H</Emphasis><Subscript>2</Subscript>-Control and the Separation Principle for Discrete-Time Markovian Jump Linear Systems

In this paper we consider the H₂-control problem of discrete-time Markovian jump linear systems. We assume that only an output and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed-loop system is mean square stable and minimizes the H₂-norm of the system. As in the case with no jumps, we show that an optimal controller can be obtained from two sets of coupled algebraic Riccati equations, one associated with the optimal control problem when the state variable is available, and the other associated with the optimal filtering problem. This is the principle of separation for discrete-time Markovian jump linear systems. When there is only one mode of operation our results coincide with the traditional separation principle for the H₂-control of discrete-time linear systems. Date received: June 1, 2001. Date revised: October 13, 2003.     

Robust H 2-control for discrete-time Markovian jump linear systems

This paper deals with the robust H₂-control of discrete-time Markovian jump linear systems. It is assumed that both the state and jump variables are available to the controller. Uncertainties satisfying some norm bounded conditions are considered on the parameters of the system. An upper bound for the H₂-control problem is derived in terms of a linear matrix inequality (LMI) optimization problem. For the case in which there are no uncertainties, we show that the convex formulation is equivalent to the existence of the mean square stabilizing solution for the set of coupled algebraic Riccati equations arising on the quadratic optimal control problem of discrete-time Markovian jump linear systems. Therefore, for the case with no uncertainties, the convex formulation considered in this paper imposes no extra conditions than those in the usual dynamic programming approach. Finally some numerical examples are presented to illustrate the technique.     

一类离散时间非齐次马尔可夫跳跃系统最优控制

本文研究了一类离散时间非齐次马尔可夫跳跃线性系统的线型二次高斯(linear quadratic Gaussian,LQG)问题,其中系统模态转移概率矩阵随时间随机变化,其变化特性由一高阶马尔可夫链描述.对于该系统的LQG问题,文中首先给出了线性最优滤波器,得到最优状态估计;其次,验证分离定理成立,并利用利用动态规划方法设计了系统最优控制器;最后,数值仿真结果验证了所设计控制器的有效性.     

事件驱动的网络化系统最优控制

针对随机事件驱动的网络化控制系统, 研究其中的有限时域和无限时域内最优控制器的设计问题. 首先, 根据执行器介质访问机制将网络化控制系统建模为具有多个状态的马尔科夫跳变系统; 然后, 基于动态规划和马尔科夫跳变线性系统理论设计满足二次型性能指标的最优控制序列, 通过求解耦合黎卡提方程的镇定解, 给出最优控制律的计算方法, 使得网络化控制系统均方指数稳定; 最后, 通过仿真实验表明了所提出方法的有效性.

     

Feedback control on Nash equilibrium for discrete-time stochastic systems with Markovian jumps: Finite-horizon case

In this paper, we consider the feedback control on nonzero-sum linear quadratic (LQ) differential games in finite horizon for discrete-time stochastic systems with Markovian jump parameters and multiplicative noise. Four-coupled generalized difference Riccati equations (GDREs) are obtained, which are essential to find the optimal Nash equilibrium strategies and the optimal cost values of the LQ differential games. Furthermore, an iterative algorithm is given to solve the four-coupled GDREs. Finally, a suboptimal solution of the LQ differential games is proposed based on a convex optimization approach and a simplification of the suboptimal solution is given. Simulation examples are presented to illustrate the effectiveness of the iterative algorithm and the suboptimal solution.     

Existence of Solution for Generalized Coupled Differential Riccati Equation

By means of the recursive method, the existence of solution is obtained for the generalized coupled differential Riccati equation. As an application, we apply the existence results to consider the optimal control of Markovian jump linear singular system, and obtain the desired explicit representation of the optimal controller for the optimal control problem with the finite horizon.     

Maximal and Stabilizing Hermitian Solutions for Discrete-Time Coupled Algebraic Riccati Equations

Discrete-time coupled algebraic Riccati equations that arise in quadratic optimal control and H _∞-control of Markovian jump linear systems are considered. First, the equations that arise from the quadratic optimal control problem are studied. The matrix cost is only assumed to be hermitian. Conditions for the existence of the maximal hermitian solution are derived in terms of the concept of mean square stabilizability and a convex set not being empty. A connection with convex optimization is established, leading to a numerical algorithm. A necessary and sufficient condition for the existence of a stabilizing solution (in the mean square sense) is derived. Sufficient conditions in terms of the usual observability and detectability tests for linear systems are also obtained. Finally, the coupled algebraic Riccati equations that arise from the H _∞-control of discrete-time Markovian jump linear systems are analyzed. An algorithm for deriving a stabilizing solution, if it exists, is obtained. These results generalize and unify several previous ones presented in the literature of discrete-time coupled Riccati equations of Markovian jump linear systems. Date received: November 14, 1996. Date revised: January 12, 1999.     

Predictive control for networked systems affected by correlated packet loss

In this paper, we consider the predictive control problem of designing receding horizon controllers for networked linear systems subject to random packet loss in the controller to actuator link. The packet dropouts are temporarily correlated in the sense that they obey a Markovian transition model. Our design task is to solve the optimal controller that minimizes a given receding horizon cost function, using the available packet loss history. Due to the correlated nature of the packet loss, standard linear quadratic regulator methods do not apply. We first present the optimal control law by considering the correlations. This controller turns out to depend on the packet loss history and would typically require a large lookup table for implementation when the Markovian order is high. To address this issue, we present and compare several suboptimal design approaches to reduce the number of control laws.     

Model predictive control for networked control systems

This paper investigates the problem of model predictive control for a class of networked control systems. Both sensor‐to‐controller and controller‐to‐actuator delays are considered and described by Markovian chains. The resulting closed‐loop systems are written as jump linear systems with two modes. The control scheme is characterized as a constrained delay‐dependent optimization problem of the worst‐case quadratic cost over an infinite horizon at each sampling instant. A linear matrix inequality approach for the controller synthesis is developed. It is shown that the proposed state feedback model predictive controller guarantees the stochastic stability of the closed‐loop system. Copyright © 2008 John Wiley & Sons, Ltd.     

Optimal control of discrete-time linear fractional-order systems with multiplicative noise

A finite horizon linear quadratic (LQ) optimal control problem is studied for a class of discrete-time linear fractional systems (LFSs) affected by multiplicative, independent random perturbations. Based on the dynamic programming technique, two methods are proposed for solving this problem. The first one seems to be new and uses a linear, expanded-state model of the LFS. The LQ optimal control problem reduces to a similar one for stochastic linear systems and the solution is obtained by solving Riccati equations. The second method appeals to the principle of optimality and provides an algorithm for the computation of the optimal control and cost by using directly the fractional system. As expected, in both cases, the optimal control is a linear function in the state and can be computed by a computer program. A numerical example and comparative simulations of the optimal trajectory prove the effectiveness of the two methods. Some other simulations are obtained for different values of the fractional order.     

Jump linear quadratic regulator with controlled jump rates

Deals with the class of continuous-time linear systems with Markovian jumps. We assume that jump rates are controlled. Our purpose is to study the jump linear quadratic (JLQ) regulator of the class of systems. The structure of the optimal controller is established. For a one-dimensional (1-D) system, an algorithm for solving the corresponding set of coupled Riccati equations of this optimal control problem is provided. Two numerical examples are given to show the usefulness of our results     

线性Markov 切换系统的随机Nash 微分博弈及混合H2/H∞ 控制

研究线性Markov切换系统的随机Nash微分博弈问题。首先借助线性Markov切换系统随机最优控制的相关结果,得到了有限时域和无线时域Nash均衡解的存在条件等价于其相应微分(代数) Riccati方程存在解,并给出了最优解的显式形式;然后应用相应的微分博弈结果分析线性Markov切换系统的混合H2/H∞控制问题;最后通过数值算例验证了所提出方法的可行性。     

离散Markov跳变系统的滚动时域H_∞跟踪控制

针对离散Markov跳变系统,研究滚动时域H∞跟踪控制问题。为便于工程应用,假定系统当前时刻的状态和模态总是可测的,而系统未来时刻的模态是不可知的。利用庞特里亚金极小值原理,构造哈密尔顿函数,求解min-max优化问题,得到当前采样时刻的最优控制作用以及最严峻的外界扰动。控制器的求解可等效为在一组线性矩阵不等式约束条件下,迭代方程的可解性问题。控制律采用滚动时域结构,每次仅施加当前采样时刻计算得到的控制作用,在下一采样时刻将重新计算控制作用。该控制律保证系统在给定H∞扰动抑制水平的情形下,获得最优线性二次型性能指标以及良好的输出跟踪性能。最后仿真示例验证了该方法的可行性和有效性。     

Monte Carlo TD(λ)-methods for the optimal control of discrete-time Markovian jump linear systems

In this paper, we present an iterative technique based on Monte Carlo simulations for deriving the optimal control of the infinite horizon linear regulator problem of discrete-time Markovian jump linear systems for the case in which the transition probability matrix of the Markov chain is not known. We trace a parallel with the theory of TD(λ) algorithms for Markovian decision processes to develop a TD(λ) like algorithm for the optimal control associated to the maximal solution of a set of coupled algebraic Riccati equations (CARE). It is assumed that either there is a sample of past observations of the Markov chain that can be used for the iterative algorithm, or it can be generated through a computer program. Our proofs rely on the spectral radius of the closed loop operators associated to the mean square stability of the system being less than 1.     

Bumpless transfer for switched linear systems

This article deals with bumpless transfer for discrete-time switched linear systems. A bumpless transfer controller that is activated at every switching time for reducing the transient behavior is presented. The bumpless transfer control design is based on the finite horizon solution of a linear quadratic optimization problem. Dwell time conditions for assessing asymptotic stability of the closed-loop switched system are provided.     

The explicit linear quadratic regulator for constrained systems

For discrete-time linear time invariant systems with constraints on inputs and states, we develop an algorithm to determine explicitly, the state feedback control law which minimizes a quadratic performance criterion. We show that the control law is piece-wise linear and continuous for both the finite horizon problem (model predictive control) and the usual infinite time measure (constrained linear quadratic regulation). Thus, the on-line control computation reduces to the simple evaluation of an explicitly defined piecewise linear function. By computing the inherent underlying controller structure, we also solve the equivalent of the Hamilton-Jacobi-Bellman equation for discrete-time linear constrained systems. Control based on on-line optimization has long been recognized as a superior alternative for constrained systems. The technique proposed in this paper is attractive for a wide range of practical problems where the computational complexity of on-line optimization is prohibitive. It also provides an insight into the structure underlying optimization-based controllers.     

Controllability, stabilizability, and continuous-time Markovianjump linear quadratic control

Consideration is given to the control of continuous-time linear systems that possess randomly jumping parameters which can be described by finite-state Markov processes. The relationship between appropriately defined controllability, stabilizability properties, and the solution of the infinite time jump linear quadratic (JLQ) optimal control problems is also examined. Although the solution of the continuous-time Markov JLQ problem with finite or infinite time horizons is known, only sufficient conditions for the existence of finite cost, constant, stabilizing controls for the infinite time problem appear in the literature. In this paper necessary and sufficient conditions are established. These conditions are based on new definitions of controllability, observability, stabilizability, and detectability that are appropriate for continuous-time Markovian jump linear systems. These definitions play the same role for the JLQ problem as the deterministic properties do for the linear quadratic regulator (LQR) problem     

Uncoupled Riccati iterations for the linear quadratic controlproblem of discrete-time Markov jump linear systems

This paper deals with recursive methods for solving coupled Riccati equations arising in the linear quadratic control for Markovian jump linear systems. Two algorithms, based on solving uncoupled Riccati equations at each iteration, are presented. The standard method for this problem relies on finite stage approximations with receding horizon, whereas the methods presented here are based on sequences of stopping times to define the terminal time of the approximating control problems. The methods can be ordered in terms of rate of convergence. Comparisons with other methods in the current literature are also presented     

Robust stability and H-infinity control for uncertain discrete-time Markovian jump singular systems

The robust stability and stabilization, and H-infinity control problems for discrete-time Markovian jump singular systems with parameter uncertainties are discussed. Based on the restricted system equivalent （r.s.e.） transformation and by introducing new state vectors, the singular system is transformed into a discrete-time Markovian jump standard linear system, and the linear matrix inequality （LMI） conditions for the discrete-time Markovian jump singular systems to be regular, causal, stochastically stable, and stochastically stable with 7- disturbance attenuation are obtained, respectively. With these conditions, the robust state feedback stochastic stabilization problem and H-infinity control problem are solved, and the LMI conditions are obtained. A numerical example illustrates the effectiveness of the method given in the oaoer.     

Jump linear quadratic Gaussian control in continuous time

The optimal quadratic control of continuous-time linear systems that possess randomly jumping parameters which can be described by finite-state Markov processes is addressed. The systems are also subject to Gaussian input and measurement noise. The optimal solution for the jump linear-quadratic-Gaussian (JLQC) problem is given. This solution is based on a separation theorem. The optimal state estimator is sample-path dependent. If the plant parameters are constant in each value of the underlying jumping process, then the controller portion of the compensator converges to a time-invariant control law. However, the filter portion of the optimal infinite time horizon JLQC compensator is not time invariant. Thus, a suboptimal filter which does converge to a steady-state solution (under certain conditions) is derived, and a time-invariant compensator is obtained