Finite horizon quadratic optimal control and a separation principle for Markovian jump linear systems

In this note, we consider the finite-horizon quadratic optimal control problem of discrete-time Markovian jump linear systems driven by a wide sense white noise sequence. We assume that the output variable and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed-loop system minimizes the quadratic functional cost of the system over a finite horizon period of time. As in the case with no jumps, we show that an optimal controller can be obtained from two coupled Riccati difference equations, one associated to the optimal control problem when the state variable is available, and the other one associated to the optimal filtering problem. This is a principle of separation for the finite horizon quadratic optimal control problem for discrete-time Markovian jump linear systems. When there is only one mode of operation our results coincide with the traditional separation principle for the linear quadratic Gaussian control of discrete-time linear systems.     

H ∞ filtering for Markovian jump linear systems

The problem of H X filtering for continuous-time linear systems with Markovian jump is investigated. It was assumed that the jumping parameter was available. This paper develops necessary and sufficient conditions for designing a Markovian jump linear filter that ensures a prescribed bound on the L 2 -induced gain from the noise signals to the estimation error. The main result is tailored via linear matrix inequalities.     

Robust ℋ︁∞ filtering for uncertain Markovian jump linear systems

This paper investigates the problem of ??_∞ filtering for a class of uncertain Markovian jump linear systems. The uncertainty is assumed to be norm‐bounded and appears in all the matrices of the system state‐space model, including the coefficient matrices of the noise signals. It is also assumed that the jumping parameter is available. We develop a methodology for designing a Markovian jump linear filter that ensures a prescribed bound on the ??₂‐induced gain from the noise signals to the estimation error, irrespective of the uncertainty. The proposed design is given in terms of linear matrix inequalities. Copyright © 2002 John Wiley & Sons, Ltd.     

具有随机丢包的一类网络控制系统的故障检测

针对存在随机数据包丢失的网络环境,研究了一类网络控制系统的故障检测问题.考虑随机丢包同时发生在传感器与控制器以及控制器与执行器之间,将网络控制系统建模为含有4个模态的马尔可夫跳变线性系统.基于此类模型,构造了系统的残差发生器,相应的故障检测问题转化为H∞滤波问题.利用马尔可夫跳变线性系统理论,设计了故障检测滤波器,使得...     

Jump linear quadratic regulator with controlled jump rates

Deals with the class of continuous-time linear systems with Markovian jumps. We assume that jump rates are controlled. Our purpose is to study the jump linear quadratic (JLQ) regulator of the class of systems. The structure of the optimal controller is established. For a one-dimensional (1-D) system, an algorithm for solving the corresponding set of coupled Riccati equations of this optimal control problem is provided. Two numerical examples are given to show the usefulness of our results     

<Emphasis Type="BoldItalic">H</Emphasis><Subscript>2</Subscript>-Control and the Separation Principle for Discrete-Time Markovian Jump Linear Systems

In this paper we consider the H₂-control problem of discrete-time Markovian jump linear systems. We assume that only an output and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed-loop system is mean square stable and minimizes the H₂-norm of the system. As in the case with no jumps, we show that an optimal controller can be obtained from two sets of coupled algebraic Riccati equations, one associated with the optimal control problem when the state variable is available, and the other associated with the optimal filtering problem. This is the principle of separation for discrete-time Markovian jump linear systems. When there is only one mode of operation our results coincide with the traditional separation principle for the H₂-control of discrete-time linear systems. Date received: June 1, 2001. Date revised: October 13, 2003.     

Distributed fault detection over sensor networks with Markovian switching topologies

This paper deals with the distributed fault detection for discrete-time Markov jump linear systems over sensor networks with Markovian switching topologies. The sensors are scatteredly deployed in the sensor field and the fault detectors are physically distributed via a communication network. The system dynamics changes and sensing topology variations are modeled by a discrete-time Markov chain with incomplete mode transition probabilities. Each of these sensor nodes firstly collects measurement outputs from its all underlying neighboring nodes, processes these data in accordance with the Markovian switching topologies, and then transmits the processed data to the remote fault detector node. Network-induced delays and accumulated data packet dropouts are incorporated in the data transmission between the sensor nodes and the distributed fault detector nodes through the communication network. To generate localized residual signals, mode-independent distributed fault detection filters are proposed. By means of the stochastic Lyapunov functional approach, the residual system performance analysis is carried out such that the overall residual system is stochastically stable and the error between each residual signal and the fault signal is made as small as possible. Furthermore, a sufficient condition on the existence of the mode-independent distributed fault detection filters is derived in the simultaneous presence of incomplete mode transition probabilities, Markovian switching topologies, network-induced delays, and accumulated data packed dropouts. Finally, a stirred-tank reactor system is given to show the effectiveness of the developed theoretical results.     

Distributed filtering for discrete-time linear systems with fading measurements and time-correlated noise

This paper studies the problem of distributed filtering for discrete-time linear systems with fading measurements and time-correlated noise over a sensor network. To address the problem of the time-correlated measurement noise, the measurement differencing approach is adopted to define a new measurement such that the noise in the new measurement is not time-correlated any longer. Based on the new measurement, the innovation-based and the consensus-based distributed filters are proposed for each sensor by using its neighboring information. By resorting to the graph properties, the filter gain matrices are designed for each sensor to develop optimal distributed filters in the sense of minimum variance. Moreover, suboptimal distributed filters are proposed to reduce the computational cost and the communication cost. The performance of the distributed filters is analyzed with respect to the fading factor. Simulation results are provided to show the effectiveness of the proposed filters.     

Robust linear filtering for discrete-time hybrid Markov linear systems

In this paper we consider the robust linear filtering of hybrid discrete-time Markovian jump linear systems. We assume that only an output of the system is available, and therefore the values of the jump parameter are not known. It is desired to design a dynamic linear filter such that the closed loop system is mean square stable and minimizes the stationary expected value of the square error. We consider uncertainties on the parameters of the possible modes of operation of the system. A linear matrix inequalities (LMI) formulation is proposed to solve the problem. For the case in which there are no uncertainties on the modes of operation of the system, we show that the LMI formulation provides a filter with the same stationary mean square error as the one obtained from the Riccati equation approach.     

H infintity control of large-scale jump linear systems via averaging and aggregation

This paper studies, under state feedback policies, the H infinity control design for large-scale jump linear systems where the form process admits strong and weak interactions. Through an analysis that covers both finite and infinite horizon cases and using averaging and aggregation techniques, an aggregate jump linear system of considerably smaller order has been obtained, along with a corresponding (compatible) cost function. This reduced-order (aggregate) problem is another piecewise-deterministic H infinity control problem, and, on the basis of the solution of this problem, we obtain the asymptotic limit of the optimal performance level for the full-order system, as well as an approximate controller that can asymptotically achieve any desired performance level for the full-order system. A by-product of this analysis is a similar decomposition for the piecewise-deterministic linear-regulator problem.     

Controllability, stabilizability, and continuous-time Markovianjump linear quadratic control

Consideration is given to the control of continuous-time linear systems that possess randomly jumping parameters which can be described by finite-state Markov processes. The relationship between appropriately defined controllability, stabilizability properties, and the solution of the infinite time jump linear quadratic (JLQ) optimal control problems is also examined. Although the solution of the continuous-time Markov JLQ problem with finite or infinite time horizons is known, only sufficient conditions for the existence of finite cost, constant, stabilizing controls for the infinite time problem appear in the literature. In this paper necessary and sufficient conditions are established. These conditions are based on new definitions of controllability, observability, stabilizability, and detectability that are appropriate for continuous-time Markovian jump linear systems. These definitions play the same role for the JLQ problem as the deterministic properties do for the linear quadratic regulator (LQR) problem     

一类离散时间非齐次马尔可夫跳跃系统最优控制

本文研究了一类离散时间非齐次马尔可夫跳跃线性系统的线型二次高斯(linear quadratic Gaussian,LQG)问题,其中系统模态转移概率矩阵随时间随机变化,其变化特性由一高阶马尔可夫链描述.对于该系统的LQG问题,文中首先给出了线性最优滤波器,得到最优状态估计;其次,验证分离定理成立,并利用利用动态规划方法设计了系统最优控制器;最后,数值仿真结果验证了所设计控制器的有效性.     

Robust H 2-control for discrete-time Markovian jump linear systems

This paper deals with the robust H₂-control of discrete-time Markovian jump linear systems. It is assumed that both the state and jump variables are available to the controller. Uncertainties satisfying some norm bounded conditions are considered on the parameters of the system. An upper bound for the H₂-control problem is derived in terms of a linear matrix inequality (LMI) optimization problem. For the case in which there are no uncertainties, we show that the convex formulation is equivalent to the existence of the mean square stabilizing solution for the set of coupled algebraic Riccati equations arising on the quadratic optimal control problem of discrete-time Markovian jump linear systems. Therefore, for the case with no uncertainties, the convex formulation considered in this paper imposes no extra conditions than those in the usual dynamic programming approach. Finally some numerical examples are presented to illustrate the technique.     

ℋ︁∞ filtering for discrete‐time linear systems with Markovian jumping parameters†

This paper investigates the problem of ??_∞ filtering for discrete‐time linear systems with Markovian jumping parameters. It is assumed that the jumping parameter is available. This paper develops necessary and sufficient conditions for designing a discrete‐time Markovian jump linear filter which ensures a prescribed bound on the ?₂‐induced gain from the noise signals to the estimation error. The proposed filter design is given in terms of linear matrix inequalities. Copyright © 2003 John Wiley & Sons, Ltd.     

Adaptive control of linear systems with Markov perturbations

The stochastic model considered is a linear jump diffusion process X for which the coefficients and the jump processes depend on a Markov chain Z with finite state space. First, we study the optimal filtering and control problem for these systems with non-Gaussian initial conditions, given noisy observations of the state X and perfect measurements of Z. We derive a new sufficient condition which ensures the existence and the uniqueness of the solution of the nonlinear stochastic differential equations satisfied by the output of the filter. We study a quadratic control problem and show that the separation principle holds. Next, we investigate an adaptive control problem for a state process X defined by a linear diffusion for which the coefficients depend on a Markov chain, the processes X and Z being observed in independent white noises. Suboptimal estimates for the process X, Z and approximate control law are investigated for a large class of probability distributions of the initial state. Asymptotic properties of these filters and this control law are obtained. Upper bounds for the corresponding error are given     

跳变约束下马尔可夫切换非线性系统滤波

针对系统状态演化多模不确定性和状态约束多样性,本文提出了跳变约束下马尔可夫切换非线性系统的交互式多假设估计方法.定义了包含跳变马尔可夫参数可能取值的假设集,根据最优贝叶斯滤波,推导出状态与假设的后验概率递推更新.基于统计线性回归线性化非线性函数,利用伪量测法,将线性化的约束扩维到真实量测中,给出了非线性系统滤波的近似解析最优解.最终给出所提算法的稀疏网格积分近似最优估计实现.在交叉道路机动目标跟踪仿真场景中,所提算法的滤波精度优于基于泰勒展开的交互式多模型算法,基于统计线性回归的交互式多模型算法,以及基于泰勒展开的非线性系统约束滤波算法.     

Optimal time-weighted H 2 model reduction for Markovian jump systems

This article addresses the optimal time-weighted H ₂ model reduction problem for Markovian jump linear systems. That is, for a given mean square stable Markovian jump system, our aim is to find a mean square stable jump system of lower order such that the time-weighted H ₂ norm of the corresponding error system is minimised. The time-weighted H ₂ norm of the system is first defined, and then a computational method is constructed. The computation requires the solution of two sets of recursive Lyapunov-type linear matrix equations associated with the Markovian jump system. To solve the optimal time-weighted H ₂ model reduction problem, we propose a gradient flow method for its solution. A necessary condition for minimality is derived, and a computational procedure is provided to obtain the minimising reduced-order model. The necessary condition generalises the standard result for systems when Markov jumps and the time-weighting term do not appear. Finally, two numerical examples are given to demonstrate the effectiveness of the proposed approach.     

Monte Carlo TD(λ)-methods for the optimal control of discrete-time Markovian jump linear systems

In this paper, we present an iterative technique based on Monte Carlo simulations for deriving the optimal control of the infinite horizon linear regulator problem of discrete-time Markovian jump linear systems for the case in which the transition probability matrix of the Markov chain is not known. We trace a parallel with the theory of TD(λ) algorithms for Markovian decision processes to develop a TD(λ) like algorithm for the optimal control associated to the maximal solution of a set of coupled algebraic Riccati equations (CARE). It is assumed that either there is a sample of past observations of the Markov chain that can be used for the iterative algorithm, or it can be generated through a computer program. Our proofs rely on the spectral radius of the closed loop operators associated to the mean square stability of the system being less than 1.     

Output feedback control of Markov jump linear systems incontinuous-time

This paper addresses the dynamic output feedback control problem of continuous-time Markovian jump linear systems. The fundamental point in the analysis is an LMI characterization, comprising all dynamical compensators that stabilize the closed-loop system in the mean square sense. The H₂ and H_∞-norm control problems are studied, and the H₂ and H_∞ filtering problems are solved as a by product     

延迟不确定马尔科夫跳变系统的执行器和传感器故障同时估计方法

针对具有参数不确定和延迟环节的马尔科夫跳变系统,在状态转移概率矩阵（Transition probability matrix,TPM）不确定的情形下,讨论了其执行器和传感器故障同时估计的方法.通过扩展系统状态,将系统转换为一个具有马尔科夫跳变参数的广义描述系统,基于此广义描述系统设计马尔科夫跳变观测器实现对其状态和传感器故障的估计.与此同时,还设计了一组自适应律对执行器故障进行在线调节.通过求解一组线性矩阵不等式最优化问题,得到观测器存在的充分条件.最后,针对两个数值实例,验证了所设计方法的有效性.