H∞ filtering for singular Markovian jump systems with time delay

The problem of H_∞ filtering is considered for singular Markovian jump systems with time delay. In terms of linear matrix inequality (LMI) approach, a delay‐dependent bounded real lemma (BRL) is proposed for the considered system to be stochastically admissible while achieving the prescribed H_∞ performance condition. Based on the BRL and under partial knowledge of the jump rates of the Markov process, both delay‐dependent and delay‐independent sufficient conditions that guarantee the existence of the desired filter are presented. The explicit expression of the desired filter gains is also characterized by solving a set of strict LMIs. Some numerical examples are given to demonstrate the effectiveness of the proposed methods. Copyright © 2009 John Wiley & Sons, Ltd.     

一类离散时间非齐次马尔可夫跳跃系统最优控制

本文研究了一类离散时间非齐次马尔可夫跳跃线性系统的线型二次高斯(linear quadratic Gaussian,LQG)问题,其中系统模态转移概率矩阵随时间随机变化,其变化特性由一高阶马尔可夫链描述.对于该系统的LQG问题,文中首先给出了线性最优滤波器,得到最优状态估计;其次,验证分离定理成立,并利用利用动态规划方法设计了系统最优控制器;最后,数值仿真结果验证了所设计控制器的有效性.     

Adaptive control of linear systems with Markov perturbations

The stochastic model considered is a linear jump diffusion process X for which the coefficients and the jump processes depend on a Markov chain Z with finite state space. First, we study the optimal filtering and control problem for these systems with non-Gaussian initial conditions, given noisy observations of the state X and perfect measurements of Z. We derive a new sufficient condition which ensures the existence and the uniqueness of the solution of the nonlinear stochastic differential equations satisfied by the output of the filter. We study a quadratic control problem and show that the separation principle holds. Next, we investigate an adaptive control problem for a state process X defined by a linear diffusion for which the coefficients depend on a Markov chain, the processes X and Z being observed in independent white noises. Suboptimal estimates for the process X, Z and approximate control law are investigated for a large class of probability distributions of the initial state. Asymptotic properties of these filters and this control law are obtained. Upper bounds for the corresponding error are given     

随机观测时滞系统的最优估计

研究了在观测中存在Markov跳跃时滞的离散时间系统的线性最小方差状态估计问题. 首先, 通过引入跳跃时滞的示性函数, 将带有跳跃时滞的观测方程转化为带有乘性噪声的定常时滞系统. 进一步采用状态扩维的方法, 将定常时滞系统转化为无时滞的Markov跳跃系统. 最后, 基于得到的无时滞系统, 采用Hilbert空间已有的几何论知识, 设计线性最优状态估计器, 得出基于Riccati方程的滤波器的表达式, 并证明了所得滤波器的渐渐收敛性.     

马尔可夫跳变线性系统最优控制的研究现状与进展

马尔可夫跳变线性系统（MJLS）是一种具有多个模态的随机系统，系统在各个模态之间的跳变转移由一组马尔可夫链来决定。MJLS模型因其在表示过程中可以产生突变而更能精确的描述实际工程应用中的系统。近年来，MJLS的最优控制问题成为了研究的热点，动态规划、极大值原理以及线性矩阵不等式等成为了解决此类问题的主流方法。本文对MJLS最优控制领域的研究现状进行了综述。分别对一般情况下、带有噪声的情况下、带有时滞的情况下以及某些特定情况下的MLJS最优控制问题的国内外研究现状进行论述。最后进行了总结并提出MJLS最优控制领域未来值得关注的研究方向。     

Kalman filtering for linear systems with coefficients driven by a hidden Markov jump process

We present an explicit algorithm for the solution of the nonlinear filtering problem concerning a linear, partially observed diffusion-type model, where the coefficients are driven by a hidden Markov jump process, of which one can observe the occurrence of a jump. The (recursive) filter is of a “branching” type with its (finite) dimension growing for each jump of the hidden Markov process.     

Analysis of linear asynchronous hybrid stochastic systems and its application to multi-agent systems with Markovian switching topologies

This paper studies the stability and stabilisation of a class of asynchronous hybrid stochastic systems described by linear Markov jump systems with asynchronous parameters induced by hysteretic mode and input delay. By exploring the essential properties of the Markov chain and developing novel Lyapunov function/functional based approaches, we derive sufficient conditions in terms of sizes of the delays ensuring the mean square asymptotical stability of the systems. Interestingly, the Lyapunov functional based approach further leads to the mean square stability criteria that depend only on the drift part and allow an arbitrary large delay for the diffusion part, which is different from the Lyapunov-Razumikhin based results. The theoretical results are applied to solve the consensus problems of multi-agent systems with Markovian switching topologies and input delay. Numerical examples demonstrate the established results.     

Mean-field stochastic linear–quadratic optimal control with Markov jump parameters

This paper considers a class of mean-field stochastic linear–quadratic optimal control problems with Markov jump parameters. The new feature of these problems is that means of state and control are incorporated into the systems and the cost functional. Based on the modes of Markov chain, the corresponding decomposition technique of augmented state and control is introduced. It is shown that, under some appropriate conditions, there exists a unique optimal control, which can be explicitly given via solutions of two generalized difference Riccati equations. A numerical example sheds light on the theoretical results established.     

Optimal linear estimation for continuous stochastic systems with random observation delays

This paper is concerned with the linear minimum mean square error estimation for Itô‐type differential equation systems with random delays, where the delay process is modeled as a finite‐state Markov chain. By first introducing a set of equivalent delay‐free observations and then defining two reorganized Markov chains, the estimation problem of random delayed systems is reduced to the one of delay‐free Markov jump linear systems. The estimator is derived by using the innovation analysis method based on the Itô differential formula. And the analytical solution to this estimator is given in terms of two Riccati differential equations that are of finite dimensions. Conditions for existence, uniqueness, and stability of the steady‐state optimal estimator are studied for time‐invariant cases. In this case, the obtained estimator is very easy to implement, and all calculation can be performed off line, leading to a linear time‐invariant estimator. Copyright © 2011 John Wiley & Sons, Ltd.     

An LP approach to full-order and reduced-order state estimations of positive Markov jump systems with delay

In this paper, the problems of full-order and reduced-order positive state estimations are developed for discrete-time positive Markov jump linear systems with delay. By establishing a positive time-delay system for 1-moment function of the state vector, delay-dependent necessary and sufficient exponential stability conditions are first derived based on a comparison technique for positive systems. Then, on the basis of the derived stability conditions, the problems of designing positive state observers in full-order and reduced-order for positive Markov jump systems with delay are addressed. The existence conditions of positive observers and the design schemes are formulated in terms of linear programming problems which can be effectively solved by various optimisation algorithms to obtain the observer parameters. Finally, three numerical examples with simulations are given to validate the analysis and design results.     

Linear estimation for random delay systems

This paper is concerned with the linear estimation problems for discrete-time systems with random delayed observations. When the random delay is known online, i.e., time-stamped, the random delayed system is reconstructed as an equivalent delay-free one by using measurement reorganization technique, and then an optimal linear filter is presented based on the Kalman filtering technique. However, the optimal filter is time-varying, stochastic, and does not converge to a steady state in general. Then an alternative suboptimal filter with deterministic gains is developed under a new criteria. The estimator performance in terms of their error covariances is provided, and its mean square stability is established. Finally, a numerical example is presented to illustrate the efficiency of proposed estimators.     

Fault detection for discrete-time networked nonlinear systems with incomplete measurements

This article addresses the problem of fault detection (FD) for discrete-time networked systems with global Lipschitz nonlinearities and incomplete measurements, including time delays, packet dropouts and signal quantisation. By utilising a discrete-time homogeneous Markov chain, an improved model which considers packet dropout compensation has been proposed to describe the above network-induced phenomena. We aim to design a mode-dependent fault detection filter (FDF) such that the FD system is asymptotically mean-square stable and satisfies a prescribed attenuation level. The addressed FD problem is then converted into an auxiliary H _∞ filtering problem of Markov jump system with time-varying delay. A sufficient condition for the existence of the FDF is derived in terms of certain linear matrix inequalities (LMIs). When these LMIs are feasible, the explicit expression of the desired FDF can also be characterised. A numerical example is exploited to show the effectiveness of the results obtained.     

Monte Carlo TD(λ)-methods for the optimal control of discrete-time Markovian jump linear systems

In this paper, we present an iterative technique based on Monte Carlo simulations for deriving the optimal control of the infinite horizon linear regulator problem of discrete-time Markovian jump linear systems for the case in which the transition probability matrix of the Markov chain is not known. We trace a parallel with the theory of TD(λ) algorithms for Markovian decision processes to develop a TD(λ) like algorithm for the optimal control associated to the maximal solution of a set of coupled algebraic Riccati equations (CARE). It is assumed that either there is a sample of past observations of the Markov chain that can be used for the iterative algorithm, or it can be generated through a computer program. Our proofs rely on the spectral radius of the closed loop operators associated to the mean square stability of the system being less than 1.     

A linear distributed filter inspired by the Markovian jump linear system filtering problem

In this paper we introduce a consensus-based distributed filter, executed by a sensor network, inspired by the Markovian jump linear system filtering theory. We show that the optimal filtering gains of the Markovian jump linear system can be used as an approximate solution of the optimal distributed filtering problem. This parallel allows us to interpret each filtering gain corresponding to a mode of operation of the Markovian jump linear system as a filtering gain corresponding to a sensor in the network. The approximate solution can be implemented distributively and guarantees a quantifiable level of performance.     

A partially delay‐dependent and disordered controller design for discrete‐time delayed systems

This paper considers the stabilization problem for a class of discrete‐time delayed systems by exploiting a partially delay‐dependent controller whose gains suffer a disordering phenomenon simultaneously. Two stochastic variables are used to describe the partially delay‐dependent and disordering properties, which are not independent, and referred to the original operation modes here. By introducing an augmented Markov chain, the corresponding closed‐loop system is transformed into a Markovian jump system with four new operation modes (NOMs). Based on the proposed model, a kind of controller depending on NOMs is firstly proposed with linear matrix inequalities forms. Moreover, without designing a controller containing NOMs directly, another kind of stabilizing controller referring to one depending on original operation modes is developed, which is composed of a series of NOM‐dependent controllers and satisfies a minimum variance approximation. Finally, two numerical examples are used to demonstrate the utility and superiority of the proposed methods. Copyright © 2016 John Wiley & Sons, Ltd.     

基于扩展状态观测器的随机隐Markov正跳变系统有限时间异步控制

针对一类含不匹配扰动的随机隐Markov跳变系统, 本文研究了基于扩展状态观测器(ESO)的有限时间异步 控制问题. 首先, 引入一组扩展变量将隐Markov跳变系统转换成一组新的随机扩展系统, 补偿不匹配扰动对系统控 制输出的影响. 基于Lyapunov–Krasovskii泛函方法, 给出使得基于ESO的闭环随机隐Markov增广跳变系统是正系 统, 且有限时间有界的充分条件. 进而得到直接求解观测器增益和控制器增益的线性矩阵不等式. 最后, 通过仿真结 果验证了本文所设计的异步状态反馈控制器和观测器的有效性和可行性.     

Optimal estimation for continuous-time Markovian jump linear systems with delayed measurements

This paper is to investigate the linear minimum mean square error estimation for continuous-time Markovian jump linear systems with delayed measurements. The key technique applied for treating the measurement delay is the reorganization innovation analysis, by which the state estimation with delayed measurements is transformed into a standard linear mean square filter of an associated delay-free system. The optimal filter is derived based on the innovation analysis method together with geometric arguments in Hilbert space. An analytical solution to the filter is obtained in terms of two Riccati differential equations, and hence is very simple in computation. Computer simulations are carried out to evaluate the performance of the proposed algorithms. The problem of tracking a maneuvering target is addressed.     

Stability analysis and stabilization of networked linear systems with random packet losses

This paper is concerned with the stability analysis and stabilization of networked discrete-time and sampled-data linear systems with random packet losses. Asymptotic stability, mean-square stability, and stochastic stability are considered. For networked discrete-time linear systems, the packet loss period is assumed to be a finite-state Markov chain. We establish that the mean-square stability of a related discrete-time system which evolves in random time implies the mean-square stability of the system in deterministic time by using the equivalence of stability properties of Markovian jump linear systems in random time. We also establish the equivalence of asymptotic stability for the systems in deterministic discrete time and in random time. For networked sampled-data systems, a binary Markov chain is used to characterize the packet loss phenomenon of the network. In this case, the packet loss period between two transmission instants is driven by an identically independently distributed sequence assuming any positive values. Two approaches, namely the Markov jump linear system approach and randomly sampled system approach, are introduced. Based on the stability results derived, we present methods for stabilization of networked sampled-data systems in terms of matrix inequalities. Numerical examples are given to illustrate the design methods of stabilizing controllers.     

网络控制系统随机稳定性研究

研究了具有随机网络诱导时延及数据包丢失的网络控制系统随机稳定性问题. 本文用一个具有两个状态的马尔可夫链来描述数据通过网络传输时随机数据包丢失过程, 利用马尔可夫跳变线性系统理论, 将网络控制系统建模为一个具有两种运行模式的马尔可夫跳变线性系统, 给出了在状态反馈控制下网络控制系统随机稳定的线性矩阵不等式形式的充分条件, 最后用一个仿真示例验证了该方法的有效性.     

一类丢包时延网络控制系统的鲁棒H∞滤波

用随机马尔可夫跳变系统描述一类具有丢包时延的网络控制系统．为这类网络控制系统设计马尔可夫跳变滤波器,保证了滤波误差系统均方意义下随机渐近稳定,且噪声信号对估计误差的影响低于指定H∞ 性能水平,滤波器参数可通过求解线性矩阵不等式得到．最后通过仿真实例验证了所得结论的正确性和滤波器设计方法的有效性．