Asynchronous H∞ filtering of continuous‐time Markov jump systems

The paper investigates the asynchronous H_∞ filtering design problem for continuous‐time linear systems with Markov jump. The hidden Markov jump principle is applied to represent the asynchronous situation between the target system and the designed filter. Via a Lyapunov technique, two sufficient conditions are developed to guarantee that the filtering error system is stochastically stable with a prescribed H_∞ noise attenuation level. Furthermore, three filtering design approaches are developed in the form of linear matrix inequalities. Finally, one example is provided to show the effectiveness and feasibility of the developed methods.     

基于扩展状态观测器的随机隐Markov正跳变系统有限时间异步控制

针对一类含不匹配扰动的随机隐Markov跳变系统, 本文研究了基于扩展状态观测器(ESO)的有限时间异步 控制问题. 首先, 引入一组扩展变量将隐Markov跳变系统转换成一组新的随机扩展系统, 补偿不匹配扰动对系统控 制输出的影响. 基于Lyapunov–Krasovskii泛函方法, 给出使得基于ESO的闭环随机隐Markov增广跳变系统是正系 统, 且有限时间有界的充分条件. 进而得到直接求解观测器增益和控制器增益的线性矩阵不等式. 最后, 通过仿真结 果验证了本文所设计的异步状态反馈控制器和观测器的有效性和可行性.     

Analysis and Estimation of the States of Special Jump Markov Processes. I. Martingale Representation

The first part of this paper was devoted to a class of continuous-time jump processes generalizing the finite-state Markov processes. Main characteristics of this process such as the transition probabilities, infinitesimal generator, and so on were established. Processes of this class were proved to be solutions of linear differential equations with a martingale in the right-hand side. Stochastic analysis of a hidden Markov model of evolution of risky assets was presented as an example.     

Hidden Markov model-based ℋ∞ control for singular Markov jump systems under denial of service attacks

This work investigates the ${\mathscr{H}}_{\infty }$ control problem of discrete-time singular Markov jump systems against denial of service attacks, in which the Markov state information is seen as restricted access. To solve this situation, a hidden Markov model is introduced. The main objective is to construct a controller with the help of hidden Markov model such that the stochastically admissible of the closed-loop singular Markov jump systems with limited access mode information can be guaranteed under denial of service attacks. To produce the needed hidden Markov model-based controller, a matrix contract transformation approach is developed. At the end, a numerical example and a tunnel diode circuit are presented to demonstrate the effectiveness and benefit of the design technique described in this research.     

Asynchronous Control for Positive Markov Jump Systems

A new result is provided for the asynchronous control analysis of positive Markov jump systems (PMJSs) in this paper. Firstly, a hidden Markov model is described to express the asynchronous circumstances that appear between the system modes and controller modes. Secondly, by utilizing a copositive stochastic Lyapunov function, a sufficient and necessary condition is given to guarantee the mean stability of PMJSs. Thirdly, we obtain another equivalent condition and design the corresponding asynchronous controller. Finally, the correctness of these results is verified by two numerical examples.

     

An auto-framing method for stochastic process signal by using a hidden Markov model based approach

In this paper, an “auto-framing” method, an algorithmic method to divide stochastic time-series process data into appropriate intervals, is developed based on the approach of hidden Markov model (HMM). While enormous amounts of process time-series data are being measured and collected today, their use is limited by the high costs to gather, store, and analyze them. “Data-framing” refers to the task of dividing stochastic signal data into time frames of distinct patterns so that the data can be stored and analyzed in an efficient manner. Data-framing is typically carried out manually, but doing so can be both laborious and ineffective. For the purpose of automating the data-framing task, stochastic signals of switching patterns are modeled using a hidden Markov model (HMM) based jump linear system (JLS), which switches the stochastic model probabilistically in accordance with the underlying Markov chain. Based on the model, an estimator is constructed to estimate from the collected signal data the state sequence of the underlying Markov chain, which is subsequently used to decide on the framing points. An Expectation Maximization (EM) algorithm, which is composed of two optimal estimators, fixed interval Kalman smoother and Viterbi algorithm, is used to estimate for the state estimation. We demonstrate the effectiveness of the HMM-based approach for auto-framing using simulated data constructed based on real industrial data.     

Hidden Markov model based filtering for singular semi-Markov jump systems

This article investigates the hidden Markov model based filter design problem for the singular semi-Markov jump systems (SSMJSs). The considered semi-Markov process is a generalization of Markov process, which can eliminate the restriction on the exponential distribution of sojourn time. Besides, the hidden Markov model based filter is introduced to tackle the asynchronous phenomenons occurred between the system modes and filter modes. To ensure the stochastic stability of the SSMJSs and derive solvable filter parameters, a filter design technic is constructed. First, the direct evolution of the states between two arbitrary close time instants is constructed from the filtering error system according to slow-fast decomposition, sufficient conditions are then proposed based on the consistent projector of the filtering error system and the constructed direct state evolution. Second, a new linear decoupling strategy is presented to deal with the coupled terms under the established stability conditions, which further derives the desired hidden Markov model based filter parameters. A numerical example is given to illustrate the effectiveness of the proposed method.     

一类丢包时延网络控制系统的鲁棒H∞滤波

用随机马尔可夫跳变系统描述一类具有丢包时延的网络控制系统．为这类网络控制系统设计马尔可夫跳变滤波器,保证了滤波误差系统均方意义下随机渐近稳定,且噪声信号对估计误差的影响低于指定H∞ 性能水平,滤波器参数可通过求解线性矩阵不等式得到．最后通过仿真实例验证了所得结论的正确性和滤波器设计方法的有效性．     

Review of several stochastic speech unit models

Hidden Markov models are commonly used for speech unit modelling. This type of model is composed of a non-observable or “hidden” process, representing the temporal structure of the speech unit, and an observation process linking the hidden process with the acoustic parameters extracted from the speech signal.Different types of hidden processes (Markov chain, semi-Markov chain, “expanded-state” Markov chain) as well as different types of observation processes (discrete, continuous, semi-continuous—multiple processes) are reviewed, showing their relationships. The maximum likelihood estimation of two-stage stochastic process parameters is presented in an a posteriori probability formalism. An intepretation of the expectation-maximization algorithm is proposed and the practical learning algorithms for hidden Markov models and hidden semi-Markov models are compared in terms of computation structure, probabilistic justification and complexity.This presentation is illustrated by experiments on a multi-speaker 130 isolated word recognition system. The implementation techniques are detailed and the different combinations of state occupancy modelling techniques and observation modelling techniques are studied from a practical point of view.     

H∞ control of continuous-time Markov jump linear systems with detector-based mode information†

This paper features new results on H_∞ analysis and control of linear systems with Markov jump disturbances, in a scenario of partial observations of the jump process. We consider the situations in which the jump process can only be measured through a suitable detector. A distinctive feature of the approach here is that it is general enough to encompass particular scenarios such as that of perfect information, no information (mode independent) and cluster observations of the Markov jump process. The main results, comprising a new bounded real lemma and a condition for state feedback synthesis, are expressed via linear matrix inequality-based optimisation problems. The method devised for the design of H_∞ controllers is applied to the control of an unmanned aerial vehicle model.     

General Smoothing Formulas for Markov-Modulated Poisson Observations

In this paper, we compute general smoothing dynamics for partially observed dynamical systems generating Poisson observations. We consider two model classes, each Markov modulated Poisson processes, whose stochastic intensities depend upon the state of an unobserved Markov process. In one model class, the hidden state process is a continuously-valued ItÔ process, which gives rise to a continuous sample-path stochastic intensity. In the other model class, the hidden state process is a continuous-time Markov chain, giving rise to a pure jump stochastic intensity. To compute filtered estimates of state process, we establish dynamics, whose solutions are unnormalized marginal probabilities; however, these dynamics include Lebesgue–Stieltjes stochastic integrals. By adapting the transformation techniques introduced by J. M. C. Clark, we compute filter dynamics which do not include these stochastic integrals. To construct smoothers, we exploit a duality between our forward and backward transformed dynamics and thereby completely avoid the technical complexities of backward evolving stochastic integral equations. The general smoother dynamics we present can readily be applied to specific smoothing algorithms, referred to in the literature as: Fixed point smoothing, fixed lag smoothing and fixed interval smoothing. It is shown that there is a clear motivation to compute smoothers via transformation techniques similar to those presented by J. M. C. Clark, that is, our smoothers are easily obtained without recourse to two sided stochastic integration. A computer simulation is included.     

H∞ filtering for singular Markovian jump systems with time delay

The problem of H_∞ filtering is considered for singular Markovian jump systems with time delay. In terms of linear matrix inequality (LMI) approach, a delay‐dependent bounded real lemma (BRL) is proposed for the considered system to be stochastically admissible while achieving the prescribed H_∞ performance condition. Based on the BRL and under partial knowledge of the jump rates of the Markov process, both delay‐dependent and delay‐independent sufficient conditions that guarantee the existence of the desired filter are presented. The explicit expression of the desired filter gains is also characterized by solving a set of strict LMIs. Some numerical examples are given to demonstrate the effectiveness of the proposed methods. Copyright © 2009 John Wiley & Sons, Ltd.     

Optimal estimation for continous system with jump process

The minimum variance estimator algorithm is derived for a class of linear continuous systems modulated by a multivalued jump Markov process. The approach adopted in this paper is as follows. First, we express the jump Markov process in terms of its initial value, the jump times and the values taken by the jump process after the jump, and then we apply the Bayes' rule and the general likelihood-ratio formula to obtain the a posteriori probability distribution of the jump process. The minimum variance estimate is given in terms of the a posteriori probability distribution of the jump process and the Kalman-filter estimates corresponding to the admissible values of the jump process. Simulation studies are also carried out to illustrate the behavior of the optimal estimator presented here.     

具有随机协议网络化系统的H_∞滤波

本文研究了一类具有随机介质访问协议网络化系统的H∞滤波问题.将传感器和滤波器的通信过程描述为一个马尔可夫链,进而将滤波误差系统建模成一个马尔可夫跳变系统.然后,运用李雅普诺夫方法和线性矩阵不等式技术,给出了滤波误差系统随机稳定且具有给定H∞性能的一个充分条件,并基于该条件给出了H∞滤波器的设计方法.最后的数值算例验证了本文方法的有效性.     

Pairwise Markov chains

We propose a model called a pairwise Markov chain (PMC), which generalizes the classical hidden Markov chain (HMC) model. The generalization, which allows one to model more complex situations, in particular implies that in PMC the hidden process is not necessarily a Markov process. However, PMC allows one to use the classical Bayesian restoration methods like maximum a posteriori (MAP), or maximal posterior mode (MPM). So, akin to HMC, PMC allows one to restore hidden stochastic processes, with numerous applications to signal and image processing, such as speech recognition, image segmentation, and symbol detection or classification, among others. Furthermore, we propose an original method of parameter estimation, which generalizes the classical iterative conditional estimation (ICE) valid for a classical hidden Markov chain model, and whose extension to possibly non-Gaussian and correlated noise is briefly treated. Some preliminary experiments validate the interest of the new model.     

Least mean square algorithms with Markov regime-switching limit

We analyze the tracking performance of the least mean square (LMS) algorithm for adaptively estimating a time varying parameter that evolves according to a finite state Markov chain. We assume the Markov chain jumps infrequently between the finite states at the same rate of change as the LMS algorithm. We derive mean square estimation error bounds for the tracking error of the LMS algorithm using perturbed Lyapunov function methods. Then combining results in two-time-scale Markov chains with weak convergence methods for stochastic approximation, we derive the limit dynamics satisfied by continuous-time interpolation of the estimates. Unlike most previous analyzes of stochastic approximation algorithms, the limit we obtain is a system of ordinary differential equations with regime switching controlled by a continuous-time Markov chain. Next, to analyze the rate of convergence, we take a continuous-time interpolation of a scaled sequence of the error sequence and derive its diffusion limit. Somewhat remarkably, for correlated regression vectors we obtain a jump Markov diffusion. Finally, two novel examples of the analysis are given for state estimation of hidden Markov models (HMMs) and adaptive interference suppression in wireless code division multiple access (CDMA) networks.     

Reduced-order filtering for linear systems with Markovian jump parameters

This paper addresses the reduced-order H_∞ filtering problem for continuous-time Makovian jump linear systems, where the jump parameters are modelled by a discrete-time Markov process. Sufficient conditions for the existence of the reduced-order H_∞ filter are proposed in terms of linear matrix inequalities (LMIs) and a coupling non-convex matrix rank constraint. In particular, the sufficient conditions for the existence of the zero-order H_∞ filter can be expressed in terms of a set of strict LMIs. The explicit parameterization of the desired filter is also given. Finally, a numerical example is given to illustrate the proposed approach.     

基于层次隐马尔可夫模型和神经网络的个性化推荐算法

传统推荐系统将推荐准确性作为主要目标,而推荐结果的多样性和个性化有所欠缺.对此,设计一种基于层次隐马尔可夫模型和神经网络的推荐算法.采用层次隐马尔可夫模型建模用户喜好和上下文环境的关系,并通过隐马尔可夫模型预测上下文.设计神经网络结构来解决协同过滤推荐的问题,同时神经网络满足贝叶斯个性化排序的条件,实现对推荐列表的个性...     

Reduced Stochastic Models for Complex Molecular Systems

We present a new numerical method for the identification of the most important metastable states of a system with complicated dynamical behavior from time series information. The approach is based on the representation of the effective dynamics of the full system by a Markov jump process between metastable states, and the dynamics within each of these metastable states by rather simple stochastic differential equations (SDEs). Its algorithmic realization exploits the concept of hidden Markov models (HMMs) with output behavior given by SDEs. A first complete algorithm including an explicit Euler–Maruyama-based likelihood estimator has already been presented in Horenko et al. (MMS, 2006a). Herein, we present a semi-implicit exponential estimator that, in contrast to the Euler–Maruyama-based estimator, also allows for reliable parameter optimization for time series where the time steps between single observations are large. The performance of the resulting method is demonstrated for some generic examples, in detail compared to the Euler–Maruyama-based estimator, and finally applied to time series originating from a 100 ns B-DNA molecular dynamics simulation.Dedicated to Peter Deuflhard on the occassion of his sixtieth birthday.Supported by the SfB 450 and DFG research center “Mathematics for key technologies” (FZT 86) in Berlin.     

Robust L2 − L∞ filtering for a class of dynamical systems with nonhomogeneous Markov jump process

This paper investigates the problem of robust L₂ ? L_∞ filtering for a class of dynamical systems with nonhomogeneous Markov jump process. The time-varying transition probabilities which evolve as a nonhomogeneous jump process are described by a polytope, and parameter-dependent and mode-dependent Lyapunov function is constructed for such system, and then a robust L₂ ? L_∞ filter is designed which guarantees that the resulting error dynamic system is robustly stochastically stable and satisfies a prescribed L₂ ? L_∞ performance index. A numerical example is given to illustrate the effectiveness of the developed techniques.