Time discretization of continuous-time filters and smoothers forHMM parameter estimation

In this paper we propose algorithms for parameter estimation of fast-sampled homogeneous Markov chains observed in white Gaussian noise. Our algorithms are obtained by the robust discretization of stochastic differential equations involved in the estimation of continuous-time hidden Markov models (HMM's) via the EM algorithm. We present two algorithms: the first is based on the robust discretization of continuous-time filters that were recently obtained by Elliott to estimate quantities used in the EM algorithm; the second is based on the discretization of continuous-time smoothers, yielding essentially the well-known Baum-Welch re-estimation equations. The smoothing formulas for continuous-time HMM's are new, and their derivation involves two-sided stochastic integrals. The choice of discretization results in equations which are identical to those obtained by deriving the results directly in discrete time. The filter-based EM algorithm has negligible memory requirements; indeed, independent of the number of observations. In comparison the smoother-based discrete-time EM algorithm requires the use of the forward-backward algorithm, which is a fixed-interval smoothing algorithm and has memory requirements proportional to the number of observations. On the other hand, the computational complexity of the filter-based EM algorithm is greater than that of the smoother-based scheme. However, the filters may be suitable for parallel implementation. Using computer simulations we compare the smoother-based and filter-based EM algorithms for HMM estimation. We provide also estimates for the discretization error     

An EM Algorithm for Markov Modulated Markov Processes

An expectation–maximization (EM) algorithm for estimating the parameter of a Markov modulated Markov process in the maximum likelihood sense is developed. This is a doubly stochastic random process with an underlying continuous-time finite-state homogeneous Markov chain. Conditioned on that chain, the observable process is a continuous-time finite-state nonhomogeneous Markov chain. The generator of the observable process at any given time is determined by the state of the underlying Markov chain at that time. The parameter of the process comprises the set of generators for the underlying and conditional Markov chains. The proposed approach generalizes an earlier approach by RydÉn for estimating the parameter of a Markov modulated Poisson process.      

Parameter estimation for Markov modulated Poisson processes via the EM algorithm with time discretization

The Markov modulated Poisson process (MMPP) has been proposed as a suitable model for characterizing the input traffic to a statistical multiplexer [6]. This paper describes a novel method of parameter estimation for MMPPs. The idea is to employ time discretization to convert an MMPP from the continuous-time domain into the discrete-time domain and then to use a powerful statistical inference technique, known as the EM algorithm, to obtain maximum-likelihood estimates of the model parameters. Tests conducted through a series of simulation experiments indicate that the new method yields results that are significantly more accurate compared to the method described in [8]. In addition, the new method is more flexible and general in that it is applicable to MMPPs with any number of states while retaining nearly constant simplicity in its implementation. Detailed experimental results on the sensitivity of the estimation accuracy to (1) the initialization of the model, (2) the size of the observation interarrival interval data available for the estimation, and (3) the inherent separability of the MMPP states are presented.     

Expectation maximization algorithms for MAP estimation of jumpMarkov linear systems

In a jump Markov linear system, the state matrix, observation matrix, and the noise covariance matrices evolve according to the realization of a finite state Markov chain. Given a realization of the observation process, the aim is to estimate the state of the Markov chain assuming known model parameters. Computing conditional mean estimates is infeasible as it involves a cost that grows exponentially with the number of observations. We present three expectation maximization (EM) algorithms for state estimation to compute maximum a posteriori (MAP) state sequence estimates [which are also known as Bayesian maximum likelihood state sequence estimates (MLSEs)]. The first EM algorithm yields the MAP estimate for the entire sequence of the finite state Markov chain. The second EM algorithm yields the MAP estimate of the (continuous) state of the jump linear system. The third EM algorithm computes the joint MAP estimate of the finite and continuous states. The three EM algorithms optimally combine a hidden Markov model (HMM) estimator and a Kalman smoother (KS) in three different ways to compute the desired MAP state sequence estimates. Unlike the conditional mean state estimates, which require computational cost exponential in the data length, the proposed iterative schemes are linear in the data length     

Estimation, Hypothesis Testing and Parameter Correlation for Markov Chains

In this report we will discuss four problems in statistical inference for Markov chains. Specifically, techniques are described to do the following: 1) estimate the transition probabilities of first- and second-order stationary Markov chains; 2) test the hypothesis that a stationary Markov chain is of first order against the alternate hypothesis that the chain is of second order; and 3) test the hypothesis that a first-order Markov chain has stationary transition probabilities against the alternate hypothesis that the transition probabilities are not stationary. A technique is also developed which can be used in testing to determine whether two parameters of a single electronic component drift independently of each other. The results of these tests are used to draw some inference about continuous-time Markov processes.     

Discrete Markov image modeling and inference on the quadtree

Noncasual Markov (or energy-based) models are widely used in early vision applications for the representation of images in high-dimensional inverse problems. Due to their noncausal nature, these models generally lead to iterative inference algorithms that are computationally demanding. In this paper, we consider a special class of nonlinear Markov models which allow one to circumvent this drawback. These models are defined as discrete Markov random fields (MRF) attached to the nodes of a quadtree. The quadtree induces causality properties which enable the design of exact, noniterative inference algorithms, similar to those used in the context of Markov chain models. We first introduce an extension of the Viterbi algorithm which enables exact maximum a posteriori (MAP) estimation on the quadtree. Two other algorithms, related to the MPM criterion and to Bouman and Shapiro's (1994) sequential-MAP (SMAP) estimator are derived on the same hierarchical structure. The estimation of the model hyper parameters is also addressed. Two expectation-maximization (EM)-type algorithms, allowing unsupervised inference with these models are defined. The practical relevance of the different models and inference algorithms is investigated in the context of image classification problem, on both synthetic and natural images.     

Improvement of integrated services on CDMA wireless LANs using thehandshake-ALOHA protocol

The author proposes a frame-based, handshake-ALOHA protocol suitable for CDMA wireless LANs. Performance evaluation using a continuous-time Markov chain and Poisson error models shows that both the data and voice throughput are significantly improved, compared to the pure ALOHA protocol     

Simulated annealing for maximum a posteriori parameter estimationof hidden Markov models

Hidden Markov models are mixture models in which the populations from one observation to the next are selected according to an unobserved finite state-space Markov chain. Given a realization of the observation process, our aim is to estimate both the parameters of the Markov chain and of the mixture model in a Bayesian framework. We present an original simulated annealing algorithm which, in the same way as the EM (expectation-maximization) algorithm, relies on data augmentation, and is based on stochastic simulation of the hidden Markov chain. This algorithm is shown to converge toward the set of maximum a posteriori (MAP) parameters under suitable regularity conditions     

Maximum-likelihood parameter estimation for unsupervised stochasticmodel-based image segmentation

An unsupervised stochastic model-based approach to image segmentation is described, and some of its properties investigated. In this approach, the problem of model parameter estimation is formulated as a problem of parameter estimation from incomplete data, and the expectation-maximization (EM) algorithm is used to determine a maximum-likelihood (ML) estimate. Previously, the use of the EM algorithm in this application has encountered difficulties since an analytical expression for the conditional expectations required in the EM procedure is generally unavailable, except for the simplest models. In this paper, two solutions are proposed to solve this problem: a Monte Carlo scheme and a scheme related to Besag's (1986) iterated conditional mode (ICM) method. Both schemes make use of Markov random-field modeling assumptions. Examples are provided to illustrate the implementation of the EM algorithm for several general classes of image models. Experimental results on both synthetic and real images are provided.     

Modeling and analysis of communication systems based oncomputational methods for Markov chains

The authors describe a computational approach for modeling and analyzing modern communication systems based on numerical methods for Markov chains. Advanced direct and iterative procedures for the calculation of the stationary distribution of a homogeneous discrete- or continuous-time Markov chain with finite state space are presented. They are implemented in a convenient software tool called MACOM for interactive modeling and performance evaluation of communication systems. MACOM provides the user with a predefined markovian model world describing modern telecommunication networks with adaptive routing schemes and advanced congestion-control mechanisms. The versatility of these algorithms is illustrated by their application to Markovian queuing models derived from telecommunications networks     

Numerical methods for reliability evaluation of Markov closedfault-tolerant systems

This paper compares three numerical methods for reliability calculation of Markov, closed, fault-tolerant systems which give rise to continuous-time, time-homogeneous, finite-state, acyclic Markov chains. The authors consider a modified version of Jensen's method (a probabilistic method, also known as uniformization or randomization), a new version of ACE (acyclic Markov chain evaluator) algorithm with several enhancements, and a third-order implicit Runge-Kutta method (an ordinary-differential-equation solution method). Modifications to Jensen's method include incorporating stable calculation of Poisson probabilities and steady-state detection of the underlying discrete-time Markov chain. The new version of Jensen's method is not only more efficient but yields more accurate results. Modifications to ACE algorithm are proposed which incorporate scaling and other refinements to make it more stable and accurate. However, the new version no longer yields solution symbolic with respect to time variable. Implicit Runge-Kutta method can exploit the acyclic structure of the Markov chain and therefore becomes more efficient. All three methods are implemented. Several reliability models are numerically solved using these methods and the results are compared on the basis of accuracy and computation cost     

MAP decoding in channels with memory

The expectation-maximization (EM) algorithm is popular in estimating the parameters of various statistical models. We consider applications of the EM algorithm to the maximum a posteriori (MAP) sequence decoding assuming that sources and channels are described by hidden Markov models (HMMs). The HMMs can accurately approximate a large variety of communication channels with memory and, in particular, wireless fading channels with noise. The direct maximization of the a posteriori probability (APP) is too complex. The EM algorithm allows us to obtain the MAP sequence estimation iteratively. Since each step of the EM algorithm increases the APP, the algorithm can improve the performance of any decoding procedure     

Robust hurwitz property of bivariate polynomials in the space of Markov parameters

Markov parameters and the associated stability criterion were first introduced for continuous-time real polynomials. Recently, robust stability of such polynomials was considered in Markov parameters space, where efficient robust stability tests were obtained based on the Markov theorem. This motivated the authors to extend the above idea, and develop Markov parameters and the associated stability criterion for more general types of polynomials such as complex continuous-time as well as real and complex discrete-time polynomials. In this paper a procedure is given for evaluating the maximum allowable perturbations in the Markov parameters of a complex coefficient univariate as well as real coefficient bivariate polynomials so that the strict Hurwitz property remains invariant.     

Signal and image segmentation using pairwise Markov chains

The aim of this paper is to apply the recent pairwise Markov chain model, which generalizes the hidden Markov chain one, to the unsupervised restoration of hidden data. The main novelty is an original parameter estimation method that is valid in a general setting, where the form of the possibly correlated noise is not known. Several experimental results are presented in both Gaussian and generalized mixture contexts. They show the advantages of the pairwise Markov chain model with respect to the classical hidden Markov chain one for supervised and unsupervised restorations.     

一种基于混合MCMC的时域波达方向估计算法

基于马尔科夫链蒙特卡洛（Markov Chain Monte Carlo,MCMC）方法的时域波达方向估计算法通过构造马尔科夫链的方式来对波达方向进行估计,但是现有的算法在马尔科夫链的收敛速度和结果上并没有表现出很好的鲁棒性。为了优化算法的性能,采用多(短)链并行的方式代替原来的长链生成方式,提高了算法收敛的稳定性;并对特定模型下的构造过程进行分析,优化了状态空间,提高了算法的搜索效率;同时结合多混合的MCMC方法,进一步提高了算法估计的精确度和收敛速度。仿真结果表明,改进后的算法对波达方向估计的准确性和实时性都有很大提升。     

基于独立混合模型的EM算法参数初始化实现方法

隐马尔科夫树( Hidden Markov Tree, HMT )的状态不能被观测到,只能观测到另一个与状态有联系的量,通过观测量估计HMT模型参数是一个不完全数据参数估计问题。期望最大化( Expectation Maximization, EM )算法是一种求参数极大似然估计的迭代算法,可以用于解决不完全数据参数估计问题,因此被广泛应用于HMT模型的参数估计中。当初始参数偏离真实参数较大时,EM算法迭代次数多,收敛速度慢,通过一个计算量不大的参数初始化处理,能够有效减少EM算法的迭代次数,加快收敛速度。本文提出了一种基于独立混合模型的参数初始化方法,详细介绍了该方法的实现过程,通过采用独立混合模型进行参数初始化,使得EM算法的迭代次数明显减少,收敛速度大大提高。最后,计算机仿真验证了该方法的可行性和有效性。      

Cognitive Medium Access: Constraining Interference Based on Experimental Models

In this paper we design a cognitive radio that can coexist with multiple parallel WLAN channels while abiding by an interference constraint. The interaction between both systems is characterized by measurement and coexistence is enhanced by predicting the WLAN's behavior based on a continuous-time Markov chain model. Cognitive medium access (CMA) is derived from this model by recasting the problem as one of constrained Markov decision processes. Solutions are obtained by linear programming. Furthermore, we show that optimal CMA admits structured solutions, simplifying practical implementations. Preliminary results for the partially observable case are presented. The performance of the proposed schemes is evaluated for a typical WLAN coexistence setup and shows a significant performance improvement.     

The EM/MPM algorithm for segmentation of textured images: analysisand further experimental results

In this paper we present new results relative to the "expectation-maximization/maximization of the posterior marginals" (EM/MPM) algorithm for simultaneous parameter estimation and segmentation of textured images. The EM/MPM algorithm uses a Markov random field model for the pixel class labels and alternately approximates the MPM estimate of the pixel class labels and estimates parameters of the observed image model. The goal of the EM/MPM algorithm is to minimize the expected value of the number of misclassified pixels. We present new theoretical results in this paper which show that the algorithm can be expected to achieve this goal, to the extent that the EM estimates of the model parameters are close to the true values of the model parameters. We also present new experimental results demonstrating the performance of the EM/MPM algorithm.     

Discrete Estimation of Continuous Angle-Modulated Signals Over Multipath Channels for Aeronautical Communication

The multipath channel for communication between an aerospace vehicle and a ground terminal is modeled by a multiplicative first-order Markov process. The multiplicative process is treated as a component of the message model and the discrete-time demodulation algorithms using the extended Kalman nonlinear estimation technique are developed for continuous-time angle-modulated signals. The equivalent baseband form of the demodulator structure is derived. Two examples of the message process are discussed for an FM system. The simulation results are presented for various values of the bandwidth expansion ratio and the additive SNR. The performance of the baseband algorithms is discussed.     

Dynamic Quantizer Design for Hidden Markov State Estimation Via Multiple Sensors With Fusion Center Feedback

This paper considers the state estimation of hidden Markov models by sensor networks. The objective is to minimize the long term average of the mean square estimation error for the underlying finite state Markov chain. By employing feedback from the fusion center, a dynamic quantization scheme for the sensor nodes is proposed and analyzed by a stochastic control approach. Dynamic rate allocation is also considered when the sensor nodes generate mode dependent measurements.