A Bayesian estimation approach for speech enhancement using hiddenMarkov models

A Bayesian estimation approach for enhancing speech signals which have been degraded by statistically independent additive noise is motivated and developed. In particular, minimum mean square error (MMSE) and maximum a posteriori (MAP) signal estimators are developed using hidden Markov models (HMMs) for the clean signal and the noise process. It is shown that the MMSE estimator comprises a weighted sum of conditional mean estimators for the composite states of the noisy signal, where the weights equal the posterior probabilities of the composite states given the noisy signal. The estimation of several spectral functionals of the clean signal such as the sample spectrum and the complex exponential of the phase is also considered. A gain-adapted MAP estimator is developed using the expectation-maximization algorithm. The theoretical performance of the MMSE estimator is discussed, and convergence of the MAP estimator is proved. Both the MMSE and MAP estimators are tested in enhancing speech signals degraded by white Gaussian noise at input signal-to-noise ratios of from 5 to 20 dB     

基于小波域双层贝叶斯模型的图像复原

为了克服正交小波变换用于图像复原引起的振铃现象,提出了一种基于小波域双层贝叶斯模型的图像复原算法.采用移不变小波变换,经过简单的转换,使计算复杂度较正交小波变换法并没有显著增加.对于涉及小波系数和超参数的估计问题,通过双层贝叶斯模型方法解决.首先使用局部高斯分布作为第一层模型,主要用于刻画原始图像小波系数的先验分布;第二层模型用于对超参数的估计,假设局部逆方差为服从Gamma分布的随机变量.基于双层贝叶斯模型,采用最大后验概率估计(MAP)同时进行参数估计与图像复原,计算机实验验证了该方法的有效性.     

Multiscale image segmentation using wavelet-domain hidden Markovmodels

We introduce a new image texture segmentation algorithm, HMTseg, based on wavelets and the hidden Markov tree (HMT) model. The HMT is a tree-structured probabilistic graph that captures the statistical properties of the coefficients of the wavelet transform. Since the HMT is particularly well suited to images containing singularities (edges and ridges), it provides a good classifier for distinguishing between textures. Utilizing the inherent tree structure of the wavelet HMT and its fast training and likelihood computation algorithms, we perform texture classification at a range of different scales. We then fuse these multiscale classifications using a Bayesian probabilistic graph to obtain reliable final segmentations. Since HMTseg works on the wavelet transform of the image, it can directly segment wavelet-compressed images without the need for decompression into the space domain. We demonstrate the performance of HMTseg with synthetic, aerial photo, and document image segmentations.     

Speaker-independent isolated word recognition using multiple hiddenMarkov models

A multi-HMM speaker-independent isolated word recognition system is described. In this system, three vector quantisation methods, the LBG algorithm, the EM algorithm, and a new MGC algorithm, are used for the classification of the speech space. These quantisations of the speech space are then used to produce three HMMs for each word in the vocabulary. In the recognition step, the Viterbi algorithm is used in the three subrecognisers. The log probabilities of the observation sequences matching-the models are multiplied by the weights determined by the recognition accuracies of individual subrecognisers and summed to give the log probability that the utterance is of a particular word in the vocabulary. This multi-HMM system results in a reduction of about 50% in the error rate in comparison with the single model system     

A tree-structured Markov random field model for Bayesian image segmentation

We present a new image segmentation algorithm based on a tree-structured binary MRF model. The image is recursively segmented in smaller and smaller regions until a stopping condition, local to each region, is met. Each elementary binary segmentation is obtained as the solution of a MAP estimation problem, with the region prior modeled as an MRF. Since only binary fields are used, and thanks to the tree structure, the algorithm is quite fast, and allows one to address the cluster validation problem in a seamless way. In addition, all field parameters are estimated locally, allowing for some spatial adaptivity. To improve segmentation accuracy, a split-and-merge procedure is also developed and a spatially adaptive MRF model is used. Numerical experiments on multispectral images show that the proposed algorithm is much faster than a similar reference algorithm based on "flat" MRF models, and its performance, in terms of segmentation accuracy and map smoothness, is comparable or even superior.     

Varying complexity in tree-structured image distribution models.

Probabilistic models of image statistics underlie many approaches in image analysis and processing. An important class of such models have variables whose dependency graph is a tree. If the hidden variables take values on a finite set, most computations with the model can be performed exactly, including the likelihood calculation, training with the EM algorithm, etc. Crouse et al. developed one such model, the hidden Markov tree (HMT). They took particular care to limit the complexity of their model. We argue that it is beneficial to allow more complex tree-structured models, describe the use of information theoretic penalties to choose the model complexity, and present experimental results to support these proposals. For these experiments, we use what we call the hierarchical image probability (HIP) model. The differences between the HIP and the HMT models include the use of multivariate Gaussians to model the distributions of local vectors of wavelet coefficients and the use of different numbers of hidden states at each resolution. We demonstrate the broad utility of image distributions by applying the HIP model to classification, synthesis, and compression, across a variety of image types, namely, electrooptical, synthetic aperture radar, and mammograms (digitized X-rays). In all cases, we compare with the HMT.     

Landmine detection with ground penetrating radar using hiddenMarkov models

Novel, general methods for detecting landmine signatures in ground penetrating radar (GPR) using hidden Markov models (HMMs) are proposed and evaluated. The methods are evaluated on real data collected by a GPR mounted on a moving vehicle at three different geographical locations. A large library of digital GPR signatures of both landmines and clutter/background was constructed and used for training. Simple, but effective, observation vector representations are constructed to naturally model the time-varying signatures produced by the interaction of the GPR and the landmines as the vehicle moves. The number and definition of the states of the HMMs are based on qualitative signature models. The model parameters are optimized using the Baum-Welch algorithm. The models were trained on landmine and background/clutter signatures from one geographical location and successfully tested at two different locations. The data used in the test were acquired from over 6000 m² of simulated dirt and gravel roads, and also off-road conditions. These data contained approximately 300 landmine signatures, over half of which were plastic-cased or completely nonmetal     

Stereophonic speech recognition in noise using compensated hiddenMarkov models

A novel procedure is presented for noise compensation in hidden Markov model speech recognisers. The procedure uses two microphone signals and, unlike previous approaches, does not require the noise spectrum to be stationary even in the short term. Results are presented showing that the performance of the compensated system equals or exceeds that obtained using matched training     

SAR image data compression using a tree-structured wavelettransform

SAR image compression is very important in reducing the costs of data storage and transmission in relatively slow channels. The authors propose a compression scheme driven by texture analysis, homogeneity mapping and speckle noise reduction within the wavelet framework. The image compressibility and interpretability are improved by incorporating speckle reduction into the compression scheme. The authors begin with the classical set partitioning in hierarchical trees (SPIHT) wavelet compression scheme, and modify it to control the amount of speckle reduction, applying different encoding schemes to homogeneous and nonhomogeneous areas of the scene. The results compare favorably with the conventional SPIHT wavelet and the JPEG compression methods     

Bayesian image reconstruction in SPECT using higher order mechanical models as priors

While the ML-EM algorithm for reconstruction for emission tomography is unstable due to the ill-posed nature of the problem. Bayesian reconstruction methods overcome this instability by introducing prior information, often in the form of a spatial smoothness regularizer. More elaborate forms of smoothness constraints may be used to extend the role of the prior beyond that of a stabilizer in order to capture actual spatial information about the object. Previously proposed forms of such prior distributions were based on the assumption of a piecewise constant source distribution. Here, the authors propose an extension to a piecewise linear model-the weak plate-which is more expressive than the piecewise constant model. The weak plate prior not only preserves edges but also allows for piecewise ramplike regions in the reconstruction. Indeed, for the authors' application in SPECT, such ramplike regions are observed in ground-truth source distributions in the form of primate autoradiographs of rCBF radionuclides. To incorporate the weak plate prior in a MAP approach, the authors model the prior as a Gibbs distribution and use a GEM formulation for the optimization. They compare quantitative performance of the ML-EM algorithm, a GEM algorithm with a prior favoring piecewise constant regions, and a GEM algorithm with their weak plate prior. Pointwise and regional bias and variance of ensemble image reconstructions are used as indications of image quality. The authors' results show that the weak plate and membrane priors exhibit improved bias and variance relative to ML-EM techniques.     

Improved hidden Markov models in the wavelet-domain

Wavelet-domain hidden Markov models (HMMs), in particular the hidden Markov tree (HMT) model, have been introduced and applied to signal and image processing, e.g., signal denoising. We develop a simple initialization scheme for the efficient HMT model training and then propose a new four-state HMT model called HMT-2. We find that the new initialization scheme fits the HMT-2 model well. Experimental results show that the performance of signal denoising using the HMT-2 model is often improved over the two-state HMT model developed by Crouse et al. (see ibid., vol.46, p.886-902, 1998)     

Error-diffused image compression using a binary-to-gray-scaledecoder and predictive pruned tree-structured vector quantization

The authors consider data compression of binary error diffused images. The original contribution is using nonlinear filters to decode error-diffused images to compress them in the gray-scale domain; this gives better image quality than directly compressing the binary images. Their method is of low computational complexity and can work with any halftoning algorithm.     

A new approximate likelihood estimator for ARMA-filtered hiddenMarkov models

Hidden Markov models (HMMs) are successfully applied in various fields of time series analysis. Colored noise, e.g., due to filtering, violates basic assumptions of the model. Although it is well known how to consider autoregressive (AR) filtering, there is no algorithm to take into account moving-average (MA) filtering in parameter estimation exactly. We present an approximate likelihood estimator for MA-filtered HMM that is generalized to deal with an autoregressive moving-average (ARMA) filtered HMM. The approximation order of the likelihood calculation can be chosen. Therefore, we obtain a sequence of estimators for the HMM parameters as well as for the filter coefficients. The recursion equations for an efficient algorithm are derived from exact expressions for the forward iterations. By simulations, we show that the derived estimators are unbiased in filter situations where standard HMM's are not able to recover the true dynamics. Special implementation strategies together with small approximations yield further acceleration of the algorithm     

Synthetic aperture radar image compression using tree-structured edge-directed orthogonal wavelet packet transform

Currently, wavelet-based coding algorithms are popular for synthetic aperture radar (SAR) image compression, which is very important for reducing the cost of data storage and transmission in relatively slow channels. However, standard wavelet transform is limited by spatial isotropy of its basis functions that is not completely adapted to represent image entities like edges or textures, which means wavelet-based coding algorithms are suboptimal to image compression. In this paper, a novel tree-structured edge-directed orthogonal wavelet packet transform is proposed for SAR image compression. Inspired by the intrinsic geometric structure of images, the new transform improves the performance of standard wavelet by filtering along the regular direction first and then along the orthogonal direction with directional lifting structure. The cost function of best basis selection is designed by textural and directional information for tree-structured edge-directed orthogonal wavelet packet transform. The new transform including speckle reduction can be used to construct SAR image coder with the embedded block coding with optimal truncation for transform coefficients, and arithmetic coding for additional information. The experimental results show that the proposed approach outperforms JPEG2000 and Fast wavelet packet (FWP), both visually and item of PSNR values.     

Target identification with wave-based matched pursuits and hiddenMarkov models

The method of matched pursuits is an algorithm by which a waveform is parsed into its fundamental constituents here, in the context of short-pulse electromagnetic scattering, wavefronts, and resonances (constituting what we have called wave-based matched pursuits). The wave-based matched-pursuits algorithm is used to develop a codebook of features that are representative of time-domain scattering from a target of interest, accounting for the variability of such as a function of target-sensor orientation. This codebook is subsequently used in the context of a hidden Markov model (HMM) in which the probability of measuring a particular codebook element is quantified as a function of target-sensor orientation. We review the wave-based matched-pursuits algorithm and its use in the context of an HMM (for target identification). Finally, this new wave-based signal processing algorithm is demonstrated with simulated scattering data, with additive noise     

Side-match tree-structured vector quantiser for image progressive coding

The balanced tree-structured vector quantiser is the traditional method of achieving image progressive coding. During image progressive coding, an image is decoded step-by-step in a decoder. The author proposes an unbalanced tree-structured vector quantiser to perform image progressive coding for a given series of rate thresholds. Side-match vector quantisation and its variants have been proposed to reduce the bit rate in image coding. The tree-structured vector quantiser and the side-match vector quantiser are combined to perform image progressive coding, achieving a better coding quality than that obtained using only the tree-structured vector quantiser at the same bit rate.     

Unsupervised Bayesian image segmentation using orthogonal series

This paper deals with the problem of unsupervised image segmentation which consists in first mixture identification phase and second a Bayesian decision phase. During the mixture identification phase, the conditional probability density function (pdf) and the a priori class probabilities must be estimated. The most difficult part is the estimation of the number of pixel classes or in other words the estimation of the number of density mixture components. To resolve this problem, we propose here a Stochastic and Nonparametric Expectation-Maximization (SNEM) algorithm. The algorithm finds the most likely number of classes, their associated model parameters and generates a segmentation of the image by classifying the pixels into these classes. The non-parametric aspect comes from the use of the orthogonal series estimator. Experimental results are promising, we have obtained accurate results on a variety of real images.     

Vector quantization using tree-structured self-organizing featuremaps

In this paper, we propose a binary-tree structure neural network model suitable for structured clustering. During and after training, the centroids of the clusters in this model always form a binary tree in the input pattern space. This model is used to design tree search vector quantization codebooks for image coding. Simulation results show that the acquired codebook not only produces better-quality images but also achieves a higher compression ratio than conventional tree search vector quantization. When source coding is applied after VQ, the new model performs better than the generalized Lloyd algorithm in terms of distortion, bits per pixel, and encoding complexity for low-detail and medium-detail images     

Performance comparison between semicontinuous and discrete hiddenMarkov models of speech

The semicontinuous hidden Markov model (SCHMM) of speech has been shown to offer improved performance in comparison to conventional discrete HMM. Experimental results with different vector quantisation levels, different probability density functions, and different numbers of tokens in the training set for the SCHMM are reported here to compare with conventional discrete HMM     

An MCMC sampling approach to estimation of nonstationary hiddenMarkov models

Hidden Markov models (HMMs) represent a very important tool for analysis of signals and systems. In the past two decades, HMMs have attracted the attention of various research communities, including the ones in statistics, engineering, and mathematics. Their extensive use in signal processing and, in particular, speech processing is well documented. A major weakness of conventional HMMs is their inflexibility in modeling state durations. This weakness can be avoided by adopting a more complicated class of HMMs known as nonstationary HMMs. We analyze nonstationary HMMs whose state transition probabilities are functions of time that indirectly model state durations by a given probability mass function and whose observation spaces are discrete. The objective of our work is to estimate all the unknowns of a nonstationary HMM, which include its parameters and the state sequence. To that end, we construct a Markov chain Monte Carlo (MCMC) sampling scheme, where sampling from all the posterior probability distributions is very easy. The proposed MCMC sampling scheme has been tested in extensive computer simulations on finite discrete-valued observed data, and some of the simulation results are presented