Joint Angular- and Delay-Domain MIMO Propagation Parameter Estimation Using Approximate ML Method

In this paper, we derive an estimation method that jointly estimates the parameters of the concentrated propagation paths and the distributed scattering component that are frequently observed in multiple-input multiple-output (MIMO) channel sounding measurements. The joint angular-delay domain model leads to a correlation matrix with high dimensionality, which makes direct implementation of a maximum-likelihood (ML) estimator unfeasible. We derive low-complexity methods for computing approximate ML estimates that exploit the structure of the covariance matrices. We propose an iterative two-step procedure that alternates between the estimation of the parameters of the concentrated propagation paths and the parameters of the distributed scattering. For the distributed scattering, the estimator first optimizes the parameters describing their time-delay structure. Then, using the estimated time-delay parameters, the parameters of the angular distributions are optimized. We present simulation results and compare the estimated time-delay and angular distributions to the actual distributions, demonstrating that high-quality estimates are obtained. The large sample performance of the estimator is studied by establishing the Cramer-Rao lower bound (CRLB) and comparing it to the variances of the estimates. The simulations show that the variance of the proposed estimation technique reaches the CRLB for relatively small sample size for most parameters, and no bias is observed.     

Estimating Kinetic Parameter Maps From Dynamic Contrast-Enhanced MRI Using Spatial Prior Knowledge

Dynamic contrast-enhanced magnetic resonance (DCE-MR) imaging can be used to study microvascular structure in vivo by monitoring the abundance of an injected diffusible contrast agent over time. The resulting spatially resolved intensity-time curves are usually interpreted in terms of kinetic parameters obtained by fitting a pharmacokinetic model to the observed data. Least squares estimates of the highly nonlinear model parameters, however, can exhibit high variance and can be severely biased. As a remedy, we bring to bear spatial prior knowledge by means of a generalized Gaussian Markov random field (GGMRF). By using information from neighboring voxels and computing the maximum a posteriori solution for entire parameter maps at once, both bias and variance of the parameter estimates can be reduced thus leading to smaller root mean square error (RMSE). Since the number of variables gets very big for common image resolutions, sparse solvers have to be employed. To this end, we propose a generalized iterated conditional modes (ICM) algorithm operating on blocks instead of sites which is shown to converge considerably faster than the conventional ICM algorithm. Results on simulated DCE-MR images show a clear reduction of RMSE and variance as well as, in some cases, reduced estimation bias. The mean residual bias (MRB) is reduced on the simulated data as well as for all 37 patients of a prostate DCE-MRI dataset. Using the proposed algorithm, average computation times only increase by a factor of 1.18 (871 ms per voxel) for a Gaussian prior and 1.51 (1.12 s per voxel) for an edge-preserving prior compared to the single voxel approach (740 ms per voxel).     

A multiscale random field model for Bayesian image segmentation

Many approaches to Bayesian image segmentation have used maximum a posteriori (MAP) estimation in conjunction with Markov random fields (MRF). Although this approach performs well, it has a number of disadvantages. In particular, exact MAP estimates cannot be computed, approximate MAP estimates are computationally expensive to compute, and unsupervised parameter estimation of the MRF is difficult. The authors propose a new approach to Bayesian image segmentation that directly addresses these problems. The new method replaces the MRF model with a novel multiscale random field (MSRF) and replaces the MAP estimator with a sequential MAP (SMAP) estimator derived from a novel estimation criteria. Together, the proposed estimator and model result in a segmentation algorithm that is not iterative and can be computed in time proportional to MN where M is the number of classes and N is the number of pixels. The also develop a computationally efficient method for unsupervised estimation of model parameters. Simulations on synthetic images indicate that the new algorithm performs better and requires much less computation than MAP estimation using simulated annealing. The algorithm is also found to improve classification accuracy when applied to the segmentation of multispectral remotely sensed images with ground truth data.     

Posterior-mean super-resolution with a causal Gaussian Markov random field prior

We propose a Bayesian image super-resolution (SR) method with a causal Gaussian Markov random field (MRF) prior. SR is a technique to estimate a spatially high-resolution image from given multiple low-resolution images. An MRF model with the line process supplies a preferable prior for natural images with edges. We improve the existing image transformation model, the compound MRF model, and its hyperparameter prior model. We also derive the optimal estimator--not the joint maximum a posteriori (MAP) or the marginalized maximum likelihood (ML) but the posterior mean (PM)--from the objective function of the L2-norm-based (mean square error) peak signal-to-noise ratio. Point estimates such as MAP and ML are generally not stable in ill-posed high-dimensional problems because of overfitting, whereas PM is a stable estimator because all the parameters in the model are evaluated as distributions. The estimator is numerically determined by using the variational Bayesian method. The variational Bayesian method is a widely used method that approximately determines a complicated posterior distribution, but it is generally hard to use because it needs the conjugate prior. We solve this problem with simple Taylor approximations. Experimental results have shown that the proposed method is more accurate or comparable to existing methods.     

Estimation of Markov random field prior parameters using Markovchain Monte Carlo maximum likelihood

Developments in statistics now allow maximum likelihood estimators for the parameters of Markov random fields (MRFs) to be constructed. We detail the theory required, and present an algorithm that is easily implemented and practical in terms of computation time. We demonstrate this algorithm on three MRF models-the standard Potts model, an inhomogeneous variation of the Potts model, and a long-range interaction model, better adapted to modeling real-world images. We estimate the parameters from a synthetic and a real image, and then resynthesize the models to demonstrate which features of the image have been captured by the model. Segmentations are computed based on the estimated parameters and conclusions drawn.     

Parameter selection for total-variation-based image restoration using discrepancy principle

There are two key issues in successfully solving the image restoration problem: 1) estimation of the regularization parameter that balances data fidelity with the regularity of the solution and 2) development of efficient numerical techniques for computing the solution. In this paper, we derive a fast algorithm that simultaneously estimates the regularization parameter and restores the image. The new approach is based on the total-variation (TV) regularized strategy and Morozov's discrepancy principle. The TV norm is represented by the dual formulation that changes the minimization problem into a minimax problem. A proximal point method is developed to compute the saddle point of the minimax problem. By adjusting the regularization parameter adaptively in each iteration, the solution is guaranteed to satisfy the discrepancy principle. We will give the convergence proof of our algorithm and numerically show that it is better than some state-of-the-art methods in terms of both speed and accuracy.     

Wavelet-Based SAR Image Despeckling and Information Extraction, Using Particle Filter

This paper proposes a new-wavelet-based synthetic aperture radar (SAR) image despeckling algorithm using the sequential Monte Carlo method. A model-based Bayesian approach is proposed. This paper presents two methods for SAR image despeckling. The first method, called WGGPF, models a prior with Generalized Gaussian (GG) probability density function (pdf) and the second method, called WGMPF, models prior with a Generalized Gaussian Markov random field (GGMRF). The likelihood pdf is modeled using a Gaussian pdf. The GGMRF model is used because it enables texture parameter estimation. The prior is modeled using GG pdf, when texture parameters are not needed. A particle filter is used for drawing particles from the prior for different shape parameters of GG pdf. When the GGMRF prior is used, the particles are drawn from prior in order to estimate noise-free wavelet coefficients and for those coefficients the texture parameter is changed in order to obtain the best textural parameters. The texture parameters are changed for a predefined set of shape parameters of GGMRF. The particles with the highest weights represents the final noise-free estimate with corresponding textural parameters. The despeckling algorithms are compared with the state-of-the-art methods using synthetic and real SAR data. The experimental results show that the proposed despeckling algorithms efficiently remove noise and proposed methods are comparable with the state-of-the-art methods regarding objective measurements. The proposed WGMPF preserves textures of the real, high-resolution SAR images well.     

Noncausal 2-D spectrum estimation for direction finding

A two-dimensional noncausal autoregressive (NCAR) plus additive noise model-based spectrum estimation method is presented for planar array data typical of signals encountered in array processing applications. Since the likelihood function for NCAR plus noise data is nonlinear in the model parameters and is further complicated by the unknown variance of the additive noise, computationally intensive gradient search algorithms are required for computing the estimates. If a doubly periodic lattice is assumed, the complexity of the approximate maximum likelihood (ML) equation is significantly reduced without destroying the theoretical asymptotic properties of the estimates and degrading the observed accuracy of the estimated spectra. Initial conditions for starting the approximate ML computation are suggested. Experimental results that can be used to evaluate the signal-plus-noise approach and compare its performance to those of signal-only methods are presented for Gaussian and simulated planar array data. Statistics of estimated spectrum parameters are given, and estimated spectra for signals with close spatial frequencies are shown. The approximate ML parameter estimate's asymptotic properties, such as consistency and normality, are established, and lower bounds for the estimate's errors are derived, assuming that the data are Gaussian     

Maximum likelihood array processing for stochastic coherent sources

Maximum likelihood (ML) estimation in array signal processing for the stochastic noncoherent signal case is well documented in the literature. We focus on the equally relevant case of stochastic coherent signals. Explicit large-sample realizations are derived for the ML estimates of the noise power and the (singular) signal covariance matrix. The asymptotic properties of the estimates are examined, and some numerical examples are provided. In addition, we show the surprising fact that the ML estimates of the signal parameters obtained by ignoring the information that the sources are coherent coincide in large samples with the ML estimates obtained by exploiting the coherent source information. Thus, the ML signal parameter estimator derived for the noncoherent case (or its large-sample realizations) asymptotically achieves the lowest possible estimation error variance (corresponding to the coherent Cramer-Rao bound)     

On the robustness of parameter estimation of superimposed signalsby dynamic programming

We analyze a recently proposed dynamic programming algorithm (REDP) for maximum likelihood (ML) parameter estimation of superimposed signals in noise. We show that it degrades gracefully with deviations from the key assumption of a limited interaction signal model (LISMO), providing exact estimates when the LISMO assumption holds exactly. In particular, we show that the deviations of the REDP estimates from the exact ML are continuous in the deviation of the signal model from the LISMO assumption. These deviations of the REDP estimates from the MLE are further quantified by a comparison to an ML algorithm with an exhaustive multidimensional search on a lattice in parameter space. We derive an explicit expression for the lattice spacing for which the two algorithms have equivalent optimization performance, which can be used to assess the robustness of REDP to deviations from the LISMO assumption. The values of this equivalent lattice spacing are found to be small for a classical example of superimposed complex exponentials in noise, confirming the robustness of REDP for this application     

Parametric GLRT for Multichannel Adaptive Signal Detection

This paper considers the problem of detecting a multichannel signal in the presence of spatially and temporally colored disturbance. A parametric generalized likelihood ratio test (GLRT) is developed by modeling the disturbance as a multichannel autoregressive (AR) process. Maximum likelihood (ML) parameter estimation underlying the parametric GLRT is examined. It is shown that the ML estimator for the alternative hypothesis is nonlinear and there exists no closed-form expression. To address this issue, an asymptotic ML (AML) estimator is presented, which yields asymptotically optimum parameter estimates at reduced complexity. The performance of the parametric GLRT is studied by considering challenging cases with limited or no training signals for parameter estimation. Such cases (especially when training is unavailable) are of great interest in detecting signals in heterogeneous, fast changing, or dense-target environments, but generally cannot be handled by most existing multichannel detectors which rely more heavily on training at an adequate level. Compared with the recently introduced parametric adaptive matched filter (PAMF) and parametric Rao detectors, the parametric GLRT achieves higher data efficiency, offering improved detection performance in general.     

Unsupervised vector image segmentation by a tree structure-ICM algorithm

In recent years, many image segmentation approaches have been based on Markov random fields (MRFs). The main assumption of the MRF approaches is that the class parameters are known or can be obtained from training data. In this paper the authors propose a novel method that relaxes this assumption and allows for simultaneous parameter estimation and vector image segmentation. The method is based on a tree structure (TS) algorithm which is combined with Besag's iterated conditional modes (ICM) procedure. The TS algorithm provides a mechanism for choosing initial cluster centers needed for initialization of the ICM. The authors' method has been tested on various one-dimensional (1-D) and multidimensional medical images and shows excellent performance. In this paper the authors also address the problem of cluster validation. They propose a new maximum a posteriori (MAP) criterion for determination of the number of classes and compare its performance to other approaches by computer simulations.     

Maximum likelihood estimation of Burr XII distribution parameters under Type II censoring

The mathematical theory for the point estimation of the parameters of the Burr Type XII distribution by maximum likelihood (ML) is developed for Type II censored samples. Also derived are necessary and sufficient conditions on the sample data that guarantee the existence, uniqueness and finiteness of the ML parameter estimates for all possible permissible parameter combinations. The asymptotic theory of ML is invoked to obtain approximate confidence intervals for the ML parameter estimates. An application to reliability data arising in a life test experiment is discussed.     

Iterative Source-Channel Decoding With Markov Random Field Source Models

We propose a joint source-channel decoding approach for multidimensional correlated source signals. A Markov random field (MRF) source model is used which exemplarily considers the residual spatial correlations in an image signal after source encoding. Furthermore, the MRF parameters are selected via an analysis based on extrinsic information transfer charts. Due to the link between MRFs and the Gibbs distribution, the resulting soft-input soft-output (SISO) source decoder can be implemented with very low complexity. We prove that the inclusion of a high-rate block code after the quantization stage allows the MRF-based decoder to yield the maximum average extrinsic information. When channel codes are used for additional error protection the MRF-based SISO source decoder can be used as the outer constituent decoder in an iterative source-channel decoding scheme. Considering an example of a simple image transmission system we show that iterative decoding can be successfully employed for recovering the image data, especially when the channel is heavily corrupted.     

Supervised segmentation of remote sensing images based on a tree-structured MRF model

Most remote sensing images exhibit a clear hierarchical structure which can be taken into account by defining a suitable model for the unknown segmentation map. To this end, one can resort to the tree-structured Markov random field (MRF) model, which describes a K-ary field by means of a sequence of binary MRFs, each one corresponding to a node in the tree. Here we propose to use the tree-structured MRF model for supervised segmentation. The prior knowledge on the number of classes and their statistical features allows us to generalize the model so that the binary MRFs associated with the nodes can be adapted freely, together with their local parameters, to better fit the data. In addition, it allows us to define a suitable likelihood term to be coupled with the TS-MRF prior so as to obtain a precise global model of the image. Given the complete model, a recursive supervised segmentation algorithm is easily defined. Experiments on a test SPOT image prove the superior performance of the proposed algorithm with respect to other comparable MRF-based or variational algorithms.     

Reliability analysis of accelerated life-test data from arepairable system

This paper analyzes accelerating testing of a repairable item modeled by a nonhomogeneous Poisson process with covariates. We extensively analyze a single accelerating variable with two stress levels, and derive closed-form maximum likelihood (ML) solutions. These closed-form solutions provide: (1) an easier way to obtain point estimates of the unknown parameters under usual operating conditions, and (2) a way to obtain confidence intervals on the process parameters and function thereof which are more accurate than those based on asymptotic normality of ML estimates. We analyze the circumstances in which a drawback to closed-form estimation arises, and guides the extent that our procedures may equally apply. An example application drawn from a real situation of accelerated testing is presented, and numerical estimates based on our procedures are derived and discussed. Theoretical and simulation results show that estimation procedures based on the power-law process and regression methods can be a flexible, useful tool for reliability analysis of a repairable item     

Maximum-likelihood versus maximum a posteriori parameter estimation of physiological system models: the C-peptide impulse response case study

Maximum-likelihood (ML), also given its connection to least-squares (LS), is widely adopted in parameter estimation of physiological system models, i.e., assigning numerical values to the unknown model parameters from the experimental data. A more sophisticated but less used approach is maximum a posteriori (MAP) estimation. Conceptually, while ML adopts a Fisherian approach, i.e., only experimental measurements are supplied to the estimator, MAP estimation is a Bayesian approach, i.e., a priori available statistical information on the unknown parameters is also exploited for their estimation. In this paper, after a brief review of the theory behind ML and MAP estimators, we compare their performance in the solution of a case study concerning the determination of the parameters of a sum of exponential model which describes the impulse response of C-peptide (CP), a key substance for reconstructing insulin secretion. The results show that MAP estimation always leads to parameter estimates with a precision (sometimes significantly) higher than that obtained through ML, at the cost of only a slightly worse fit. Thus, a three exponential model can be adopted to describe the CP impulse response model in place of the two exponential model usually identified in the literature by the ML/LS approach. Simulated case studies are also reported to evidence the importance of taking into account a priori information in a data poor situation, e.g., when a few or too noisy measurements are available. In conclusion, our results show that, when a priori information on the unknown model parameters is available, Bayes estimation can be of relevant interest, since it can significantly improve the precision of parameter estimates with respect to Fisher estimation. This may also allow the adoption of more complex models than those determinable by a Fisherian approach.     

Accurate direction‐of‐arrival estimation of multiple sources using a genetic approach

In this paper, we present an accurate direction‐of‐arrival (DOA) estimation method, which is based on the maximum likelihood (ML) principle and implemented using a modified and refined genetic algorithm (GA). With the newly introduced features—intelligent initialization and the emperor‐selective (EMS) mating scheme, carefully selected crossover and mutation operators and fine‐tuned parameters such as the population size, the probability of crossover and mutation etc., the GA‐ML estimator achieves fast global convergence. A GA operator and parameter standard is suggested for this application, which is independent of the source and array configurations except the number of sources. Simulation results demonstrate that in general scenarios, the proposed estimator is the most efficient in computation and its statistical performance is the best among all popular ML‐based DOA estimation methods. Copyright © 2004 John Wiley & Sons, Ltd.     

A model-based approach for estimation of two-dimensional maximum entropy power spectra

A stochastic model-based approach is presented for estimation of the two-dimensional maximum entropy power spectrum (MEPS) from given finite uniform array data. The method consists of fitting an appropriate two-dimensional noncausal Gaussian-Markov random field (GMRF) model to the given data using the maximum likelihood (ML) technique for parameter estimation. The nonlinear criterion function used for ML estimation is similar in structure to the function arising in the deterministic approach of Lang and McClellan. The model-based approach provides new insights into the two-dimensional MEPS estimation problem. For example, using the asymptotic normality of ML estimates, we derive simultaneous confidence bands for the estimated MEPS. It turns out that when the true correlations are generated by a noncausal GMRF model, the two-dimensional MEPS can be obtained by solving linear equations. This approach also suggests techniques for realizing two-dimensional GMRF models from the given correlation data. Several numerical examples are given to illustrate the usefulness of the approach.     

The mean field theory in EM procedures for blind Markov randomfield image restoration

A Markov random field (MRF) model-based EM (expectation-maximization) procedure for simultaneously estimating the degradation model and restoring the image is described. The MRF is a coupled one which provides continuity (inside regions of smooth gray tones) and discontinuity (at region boundaries) constraints for the restoration problem which is, in general, ill posed. The computational difficulty associated with the EM procedure for MRFs is resolved by using the mean field theory from statistical mechanics. An orthonormal blur decomposition is used to reduce the chances of undesirable locally optimal estimates. Experimental results on synthetic and real-world images show that this approach provides good blur estimates and restored images. The restored images are comparable to those obtained by a Wiener filter in mean-square error, but are most visually pleasing.