Maximum-likelihood reconstruction of transmission images in emission computed tomography via the EM algorithm

The expectation-maximization (EM) algorithm for computing maximum-likelihood estimates of transmission images in positron-emission tomography (PET) (see K. Lange and R. Carson, J. Comput. Assist. Tomogr., vol.8, no.2, p.306-16, 1984) is extended to include measurement error, accidental coincidences and Compton scatter. A method for accomplishing the maximization step using one step of Newton's method is proposed. The algorithm is regularized with the method of sieves. Evaluations using both Monte Carlo simulations and phantom studies on the Siemens 953B scanner suggest that the algorithm yields unbiased images with significantly lower variances than filtered-backprojection when the images are reconstructed to the intrinsic resolution. Large features in the images converge in under 200 iterations while the smallest features required up to 2,000 iterations. All but the smallest features in typical transmission scans converge in approximately 250 iterations. The initial implementation of the algorithm requires 50 sec per iteration on a DECStation 5000.     

Bar code recovery via the EM algorithm

The ubiquitous supermarket checkout scanner is indeed a well engineered and effective device. There is, nevertheless, demand for better devices. Existing scanners rely on simple and indeed low-cost signal processing to interpret bar code signals. These methods, nevertheless, fundamentally limit label reading and cannot be extended. A new method based on the deterministic EM algorithm is described. First results show a substantial improvement in label reading depth of field, which is an important performance parameter for bar code readers     

Maximum-likelihood estimation of circle parameters via convolution.

The accurate fitting of a circle to noisy measurements of circumferential points is a much studied problem in the literature. In this paper, we present an interpretation of the maximum-likelihood estimator (MLE) and the Delogne-K?sa estimator (DKE) for circle-center and radius estimation in terms of convolution on an image which is ideal in a certain sense. We use our convolution-based MLE approach to find good estimates for the parameters of a circle in digital images. In digital images, it is then possible to treat these estimates as preliminary estimates into various other numerical techniques which further refine them to achieve subpixel accuracy. We also investigate the relationship between the convolution of an ideal image with a "phase-coded kernel" (PCK) and the MLE. This is related to the "phase-coded annulus" which was introduced by Atherton and Kerbyson who proposed it as one of a number of new convolution kernels for estimating circle center and radius. We show that the PCK is an approximate MLE (AMLE). We compare our AMLE method to the MLE and the DKE as well as the Cramér-Rao Lower Bound in ideal images and in both real and synthetic digital images.     

Parameter estimation for Markov modulated Poisson processes via the EM algorithm with time discretization

The Markov modulated Poisson process (MMPP) has been proposed as a suitable model for characterizing the input traffic to a statistical multiplexer [6]. This paper describes a novel method of parameter estimation for MMPPs. The idea is to employ time discretization to convert an MMPP from the continuous-time domain into the discrete-time domain and then to use a powerful statistical inference technique, known as the EM algorithm, to obtain maximum-likelihood estimates of the model parameters. Tests conducted through a series of simulation experiments indicate that the new method yields results that are significantly more accurate compared to the method described in [8]. In addition, the new method is more flexible and general in that it is applicable to MMPPs with any number of states while retaining nearly constant simplicity in its implementation. Detailed experimental results on the sensitivity of the estimation accuracy to (1) the initialization of the model, (2) the size of the observation interarrival interval data available for the estimation, and (3) the inherent separability of the MMPP states are presented.     

Joint carrier frequency synchronization and channel estimation for OFDM systems via the EM algorithm

A joint carrier frequency synchronization and channel estimation scheme is proposed for orthogonal frequency-division multiplexing (OFDM) system. In the proposed scheme, carrier frequency synchronization and channel estimation are performed iteratively via the expectation-maximization (EM) algorithm using an OFDM preamble symbol. Moreover, we analytically investigate the effect of frequency offset error on the mean square error (MSE) performance of channel estimator. Simulation results present that the proposed scheme achieves almost ideal performance for both channel and frequency offset estimation.     

Adaptive MAP receiver via the EM algorithm and message passings for MIMO-OFDM mobile communications

This paper proposes two new types of maximum a posteriori probability (MAP) receivers for multiple-input-multiple-output and orthogonal frequency-division multiplexing mobile communications with a channel coding such as the low-density parity-check code. One proposed receiver employs the expectation-maximization algorithm so as to improve performance of approximated MAP detection. Differently from a conventional receiver employing the minimum mean-square estimation (MMSE) algorithm, it applies the recursive least squares (RLS) algorithm to the channel estimation in order to track a fast fading channel. For the purpose of further improvement, the other proposed receiver applies a new adaptive algorithm that can be derived from the message passing on factor graphs. The algorithm exploits all detected signals but one of targeted time, and can gain a considerable advantage over the MMSE and RLS. Computer simulations show that the first proposed receiver is superior in channel-tracking ability to the conventional receiver employing the MMSE. Furthermore, it is demonstrated that the second proposed receiver remarkably outperforms both the conventional and the first proposed ones.     

The cascade EM algorithm

The estimate-maximize (EM) algorithm is an iterative method for finding maximum-likelihood parameter estimates from incomplete data. The authors develop an extension of the EM algorithm that may be useful in accelerating the algorithm and in simplifying the computations involved. The extension works with an intermediate complete data specification, and performs intermediate steps at some iterations. The authors consider the problem of parameter identification of a continuous-time linear dynamic system given discrete-time observations, and show that the proposed algorithm accelerates the convergence of the EM algorithm and simplifies the computations involved     

Multiple frequencies and AR parameters estimation from one bitquantized signal via the EM algorithm

The authors present a novel algorithm which is used to estimate the coefficients of q AR processes from a coarsely quantized signal. The input signal to the quantizer is the superposition of q AR processes and noise. In a related problem a modified version of the above algorithm is used to estimate the frequencies of coarsely quantized data obtained from q sinusoids embedded in noise. The proposed algorithm can accommodate a nonuniform m-level quantizer, as well as the special case of a one bit quantizer. The proposed estimator is based on the maximum likelihood (ML) criterion, and is realized by judiciously combining the expectation-maximization (EM) algorithm of Dempster, Laird and Rubin (1977), and the “Gaussian fit” scheme of Curry (1970). Simulations reveal that they can accurately estimate the coefficients of several AR processes, or the frequencies of several sinusoids, from one bit quantized data at low signal to noise ratios and moderate number of observations     

Adaptive MLSDE using the EM algorithm

The theory of adaptive sequence detection incorporating estimation of channel and related parameters is studied in the context of maximum-likelihood (ML) principles in a general framework based on the expectation and maximization (EM) algorithm. A generalized ML sequence detection and estimation (GMLSDE) criterion is derived based on the EM approach, and it is shown how the per-survivor processing and per-branch processing methods emerge naturally from GMLSDE. GMLSDE is developed into a real time detection/estimation algorithm using the online EM algorithm with coupling between estimation and detection. By utilizing Titterington's (1984) stochastic approximation approach, different adaptive ML sequence detection and estimation (MLSDE) algorithms are formulated in a unified manner for different channel models and for different amounts of channel knowledge available at the receiver. Computer simulation results are presented for differentially encoded quadrature phase-shift keying in frequency flat and selective fading channels, and comparisons are made among the performances of the various adaptive MLSDE algorithms derived earlier     

MIMO-OFDM系统中基于变分贝叶斯EM算法的联合符号检测与信道估计

基于变分贝叶斯期望最大化(VBEM, variational Bayes expectation maximization)算法和Turbo原理,提出了时变信道条件下MIMO-OFDM系统中的联合符号检测与信道估计算法.设计的软入软出空时检测器在采用列表球形译码避免穷尽搜索的同时,考虑了信道估计误差方差矩阵的影响;利用空时检测获得的发送信号后验概率分布估计,推出了新的Kalman前向后向递归信道估计器.仿真结果表明,在时变多径信道条件下,提出的算法比传统EM算法和面向判决算法更加具有顽健性.     

Maximum-likelihood source localization and unknown sensor locationestimation for wideband signals in the near-field

In this paper, we derive the maximum-likelihood (ML) location estimator for wideband sources in the near field of the sensor array. The ML estimator is optimized in a single step, as opposed to other estimators that are optimized separately in relative time-delay and source location estimations. For the multisource case, we propose and demonstrate an efficient alternating projection procedure based on sequential iterative search on single-source parameters. The proposed algorithm is shown to yield superior performance over other suboptimal techniques, including the wideband MUSIC and the two-step least-squares methods, and is efficient with respect to the derived Cramer-Rao bound (CRB). From the CRB analysis, we find that better source location estimates can be obtained for high-frequency signals than low-frequency signals. In addition, large range estimation error results when the source signal is unknown, but such unknown parameter does not have much impact on angle estimation. In some applications, the locations of some sensors may be unknown and must be estimated. The proposed method is extended to estimate the range from a source to an unknown sensor location. After a number of source-location frames, the location of the uncalibrated sensor can be determined based on a least-squares unknown sensor location estimator     

Adaptive snakes using the EM algorithm.

Deformable models (e.g., snakes) perform poorly in many image analysis problems. The contour model is attracted by edge points detected in the image. However, many edge points do not belong to the object contour, preventing the active contour from converging toward the object boundary. A new algorithm is proposed in this paper to overcome this difficulty. The algorithm is based on two key ideas. First, edge points are associated in strokes. Second, each stroke is classified as valid (inlier) or invalid (outlier) and a confidence degree is associated to each stroke. The expectation maximization algorithm is used to update the confidence degrees and to estimate the object contour. It is shown that this is equivalent to the use of an adaptive potential function which varies during the optimization process. Valid strokes receive high confidence degrees while confidence degrees of invalid strokes tend to zero during the optimization process. Experimental results are presented to illustrate the performance of the proposed algorithm in the presence of clutter, showing a remarkable robustness.     

Penalized minimum matching distance-guided EM algorithm

A novel and efficient mixture model fitting technique, called penalized minimum matching distance-guided expectation–maximization (EM) algorithm, is proposed. Penalized minimum matching distance is used to find the number of mixture components very accurately. We illustrate the excellent performance of the penalized minimum matching distance-guided EM algorithm with experiments involving Gaussian mixtures.     

An EM algorithm for dynamic SPECT

In this paper we present two variants of the EM algorithm for dynamic SPECT imaging. A version based on compartmental modeling which fits a sum of exponentials and a more general approach allowing for arbitrary decaying activities. The underlying probabilistic models are discussed and the incomplete and complete data spaces are shown to be physically meaningful. We indicate that the second method, leading to a convex program in the M step, is easier to treat numerically and we present a possible numerical approach. Some preliminary numerical tests indicating the feasibility of the method are included.     

Multispectral image segmentation using the rough-set-initialized EM algorithm

The problem of segmentation of multispectral satellite images is addressed. An integration of rough-set-theoretic knowledge extraction, the Expectation Maximization (EM) algorithm, and minimal spanning tree (MST) clustering is described. EM provides the statistical model of the data and handles the associated measurement and representation uncertainties. Rough-set theory helps in faster convergence and in avoiding the local minima problem, thereby enhancing the performance of EM. For rough-set-theoretic rule generation, each band is discretized using fuzzy-correlation-based gray-level thresholding. MST enables determination of nonconvex clusters. Since this is applied on Gaussians, determined by granules, rather than on the original data points, time required is very low. These features are demonstrated on two IRS-1A four-band images. Comparison with related methods is made in terms of computation time and a cluster quality measure.     

An EM algorithm for wavelet-based image restoration

This paper introduces an expectation-maximization (EM) algorithm for image restoration (deconvolution) based on a penalized likelihood formulated in the wavelet domain. Regularization is achieved by promoting a reconstruction with low-complexity, expressed in the wavelet coefficients, taking advantage of the well known sparsity of wavelet representations. Previous works have investigated wavelet-based restoration but, except for certain special cases, the resulting criteria are solved approximately or require demanding optimization methods. The EM algorithm herein proposed combines the efficient image representation offered by the discrete wavelet transform (DWT) with the diagonalization of the convolution operator obtained in the Fourier domain. Thus, it is a general-purpose approach to wavelet-based image restoration with computational complexity comparable to that of standard wavelet denoising schemes or of frequency domain deconvolution methods. The algorithm alternates between an E-step based on the fast Fourier transform (FFT) and a DWT-based M-step, resulting in an efficient iterative process requiring O(NlogN) operations per iteration. The convergence behavior of the algorithm is investigated, and it is shown that under mild conditions the algorithm converges to a globally optimal restoration. Moreover, our new approach performs competitively with, in some cases better than, the best existing methods in benchmark tests.     

Application of the EM algorithm for the multitarget/multisensortracking problem

An important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies. A number of approaches have been proposed for this multitarget/multisensor tracking problem ranging from reasonably simple, though ad hoc, schemes to fairly complex, but theoretically optimum, approaches. In this paper, we describe an iterative procedure for time-recursive multitarget/multisensor tracking based on use of the expectation-maximization (EM) algorithm. More specifically, we pose the multitarget/multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data, whereas the target-associated sensor outputs constitute the complete data. Target updates at each time use an EM-based approach that calculates the maximum a posteriori (MAP) estimate of the target states, under the assumption of appropriate motion models, based on the outputs of disparate sensors. The approach uses a Markov random field (MRF) model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration. The advantage of this EM-based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities. We provide selected results illustrating the performance/complexity characteristics of this EM-based approach compared with competing schemes     

Improved impedance matrix localization method [EM problems]

Moment method calculations have the well-known limitations of requiring excessive storage and execution times for even modestly large electromagnetics problems. The impedance matrix localization (IML) method was introduced as a modification to standard moment method calculations to ease these limitations. It utilizes a matrix transformation which effectively changes the basis (testing) functions into ones resembling traveling waves. An improved method that uses an orthogonal transformation to generate standing-wave-like basis functions is presented here. Remarkable improvements are achieved in the numerical stability of the method and in its compatibility with iterative solvers. Furthermore, the correspondence of the large elements in this matrix to geometrical theory of diffraction (GTD) terms is strengthened, as is the possibility of further increasing the speed of iterative solutions by constructing preconditioners based on the pattern of nonzero matrix elements     

Parallelization of the EM algorithm for 3-D PET image reconstruction

The EM algorithm for PET image reconstruction has two major drawbacks that have impeded the routine use of the EM algorithm: the long computation time due to slow convergence and a large memory required for the image, projection, and probability matrix. An attempt is made to solve these two problems by parallelizing the EM algorithm on multiprocessor systems. An efficient data and task partitioning scheme, called partition-by-box, based on the message passing model is proposed. The partition-by-box scheme and its modified version have been implemented on a message passing system, Intel iPSC/2, and a shared memory system, BBN Butterfly GP1000. The implementation results show that, for the partition-by-box scheme, a message passing system of complete binary tree interconnection with fixed connectivity of three at each node can have similar performance to that with the hypercube topology, which has a connectivity of log(2) N for N PEs. It is shown that the EM algorithm can be efficiently parallelized using the (modified) partition-by-box scheme with the message passing model.     

Identification of image and blur parameters in frequency domainusing the EM algorithm

We extend a method presented previously, which considers the problem of the semicausal autoregressive (AR) parameter identification for images degraded by observation noise only. We propose a new approach to identify both the causal and semicausal AR parameters and blur parameters without a priori knowledge of the observation noise power and the PSF of the degradation. We decompose the image into 1-D independent complex scalar subsystems resulting from the vector state-space model by using the unitary discrete Fourier transform (DFT). Then, by applying the expectation-maximization (EM) algorithm to each subsystem, we identify the AR model and blur parameters of the transformed image. The AR parameters of the original image are then identified by using the least squares (LS) method. The restored image is obtained as a byproduct of the EM algorithm.