Error bounds for convex parameter estimation

We evaluate the accuracy of sparsity-based estimation methods inspired from compressed sensing. Typical estimation approaches consist of minimizing a non-convex cost function that exhibits local minima, and require excessive computational resources. A tractable alternative relies on a sparse representation of the observation vector using a large dictionary matrix and a convex cost function. This estimation approach converts the intractable high-dimensional non-convex problem into a simpler convex problem with reduced dimension. Unfortunately, the advantages come at the expense of increased estimation error. Therefore, an evaluation of the estimation error is of considerable interest. We consider the case of estimating a single parameter vector, and provide upper bounds on the achievable accuracy. The theoretical results are corroborated by simulations.     

Bayesian bounds for matched-field parameter estimation

Matched-field methods concern estimation of source locations and/or ocean environmental parameters by exploiting full wave modeling of acoustic waveguide propagation. Because of the nonlinear parameter-dependence of the signal field, the estimate is subject to ambiguities and the sidelobe contribution often dominates the estimation error below a threshold signal-to-noise ratio (SNR). To study the matched-field performance, three Bayesian lower bounds on mean-square error are developed: the Bayesian Crame/spl acute/r-Rao bound (BCRB), the Weiss-Weinstein bound (WWB), and the Ziv-Zakai bound (ZZB). Particularly, for a multiple-frequency, multiple-snapshot random signal model, a closed-form minimum probability of error associated with the likelihood ratio test is derived, which facilitates error analysis in a wide scope of applications. Analysis and example simulations demonstrate that 1) unlike the local CRB, the BCRB is not achieved by the maximum likelihood estimate (MLE) even at high SNR if the local performance is not uniform across the prior parameter space; 2) the ZZB gives the closest MLE performance prediction at most SNR levels of practical interest; 3) the ZZB can also be used to determine the necessary number of independent snapshots achieving the asymptotic performance of the MLE at a given SNR; 4) incoherent frequency averaging, which is a popular multitone processing approach, reduces the peak sidelobe error but may not improve the overall performance due to the increased ambiguity baseline; and finally, 5) effects of adding additional parameters (e.g., environmental uncertainty) can be well predicted from the parameter coupling.     

Performance bounds for adaptive estimation

The performance of adaptive state estimators for linear dynamic systems is investigated. The adaptive state estimates are formed under the assumption that the unknown system parameter belongs to a finite set and is thus readily implementable. It is shown that, for the true parameter value in a prescribed region in the parameter space, the corresponding a posteriori probablity (or weighting coefficient in the adaptive estimator) converges exponentially in the vth mean (v > 1) and almost surely to unity. The analysis is based on the asymptotic per sample formula for the Kullback information function, which is derived in this paper. The significance of the analysis for applications is also examined.     

Feasibility study of parameter estimation of random sampling jitter using the bispectrum

An actual sampling process can be modeled as a random process, which consists of the regular (uniform) deterministic sampling process plus an error in the sampling times which constitutes a zero-mean noise (the jitter). In this paper we discuss the problem of estimating the jitter process. By assuming that the jitter process is an i.i.d. one, with standard deviation that is small compared to the regular sampling time, we show that the variance of the jitter process can be estimated from thenth order spectrum of the sampled data,n=2, 3, i.e., the jitter variance can be extracted from the 2nd-order spectrum or the 3rd-order spectrum (the bispectrum) of the sampled data, provided the continuous signal spectrum is known. However when the signal skewness exceeds a certain level, the potential performance of the bispectrum-based estimation is better than that of the spectrum-based estimation. Moreover, the former can also provide jitter variance estimates when the continuous signal spectrum is unknown while the latter cannot. This suggests that the bispectrum of the sampled data is potentially better for estimating any parameter of the sampling jitter process, once the signal skewness is sufficiently large.     

Some lower bounds on signal parameter estimation

New bounds are presented for the maximum accuracy with which parameters of signals imbedded in white noise can be estimated. The bounds are derived by comparing the estimation problem with related optimal detection problems. They are, with few exceptions, independent of the bias and include explicitly the dependence on the a priori interval. The new results are compared with previously known results.     

Cramer-Rao lower bounds for QAM phase and frequency estimation

In this paper, we present the true Cramer-Rao lower bounds (CRLBs) for the estimation of phase offset for common quadrature amplitude modulation (QAM), PSK, and PAM signals in AWGN channels. It is shown that the same analysis also applies to the QAM, FSK, and PAM CRLBs for frequency offset estimation. The ratio of the modulated to the unmodulated CRLBs is derived for all QAM, PSK, and PAM signals and calculated for specific cases of interest. This is useful to determine the limiting performance of synchronization circuits for coherent receivers without the need to simulate particular algorithms. The hounds are compared to the existing true CRLBs for an unmodulated carrier wave (CW), BPSK, and QPSK. We investigated new and existing QAM phase estimation algorithms in order to verify the new phase CRLB. This showed that new minimum distance estimator performs close to the QAM bound and provides a large improvement over the power law estimator at moderate to high signal-to-noise ratios     

A comparison of two parameter estimation schemes

This letter compares two parameter estimation schemes using the variance of the estimate as the criterion for comparison.     

Optimized sampling and parameter estimation for quantification inwhole body PET

Whole-body positron emission tomography (PET) has recently emerged as an important imaging tool for cancer detection and staging. Initial applications of the technique have been primarily qualitative. One of the major reasons is the limits imposed by kinetically undersampled data over the whole body, as opposed to the standard method of continuous dynamic sampling in one body location. Here, a new estimation method using weighted nonlinear least squares (WNLS) for the first bed position and Bayesian regression (BR) for subsequent positions is proposed. A general criterion for designing optimal sampling schedules which maximizes the measurement information with multiple bed positions is developed. The overall approach is illustrated with the problem of estimating the metabolic rate of glucose (MRGlu) in tumors at different axial positions (image bed positions) in the body by using computer simulations and patient data. The results show that estimates of MRGlu using sparse data and the optimized Bayesian approach are comparable with those obtained by standard methods and fully sampled data. This study demonstrates the potential of the technique described for quantification where several bed positions have to be used to image all the regions of interest (ROI)     

Performance limitations and error calculations for parameter estimation

Error calculations cannot be carried out precisely when parameters are estimated which affect the observation nonlinearly. This paper summarizes the available approaches to studying performance and compares the resulting answers for a specific case. It is shown that the familiar Cramér-Rao lower bound on rms error yields an accurate answer only for large signal-to-noise ratios (SNR). For low SNR, lower bounds on rms error obtained by Ziv and Zakai give easily calculated and fairly tight answers. Rate distortion theory gives a lower bound on the error achievable with any system. The Barankin lower bound does not appear to give useful information as a computational tool. A technique for approximating the error can be used effectively for a large class of systems. With numerical integration, an upper bound obtained by Seidman gives a fairly tight answer. Recent work by Ziv gives bounds on the bias of estimators but, in general, these appear to be rather weak. Tighter results are obtained for maximum-likelihood estimators with certain symmetry conditions. Applying these techniques makes it possible to locate the threshold level to within a few decibels of channel signal-to-noise ratio. Further, these calculations can be easily carried out for any system.     

Importance sampling for the random phase Gaussian channel

Importance sampling (IS) is developed as a variance reduction technique for Monte Carlo simulation of data communications over random phase additive white Gaussian noise channels. The binary problem (with known performance) is examined initially to determine parameter values and estimate the performance gain of IS. These results can then be applied to intractable m-ary signaling problems through composite IS. An example compares the performance of linear, square-law, and optimum receivers for binary block coded data     

A general class of lower bounds in parameter estimation

A general class of Bayesian lower bounds on moments of the error in parameter estimation is formulated, and it is shown that the Cramer-Rao, the Bhattacharyya, the Bobrovsky-Zakai, and the Weiss-Weinstein lower bounds are special cases in the class. The bounds can be applied to the estimation of vector parameters and any given function of the parameters. The extension of these bounds to multiple parameter is discussed     

System error bounds for Lagrange polynomial estimation of band-limited functions (Corresp.)

Several recent articles [1]-[3] have discussed the use of Lagrange polynomials for band-limited signal estimation. We find error bounds when using Lagrange polynomials for interpolation and extrapolation of finite-power band-limited signals if a finite number of regularly spaced noisy samples are used.     

Information-theoretic bounds for authentication schemes

Consider the situation whereby each member of a sequence of n distinct messages from a finite source space is provided with an authentication check, generated using one and the same encoding rule selected from a finite space of such rules. Let P(n) be the expected probability that a spoofer, who observes such a sequence of messages and their corresponding authentication checks, but does not know the encoding rule in advance, succeeds in generating the correct authentication check for a different message in the source space. The purpose of this paper is to derive a lower bound for the average P _Nof the probabilities P(0), P(1),...;, P(N) in terms of the entropy of the space of encoding rules, and to derive necessary and sufficient conditions which the encoding scheme must satisfy in order to ensure that this bound cannot be exceeded.     

Model selection of random amplitude polynomial phase signals

Procedures for selecting the amplitude and phase models of a polynomial phase signal (PPS) modulated by a nonstationary random process are presented. The model selection is performed by using nonlinear least-squares type phase and amplitude parameter estimators with sequential generalizations of the Bonferroni test. Simulation results are used to analyze the performance of the procedures. An application to modeling of real passive acoustic data is presented     

An estimation algorithm for 2-D polynomial phase signals.

We consider nonhomogeneous 2-D signals that can be represented by a constant modulus polynomial-phase model. A novel 2-D phase differencing operator is introduced and used to develop a computationally efficient estimation algorithm for the parameters of this model. The operation of the algorithm is illustrated using an example.     

Error bounds for floating-point polynomial interpolators

Novel roundoff error bounds for floating-point addition and multiplication are introduced. The error bounds are found to be tighter than conventional error bounds. Tight error bounds are of practical interest as they allow the reduction of intermediate result word lengths, consequently reducing the complexity of dedicated arithmetic hardware. The use of the proposed error bounds is demonstrated by deriving a roundoff error bound for polynomials when used as interpolators     

Performance of diversity schemes for OFDM systems with frequency offset, phase noise, and channel estimation errors

We provide expressions for the bit error rate of various transmit and receive diversity schemes for orthogonal frequency division multiplexing (OFDM) systems in the presence of frequency offset, phase noise, and channel estimation errors. The derivations are also applicable for a general multiplicative distortion of the received signal. Our results show that with perfect channel estimates, practical values of the phase noise do not significantly degrade the performance of the various diversity methods for binary phase-shift keying modulation. In contrast, the transmit diversity schemes for OFDM are much more sensitive to channel estimation errors than maximal ratio combining receive diversity.     

ML parameter estimation for Markov random fields with applicationsto Bayesian tomography

Markov random fields (MRFs) have been widely used to model images in Bayesian frameworks for image reconstruction and restoration. Typically, these MRF models have parameters that allow the prior model to be adjusted for best performance. However, optimal estimation of these parameters (sometimes referred to as hyperparameters) is difficult in practice for two reasons: (i) direct parameter estimation for MRFs is known to be mathematically and numerically challenging; (ii) parameters can not be directly estimated because the true image cross section is unavailable. We propose a computationally efficient scheme to address both these difficulties for a general class of MRF models, and we derive specific methods of parameter estimation for the MRF model known as generalized Gaussian MRF (GGMRF). We derive methods of direct estimation of scale and shape parameters for a general continuously valued MRF. For the GGMRF case, we show that the ML estimate of the scale parameter, sigma, has a simple closed-form solution, and we present an efficient scheme for computing the ML estimate of the shape parameter, p, by an off-line numerical computation of the dependence of the partition function on p. We present a fast algorithm for computing ML parameter estimates when the true image is unavailable. To do this, we use the expectation maximization (EM) algorithm. We develop a fast simulation method to replace the E-step, and a method to improve the parameter estimates when the simulations are terminated prior to convergence. Experimental results indicate that our fast algorithms substantially reduce the computation and result in good scale estimates for real tomographic data sets.     

Estimating random amplitude polynomial phase signals: acyclostationary approach

Modeling of a class of nonstationary signals with randomly time-varying amplitude and parametric polynomial phase is addressed. A novel approach is proposed for the estimation of the time-varying phase by exploiting the higher order cyclostationarity of these signals. The method does not require nonlinear search, is easy to implement, and yields consistent estimates for the parameters. The resulting algorithms are theoretically tolerant to a large class of noises including additive stationary non-Gaussian noise and any Gaussian noise. Simulation examples supporting the theory are provided     

Wavelet parameter and phase estimation using cumulant slices

A simple, closed-form expression relates one-dimensional output cumulant statistics with the parameters of a known-order moving-average wavelet. Based on this relationship the author obtains unique parameter and phase estimates of autoregressive moving-average seismic wavelets. The input reflectivity sequence is assumed to be non-Gaussian, independent, and identically distributed. The wavelet is not assumed to be minimum phase and is allowed to include all-pass factors. The seismogram is contaminated by additive-colored Gaussian noise. Simulations demonstrate that the algorithm works well for moderate-size data records with a relatively low signal-to-noise ratio