Minimum variance in biased estimation: bounds and asymptotically optimal estimators

We develop a uniform Cramer-Rao lower bound (UCRLB) on the total variance of any estimator of an unknown vector of parameters, with bias gradient matrix whose norm is bounded by a constant. We consider both the Frobenius norm and the spectral norm of the bias gradient matrix, leading to two corresponding lower bounds. We then develop optimal estimators that achieve these lower bounds. In the case in which the measurements are related to the unknown parameters through a linear Gaussian model, Tikhonov regularization is shown to achieve the UCRLB when the Frobenius norm is considered, and the shrunken estimator is shown to achieve the UCRLB when the spectral norm is considered. For more general models, the penalized maximum likelihood (PML) estimator with a suitable penalizing function is shown to asymptotically achieve the UCRLB. To establish the asymptotic optimality of the PML estimator, we first develop the asymptotic mean and variance of the PML estimator for any choice of penalizing function satisfying certain regularity constraints and then derive a general condition on the penalizing function under which the resulting PML estimator asymptotically achieves the UCRLB. This then implies that from all linear and nonlinear estimators with bias gradient whose norm is bounded by a constant, the proposed PML estimator asymptotically results in the smallest possible variance.     

Asymptotically optimal blind fractionally spaced channel estimationand performance analysis

When the received data are fractionally sampled, the magnitude and phase of most linear time-invariant FIR communications channels can be estimated from second-order output only statistics. We present a general cyclic correlation matching algorithm for known order FIR blind channel identification that has closed-form expressions for calculating the asymptotic variance of the channel estimates. We show that for a particular choice of weights, the weighted matching estimator yields (at least for large samples) the minimum variance channel estimator among all unbiased estimators based on second-order statistics. Furthermore, the matching approach, unlike existing methods, provides a useful estimate even when the channel is not uniquely identifiable from second-order statistics     

Uniformly Improving the CramÉr-Rao Bound and Maximum-Likelihood Estimation

An important aspect of estimation theory is characterizing the best achievable performance in a given estimation problem, as well as determining estimators that achieve the optimal performance. The traditional CramÉr–Rao type bounds provide benchmarks on the variance of any estimator of a deterministic parameter vector under suitable regularity conditions, while requiring a-priori specification of a desired bias gradient. In applications, it is often not clear how to choose the required bias. A direct measure of the estimation error that takes both the variance and the bias into account is the mean squared error (MSE), which is the sum of the variance and the squared-norm of the bias. Here, we develop bounds on the MSE in estimating a deterministic parameter vector$ bf x_0$over all bias vectors that are linear in$ bf x_0$, which includes the traditional unbiased estimation as a special case. In some settings, it is possible to minimize the MSE over all linear bias vectors. More generally, direct minimization is not possible since the optimal solution depends on the unknown$ bf x_0$. Nonetheless, we show that in many cases, we can find bias vectors that result in an MSE bound that is smaller than the CramÉr–Rao lower bound (CRLB) for all values of$ bf x_0$. Furthermore, we explicitly construct estimators that achieve these bounds in cases where an efficient estimator exists, by performing a simple linear transformation on the standard maximum likelihood (ML) estimator. This leads to estimators that result in a smaller MSE than the ML approach for all possible values of$ bf x_0$.     

Optimal blind nonlinear least-squares carrier phase and frequency offset estimation for general QAM modulations

This paper introduces a family of blind feedforward nonlinear least-squares (NLS) estimators for joint estimation of the carrier phase and frequency offset of general quadrature amplitude modulated (QAM) transmissions. As an extension of the Viterbi and Viterbi (1983) estimator, a constellation-dependent optimal matched nonlinear estimator is derived such that its asymptotic (large sample) variance is minimized. A class of conventional monomial estimators is also proposed. The asymptotic performance of these estimators is established in closed-form expression and compared with the Cramer-Rao lower bound. A practical implementation of the optimal matched estimator, which is a computationally efficient approximation of the latter and exhibits negligible performance loss, is also derived. Finally, computer simulations are presented to corroborate the theoretical performance analysis and indicate that the proposed optimal matched nonlinear estimator improves significantly the performance of the classic fourth-power estimator.     

A wavelet-based joint estimator of the parameters of long-rangedependence

A joint estimator is presented for the two parameters that define the long-range dependence phenomenon in the simplest case. The estimator is based on the coefficients of a discrete wavelet decomposition, improving a wavelet-based estimator of the scaling parameter (Abry and Veitch 1998), as well as extending it to include the associated power parameter. An important feature is its conceptual and practical simplicity, consisting essentially in measuring the slope and the intercept of a linear fit after a discrete wavelet transform is performed, a very fast (O(n)) operation. Under well-justified technical idealizations the estimator is shown to be unbiased and of minimum or close to minimum variance for the scale parameter, and asymptotically unbiased and efficient for the second parameter. Through theoretical arguments and numerical simulations it is shown that in practice, even for small data sets, the bias is very small and the variance close to optimal for both parameters. Closed-form expressions are given for the covariance matrix of the estimator as a function of data length, and are shown by simulation to be very accurate even when the technical idealizations are not satisfied. Comparisons are made against two maximum-likelihood estimators. In terms of robustness and computational cost the wavelet estimator is found to be clearly superior and statistically its performance is comparable. We apply the tool to the analysis of Ethernet teletraffic data, completing an earlier study on the scaling parameter alone     

A sinusoidal contrast function for the blind separation of statistically independent sources

The authors propose a new solution to the blind separation of sources (BSS) based on statistical independence. In the two-dimensional (2-D) case, we prove that, under the whiteness constraint, the fourth-order moment-based approximation of the marginal entropy (ME) cost function yields a sinusoidal objective function. Therefore, we can minimize it by simply estimating its phase. We prove that this estimator is consistent for any source distribution. In addition, such results are useful for interpreting other algorithms such as the cumulant-based independent component analysis (CuBICA) and the weighted approximate maximum likelihood (WAML) [or weighted estimator (WE)]. Based on the WAML, we provide a general unifying form for several previous approximations to the ME contrast. The bias and the variance of this estimator have been included. Finally, simulations illustrate the good consistency, convergence, and accuracy of the proposed method.     

Restricted structure optimal linear estimators

The restricted structure optimal deconvolution filtering, smoothing and prediction problem for multivariable, discrete-time linear signal processing problems is considered. A new class of discrete-time optimal linear estimators is introduced that directly minimises a minimum variance criterion but where the structure is prespecified to have a relatively simple form. The resulting estimator can be of much lower order than a Kalman or Wiener estimator and it minimises the estimation error variance, subject to the constraint referred to above. The numerical optimisation algorithm is simple to implement and the full-order optimal solutions are available as a by-product of the analysis. Moreover, the restricted structure solution may be used to compute both IIR and FIR estimators. A weighted H/sub 2/ cost-function is minimised, where the dynamic weighting function can be chosen for robustness improvement. The signal and noise sources can be correlated and the signal channel dynamics can be included in the system model. The algorithm enables low-order optimal estimators to be computed that directly minimise the cost index. The main technical advance is in the pre-processing, which enables the expanded cost expression to be simplified considerably before the numerical solution is obtained. The optimisation provides a direct minimisation over the unknown parameters for the particular estimator structure chosen. This should provide advantages over the simple approximation of a high-order optimal estimator. The results are demonstrated in the estimation of a signal heavily contaminated by both coloured and white noise.     

Optimal blind carrier recovery for MPSK burst transmissions

The paper introduces and analyzes the asymptotic (large sample) performance of a family of blind feedforward nonlinear least-squares (NLS) estimators for joint estimation of carrier phase, frequency offset, and Doppler rate for burst-mode phase-shift keying transmissions. An optimal or "matched" nonlinear estimator that exhibits the smallest asymptotic variance within the family of envisaged blind NLS estimators is developed. The asymptotic variance of these estimators is established in closed-form expression and shown to approach the Cramer-Rao lower bound of an unmodulated carrier at medium and high signal-to-noise ratios (SNR). Monomial nonlinear estimators that do not depend on the SNR are also introduced and shown to perform similarly to the SNR-dependent matched nonlinear estimator. Computer simulations are presented to corroborate the theoretical performance analysis.     

Bias compensation for the bearings-only pseudolinear target track estimator

The bearings-only pseudolinear target track estimator is known to suffer from severe bias problems. This paper presents a bias analysis for the pseudolinear estimator and develops a method of bias compensation, resulting in a closed-form reduced-bias pseudolinear estimator. The reduced-bias estimator is then incorporated into an instrumental variable estimator to produce asymptotically unbiased target motion parameter estimates. Unlike batch iterative estimators, the proposed instrumental variable estimator has a closed-from solution and therefore avoids the convergence problems associated with iterative estimators. The performance of the proposed instrumental variable estimator is illustrated by way of simulation examples and is shown to be almost identical to that of the computationally more demanding iterative maximum likelihood estimator.     

Blind feedforward cyclostationarity-based timing estimation for linear modulations

By exploiting a general cyclostationary (CS) statistics-based framework, this letter develops a rigorous and unified asymptotic (large sample) performance analysis setup for a class of blind feedforward timing epoch estimators for linear modulations transmitted through time nonselective flat-fading channels. Within the proposed CS framework, it is shown that several estimators proposed in the literature can be asymptotically interpreted as maximum likelihood (ML) estimators applied on a (sub)set of the second- (and/or higher) order statistics of the received signal. The asymptotic variance of these ML estimators is established in closed-form expression and compared with the modified Crame/spl acute/r-Rao bound. It is shown that the timing estimator proposed by Oerder and Meyr achieves asymptotically the best performance in the class of estimators which exploit all the second-order statistics of the received signal, and its performance is insensitive to oversampling rates P as long as P/spl ges/3. Further, an asymptotically best consistent estimator, which achieves the lowest asymptotic variance among all the possible estimators that can be derived by exploiting jointly the second- and fourth-order statistics of the received signal, is also proposed.     

Class of cyclic-based estimators for frequency-offset estimation ofOFDM systems

We present a new class of blind cyclic-based estimators for carrier frequency-offset and symbol-timing error estimation of orthogonal frequency-division multiplexing (OFDM) systems. The proposed approach exploits the properties of the cyclic prefix subset to reveal the synchronization parameters in the likelihood function of the received vector. A new likelihood function for the joint timing and frequency-offset estimation is derived, which globally characterizes the estimation problem. The resulting probabilistic measure is used to develop three classes of unbiased estimators, namely, maximum-likelihood, minimum variance unbiased, and moment estimator. In comparison to the previously proposed methods, the proposed estimators in this study are computationally and statistically efficient, which makes the estimators more attractive for real-time applications. The performance of the estimators is assessed by simulation for an OFDM system     

An alternative blind feedforward symbol timing estimator using two samples per symbol

Recently, S.J. Lee proposed a blind feedforward symbol timing estimator that exhibits low computational complexity and requires only two samples per symbol (see IEEE Commun. Lett., vol.6, p.205-7, 2002). We analyze Lee's estimator rigorously by exploiting efficiently the cyclostationary statistics present in the received oversampled signal; its asymptotic (large sample) bias and mean-square error (MSE) are derived in closed-form expression. A new blind feedforward timing estimator that requires only two samples per symbol and presents the same computational complexity as Lee's estimator is proposed. It is shown that the proposed new estimator is asymptotically unbiased and exhibits smaller MSE than Lee's estimator. Computer simulations are presented to illustrate the performance of the proposed new estimator with respect to Lee's estimator and existing conventional estimators.     

Cramer–Rao bound and minimum variance unbiased estimator for joint sampling clock offset and channel taps in MC‐CDMA systems

Exact closed‐form expressions of the Cramer–Rao bound (CRB) for joint sampling clock offset and channel taps are obtained in multi‐carrier code division multiple access systems. CRB is undoubtedly the most well known variance's bound to determine. It provides a benchmark against which we can compare the performance of any unbiased estimator. Furthermore, minimum variance unbiased (MVU) estimator for these parameters is proposed. Moreover, maximum likelihood (ML) and least‐squares estimators for joint sampling clock offset and channel taps are presented. Best linear unbiased estimator is also introduced just for channel taps. The performances of the estimators are compared through simulation results with the proposed CRB. Our results show the better performances of MVU and ML estimators with more computational complexity compared with the others. Copyright © 2008 John Wiley & Sons, Ltd.     

Parameter estimator based on a minimum discrepancy criterion: aBayesian approach

A new estimation criterion based on the discrepancy between the estimator's error covariance and its information lower bound is proposed. This discrepancy measure criterion tries to take the information content of the observed data into account. A minimum discrepancy estimator (MDE) is then obtained under a linearity assumption. This estimator is shown to be equivalent to the maximum likelihood estimator (MLE), if one assumes that a linear efficient estimator exists and the prior distribution of parameters is uniform. Moreover, it is equivalent to the minimum variance unbiased estimator (MVUE) if the MDE is required to be unbiased. Illustrative examples of MDE and its comparisons with other estimators are given     

Low-rank estimation of higher order statistics

Low-rank estimators for higher order statistics are considered in this paper. The bias-variance tradeoff is analyzed for low-rank estimators of higher order statistics using a tensor product formulation for the moments and cumulants. In general, the low-rank estimators have a larger bias and smaller variance than the corresponding full-rank estimator, and the mean-squared error can be significantly smaller. This makes the low-rank estimators extremely useful for signal processing algorithms based on sample estimates of the higher order statistics. The low-rank estimators also offer considerable reductions in the computational complexity of such algorithms. The design of subspaces to optimize the tradeoffs between bias, variance, and computation is discussed, and a noisy input, noisy output system identification problem is used to illustrate the results     

The statistical performance of some instantaneous frequencyestimators

The authors examine the class of smoothed central finite difference (SCFD) instantaneous frequency (IF) estimators which are based on finite differencing of the phase of the analytic signal. These estimators are closely related to IF estimation via the (periodic) first moment, with respect to frequency of discrete time-frequency representations (TFRs) in L. Cohen's (1966) class. The authors determine the distribution of this class of estimators and establish a framework which allows the comparison of several other estimators such as the zero-crossing estimator and one based on linear regression on the signal phase. It is found that the regression IF estimator is biased and exhibits a large threshold for much of the frequency range. By replacing the linear convolution operation in the regression estimator with the appropriate convolution operation for circular data the authors obtain the parabolic SCFD (PSCFD) estimator, which is unbiased and has a frequency-independent variance, yet retains the optimal performance and simplicity of the original estimator     

Properties of characteristics estimators of periodically correlated random processes in preliminary determination of the period of correlation

The coherent estimators of probabilistic characteristics of periodically correlated random processes with unknown period have been investigated. It is shown that these estimators are asymptotically unbiased and consistent. In a first approximation formulas were obtained for the bias and dispersion of estimators defining the impact of the preliminary determination of the period on the value of estimation error.     

Efficient Subspace-Based Estimator for Localization of Multiple Incoherently Distributed Sources

In this paper, a new subspace-based algorithm for parametric estimation of angular parameters of multiple incoherently distributed sources is proposed. This approach consists of using the subspace principle without any eigendecomposition of the covariance matrix, so that it does not require the knowledge of the effective dimension of the pseudosignal subspace, and therefore the main difficulty of the existing subspace estimators can be avoided. The proposed idea relies on the use of the property of the inverse of the covariance matrix to exploit approximately the orthogonality property between column vectors of the noise-free covariance matrix and the sample pseudonoise subspace. The resulting estimator can be considered as a generalization of the Pisarenko's extended version of Capon's estimator from the case of point sources to the case of incoherently distributed sources. Theoretical expressions are derived for the variance and the bias of the proposed estimator due to finite sample effect. Compared with other known methods with comparable complexity, the proposed algorithm exhibits a better estimation performance, especially for close source separation, for large angular spread and for low signal-to-noise ratio.     

Variance estimation and adaptive quantization (Corresp.)

The relationship between variance estimation and adaptive quantization is investigated for memoryless Laplacian and Gaussian sources. Comparison of block average, exponential average, and maximum likelihood estimators in an adaptive quantization scheme indicates that estimator precision (variance) is more important than accuracy (bias) in minimizing distortion. Further, the block average and exponential average estimators are inconsistent when used in backward adaptive quantization.     

Admissible, Minimax and Equivariant Estimators of Life Distributions from Type II Censored Samples

In life testing, the unique minimum variance unbiased estimator (MVUE) ? is often used when it exists. However it has been shown for certain distributions that an estimator of the form k? with uniformly smaller mean square error exists. Such extimators are derived here for a class of life distributions and are shown to be admissible, minimax, and (in most cases) equivariant. The underlying distribution from which the samples are drawn follows a generalized life model (GLM) which includes a model proposed by Epstein & Sobel, Weibull, exponential, and Rayleigh distributions as special cases. Results are also given for the Type II asymptotic distribution of largest values, Pareto, and limited distributions. In addition, admissible linear estimators of the form a? + b are obtained and it is shown that they are a form of locally best estimators for some portion of the parameter space. Both k? and a? + b could be used in nonrepetitive estimation problems where bias causes no difficulty.