Uniformly Improving the CramÉr-Rao Bound and Maximum-Likelihood Estimation

An important aspect of estimation theory is characterizing the best achievable performance in a given estimation problem, as well as determining estimators that achieve the optimal performance. The traditional CramÉr–Rao type bounds provide benchmarks on the variance of any estimator of a deterministic parameter vector under suitable regularity conditions, while requiring a-priori specification of a desired bias gradient. In applications, it is often not clear how to choose the required bias. A direct measure of the estimation error that takes both the variance and the bias into account is the mean squared error (MSE), which is the sum of the variance and the squared-norm of the bias. Here, we develop bounds on the MSE in estimating a deterministic parameter vector$ bf x_0$over all bias vectors that are linear in$ bf x_0$, which includes the traditional unbiased estimation as a special case. In some settings, it is possible to minimize the MSE over all linear bias vectors. More generally, direct minimization is not possible since the optimal solution depends on the unknown$ bf x_0$. Nonetheless, we show that in many cases, we can find bias vectors that result in an MSE bound that is smaller than the CramÉr–Rao lower bound (CRLB) for all values of$ bf x_0$. Furthermore, we explicitly construct estimators that achieve these bounds in cases where an efficient estimator exists, by performing a simple linear transformation on the standard maximum likelihood (ML) estimator. This leads to estimators that result in a smaller MSE than the ML approach for all possible values of$ bf x_0$.     

EM/SAGE Based ML Channel Estimation for Uplink DS-CDMA Systems over Time-Varying Fading Channels

The matrix inversion for the maximum likelihood (ML) channel estimation requires high complexity for the direct-sequence code-division multiple-access (DS-CDMA) systems. The prime motivation of the paper is to propose channel estimators that achieve mean square error (MSE) performance of ML channel estimator in an iterative manner without any matrix inversion. Therefore, two computationally efficient solutions to the problem of ML channel estimation are proposed.We compare the both algorithms in terms of the number of used iteration and show that the proposed algorithms converge the same MSE performance of the ML estimator as the increasing number of iterations.     

Cramer–Rao bound and minimum variance unbiased estimator for joint sampling clock offset and channel taps in MC‐CDMA systems

Exact closed‐form expressions of the Cramer–Rao bound (CRB) for joint sampling clock offset and channel taps are obtained in multi‐carrier code division multiple access systems. CRB is undoubtedly the most well known variance's bound to determine. It provides a benchmark against which we can compare the performance of any unbiased estimator. Furthermore, minimum variance unbiased (MVU) estimator for these parameters is proposed. Moreover, maximum likelihood (ML) and least‐squares estimators for joint sampling clock offset and channel taps are presented. Best linear unbiased estimator is also introduced just for channel taps. The performances of the estimators are compared through simulation results with the proposed CRB. Our results show the better performances of MVU and ML estimators with more computational complexity compared with the others. Copyright © 2008 John Wiley & Sons, Ltd.     

Error probability estimator structures based on analysis of thereceiver decision variable

The paper presents schemes for rapid on-line error probability estimation of digital communications links. Several estimator structures are proposed based on the assumption of sample independence, including weighted least squares (WLS) and maximum likelihood (ML) forms. The continuous-form ML estimator is shown to lie on the Rao-Cramer bound, making it a most efficient estimator of probability of error. The design, performance, implementation complexity and behavior of these estimators is described for AWGN     

Asymptotically Efficient Reduced Complexity Frequency Offset and Channel Estimators for Uplink MIMO-OFDMA Systems

In this paper, we address the joint data-aided estimation of frequency offsets and channel coefficients in uplink multiple-input multiple-output orthogonal frequency-division multiple access (MIMO-OFDMA) systems. As the maximum-likelihood (ML) estimator is impractical in this context, we introduce a family of suboptimal estimators with the aim of exhibiting an attractive tradeoff between performance and complexity. The estimators do not rely on a particular subcarrier assignment scheme and are, thus, valid for a large number of OFDMA systems. As far as complexity is concerned, the computational cost of the proposed estimators is shown to be significantly reduced compared to existing estimators based on ML. As far as performance is concerned, the proposed suboptimal estimators are shown to be asymptotically efficient, i.e., the covariance matrix of the estimation error achieves the Cramer-Rao bound when the total number of subcarriers increases. Simulation results sustain our claims.     

Frequency estimation of a single complex sinusoid using a generalized Kay estimator

This letter introduces a generalized version of Kay's estimator for the frequency of a single complex sinusoid in complex additive white Gaussian noise. The Kay estimator is a maximum-likelihood (ML) estimator at high signal-to-noise ratio (SNR) based on differential phase measurements with a delay of one symbol interval. In this letter, the corresponding ML estimator with an arbitrary delay in the differential phase measurements is derived. The proposed estimator reduces the variance at low SNR, compared with Kay's original estimator. For certain delay values, explicit expressions for the window function and the corresponding high SNR variance of the proposed generalized Kay (GK) estimator are presented. Furthermore, for some delay values, the window function is nearly uniform and the implementation complexity is reduced, compared with the original Kay estimator. For a delay value of two, we show that the variance at asymptotically high SNR approaches the Cramer-Rao bound as the sequence length tends to infinity. We also explore the effect of exchanging the order of summation and phase extraction for reduced-complexity reasons. The resulting generalized weighted linear predictor estimator and the GK estimator are compared with both autocorrelation-based and periodogram-based estimators in terms of computational complexity, estimation range, and performance at both low and high SNRs.     

Performance of robust symbol‐timing and carrier‐frequency estimation for OFDM systems

In recent years, many maximum likelihood (ML) blind estimators have been proposed to estimate timing and frequency offsets for orthogonal frequency division multiplexing (OFDM) systems. However, the previously proposed ML blind estimators utilizing cyclic prefix do not fully characterize the random observation vector over the entire range of the timing offset and will significantly degrade the estimation performance. In this paper, we present a global ML blind estimator to compensate the estimation error. Moreover, we extend the global ML blind estimator by accumulating the ML function of the estimation parameters to achieve a better accuracy without increasing the hardware or computational complexity. The simulation results show that the proposed algorithm can significantly improve the estimation performance in both additional white Gaussian noise and ITU‐R M.1225 multipath channels. Copyright © 2008 John Wiley & Sons, Ltd.     

Multiscale Poisson Intensity and Density Estimation

The nonparametric Poisson intensity and density estimation methods studied in this paper offer near minimax convergence rates for broad classes of densities and intensities with arbitrary levels of smoothness. The methods and theory presented here share many of the desirable features associated with wavelet-based estimators: computational speed, spatial adaptivity, and the capability of detecting discontinuities and singularities with high resolution. Unlike traditional wavelet-based approaches, which impose an upper bound on the degree of smoothness to which they can adapt, the estimators studied here guarantee nonnegativity and do not require any a priori knowledge of the underlying signal's smoothness to guarantee near-optimal performance. At the heart of these methods lie multiscale decompositions based on free-knot, free-degree piecewise-polynomial functions and penalized likelihood estimation. The degrees as well as the locations of the polynomial pieces can be adapted to the observed data, resulting in near-minimax optimal convergence rates. For piecewise-analytic signals, in particular, the error of this estimator converges at nearly the parametric rate. These methods can be further refined in two dimensions, and it is demonstrated that platelet-based estimators in two dimensions exhibit similar near-optimal error convergence rates for images consisting of smooth surfaces separated by smooth boundaries.     

Performance of the maximum likelihood constant frequency estimatorfor frequency tracking

The performance of maximum likelihood (ML) estimators for an important frequency estimation problem is considered when the signal model assumptions are not valid. The motivation for this problem is to understand the robustness of the hidden Markov model-maximum likelihood (HMM-ML) tandem frequency estimator, where the signal is divided into time blocks, and the frequency in each time block is estimated using the ML approach under the assumption that the signal has a constant frequency in each time block. In order to analyze the sensitivity of ML estimators to the model assumptions, the mean frequency of a discrete complex tone that has a time-varying (ramp) frequency is estimated under the incorrect assumption that it has a constant frequency. In particular, the behavior of the threshold region with respect to different chirp rates is analyzed, and a simple rule is given. The mean squared error above the threshold region is shown to be constant even at very high SNR levels. These results are supported by simulations     

Maximum Likelihood Localization of a Diffusive Point Source Using Binary Observations

In this paper, we investigate the problem of localization of a diffusive point source of gas based on binary observations provided by a distributed chemical sensor network. We motivate the use of the maximum likelihood (ML) estimator for this scenario by proving that it is consistent and asymptotically efficient, when the density of the sensors becomes infinite. We utilize two different estimation approaches, ML estimation based on all the observations (i.e., batch processing) and approximate ML estimation using only new observations and the previous estimate (i.e., real time processing). The performance of these estimators is compared with theoretical bounds and is shown to achieve excellent performance, even with a finite number of sensors     

Threshold region performance of maximum likelihood direction of arrival estimators

This paper presents a performance analysis of the maximum likelihood (ML) estimator for finding the directions of arrival (DOAs) with a sensor array. The asymptotic properties of this estimator are well known. In this paper, the performance under conditions of low signal-to-noise ratio (SNR) and a small number of array snapshots is investigated. It is well known that the ML estimator exhibits a threshold effect, i.e., a rapid deterioration of estimation accuracy below a certain SNR or number of snapshots. This effect is caused by outliers and is not captured by standard techniques such as the Crame/spl acute/r-Rao bound and asymptotic analysis. In this paper, approximations to the mean square estimation error and probability of outlier are derived that can be used to predict the threshold region performance of the ML estimator with high accuracy. Both the deterministic ML and stochastic ML estimators are treated for the single-source and multisource estimation problems. These approximations alleviate the need for time-consuming computer simulations when evaluating the threshold region performance. For the special case of a single stochastic source signal and a single snapshot, it is shown that the ML estimator is not statistically efficient as SNR/spl rarr//spl infin/ due to the effect of outliers.     

A new approximate likelihood estimator for ARMA-filtered hiddenMarkov models

Hidden Markov models (HMMs) are successfully applied in various fields of time series analysis. Colored noise, e.g., due to filtering, violates basic assumptions of the model. Although it is well known how to consider autoregressive (AR) filtering, there is no algorithm to take into account moving-average (MA) filtering in parameter estimation exactly. We present an approximate likelihood estimator for MA-filtered HMM that is generalized to deal with an autoregressive moving-average (ARMA) filtered HMM. The approximation order of the likelihood calculation can be chosen. Therefore, we obtain a sequence of estimators for the HMM parameters as well as for the filter coefficients. The recursion equations for an efficient algorithm are derived from exact expressions for the forward iterations. By simulations, we show that the derived estimators are unbiased in filter situations where standard HMM's are not able to recover the true dynamics. Special implementation strategies together with small approximations yield further acceleration of the algorithm     

Minimum variance in biased estimation: bounds and asymptotically optimal estimators

We develop a uniform Cramer-Rao lower bound (UCRLB) on the total variance of any estimator of an unknown vector of parameters, with bias gradient matrix whose norm is bounded by a constant. We consider both the Frobenius norm and the spectral norm of the bias gradient matrix, leading to two corresponding lower bounds. We then develop optimal estimators that achieve these lower bounds. In the case in which the measurements are related to the unknown parameters through a linear Gaussian model, Tikhonov regularization is shown to achieve the UCRLB when the Frobenius norm is considered, and the shrunken estimator is shown to achieve the UCRLB when the spectral norm is considered. For more general models, the penalized maximum likelihood (PML) estimator with a suitable penalizing function is shown to asymptotically achieve the UCRLB. To establish the asymptotic optimality of the PML estimator, we first develop the asymptotic mean and variance of the PML estimator for any choice of penalizing function satisfying certain regularity constraints and then derive a general condition on the penalizing function under which the resulting PML estimator asymptotically achieves the UCRLB. This then implies that from all linear and nonlinear estimators with bias gradient whose norm is bounded by a constant, the proposed PML estimator asymptotically results in the smallest possible variance.     

Minimax nonparametric classification. II. Model selection foradaptation

For pt.I see ibid., vol.45, no.7, p.2271-84 (1999). We study nonparametric estimation of a conditional probability for classification based on a collection of finite-dimensional models. For the sake of flexibility, different types of models, linear or nonlinear, are allowed as long as each satisfies a dimensionality assumption. We show that with a suitable model selection criterion, the penalized maximum-likelihood estimator has a risk bounded by an index of resolvability expressing a good tradeoff among approximation error, estimation error, and model complexity. The bound does not require any assumption on the target conditional probability and can be used to demonstrate the adaptivity of estimators based on model selection. Examples are given with both splines and neural nets, and problems of high-dimensional estimation are considered. The resulting adaptive estimator is shown to behave optimally or near optimally over Sobolev classes (with unknown orders of interaction and smoothness) and classes of integrable Fourier transform of gradient. In terms of rates of convergence, the performance is the same as if one knew which of them contains the true conditional probability in advance. The corresponding classifier also converges optimally or nearly optimally simultaneously over these classes     

Monte Carlo Comparisons of Parameter Estimators of the 2-parameter Weibull Distribution

A Monte Carlo Simulation was carried out in order to compare three different estimators of the 2-parameter Weibull distribution. The estimators were the ML (maximum likelihood) estimators and two other estimator pairs suggested by Bain & Antle. The Bain-Antle estimators are better than the ML estimator for small samples (in that their bias, standard deviation, and rms error are smaller), whereas the ML estimator is superior in large samples.     

Second-order parameter estimation

This work provides a general framework for the design of second-order blind estimators without adopting any approximation about the observation statistics or the a priori distribution of the parameters. The proposed solution is obtained minimizing the estimator variance subject to some constraints on the estimator bias. The resulting optimal estimator is found to depend on the observation fourth-order moments that can be calculated analytically from the known signal model. Unfortunately, in most cases, the performance of this estimator is severely limited by the residual bias inherent to nonlinear estimation problems. To overcome this limitation, the second-order minimum variance unbiased estimator is deduced from the general solution by assuming accurate prior information on the vector of parameters. This small-error approximation is adopted to design iterative estimators or trackers. It is shown that the associated variance constitutes the lower bound for the variance of any unbiased estimator based on the sample covariance matrix. The paper formulation is then applied to track the angle-of-arrival (AoA) of multiple digitally-modulated sources by means of a uniform linear array. The optimal second-order tracker is compared with the classical maximum likelihood (ML) blind methods that are shown to be quadratic in the observed data as well. Simulations have confirmed that the discrete nature of the transmitted symbols can be exploited to improve considerably the discrimination of near sources in medium-to-high SNR scenarios.     

A Hybrid SS–ToA Wireless NLoS Geolocation Based on Path Attenuation: ToA Estimation and CRB for Mobile Position Estimation

We propose a new hybrid wireless geolocation scheme that requires only one observation quantity, namely, the received signal. The attenuation model is explored herein to capture the propagation features from the received signal. Thus, it provides a more accurate approach for wireless geolocation. To investigate geolocation accuracy, we consider the time-of-arrival (ToA) estimation in the presence of path attenuation. The maximum-correlation (MC) estimator is revisited, and the exact maximum-likelihood (ML) estimator is derived to estimate the ToA. The error performance of the ToA estimates is derived using a Taylor expansion. It is shown that the ML estimate is unbiased and has a smaller error variance than the MC estimate. Numerical results illustrate that, for a low effective bandwidth, the ML estimator well outperforms the MC estimator. Afterward, we derive the CramÉr–Rao bound (CRB) for the mobile position estimation. The obtained result, which is applicable to any value of path loss exponents, gives a generalized form of the CRB for the ordinary geolocation approach. In seven hexagonal cells, numerical examples show that the accuracy of the mobile position estimation exploring the path loss is improved compared with that obtained by the usual geolocation.      

广义Pareto分布海杂波模型参数的组合双分位点估计方法

广义Pareto分布的复合高斯模型可以很好地描述高分辨低擦地角对海探测场景中海杂波的重拖尾特性,实现该杂波模型下双参数的有效估计对雷达检测性能具有重要意义。对此,该文提出一种双参数的组合双分位点(CBiP)估计方法。该估计方法基于低阶多项式方程的显式求根表达式,充分组合利用回波中的样本信息,旨在实现高精度的双参数估计过程。此外,考虑到实际雷达工作中存在岛礁、渔船等造成的功率异常大的野点样本时,不同于传统的矩估计、最大似然(ML)估计等方法,组合双分位点估计方法仍可保持估计性能的鲁棒性。仿真及实测数据实验表明,在纯杂波环境中,组合双分位点估计方法可以实现与最大似然估计方法近似的估计精度,若存在异常样本,组合双分位点估计方法的估计性能优于上述几种传统估计方法。     

Maximum-likelihood source localization and unknown sensor locationestimation for wideband signals in the near-field

In this paper, we derive the maximum-likelihood (ML) location estimator for wideband sources in the near field of the sensor array. The ML estimator is optimized in a single step, as opposed to other estimators that are optimized separately in relative time-delay and source location estimations. For the multisource case, we propose and demonstrate an efficient alternating projection procedure based on sequential iterative search on single-source parameters. The proposed algorithm is shown to yield superior performance over other suboptimal techniques, including the wideband MUSIC and the two-step least-squares methods, and is efficient with respect to the derived Cramer-Rao bound (CRB). From the CRB analysis, we find that better source location estimates can be obtained for high-frequency signals than low-frequency signals. In addition, large range estimation error results when the source signal is unknown, but such unknown parameter does not have much impact on angle estimation. In some applications, the locations of some sensors may be unknown and must be estimated. The proposed method is extended to estimate the range from a source to an unknown sensor location. After a number of source-location frames, the location of the uncalibrated sensor can be determined based on a least-squares unknown sensor location estimator     

Computationally efficient angle estimation for signals with knownwaveforms

This paper presents a large sample decoupled maximum likelihood (DEML) angle estimator for uncorrelated narrowband plane waves with known waveforms and unknown amplitudes arriving at a sensor array in the presence of unknown and arbitrary spatially colored noise. The DEML estimator decouples the multidimensional problem of the exact ML estimator to a set of 1-D problems and, hence, is computationally efficient. We shall derive the asymptotic statistical performance of the DEML estimator and compare the performance with its Cramer-Rao bound (CRB), i.e., the best possible performance for the class of asymptotically unbiased estimators. We will show that the DEML estimator is asymptotically statistically efficient for uncorrelated signals with known waveforms. We will also show that for moderately correlated signals with known waveforms, the DEML estimator is no longer a large sample maximum likelihood (ML) estimator, but the DEML estimator may still be used for angle estimation, and the performance degradation relative to the CRB is small. We shall show that the DEML estimator can also be used to estimate the arrival angles of desired signals with known waveforms in the presence of interfering or jamming signals by modeling the interfering or jamming signals as random processes with an unknown spatial covariance matrix. Finally, several numerical examples showing the performance of the DEML estimator are presented in this paper