Exploring estimator bias-variance tradeoffs using the uniform CRbound

We introduce a plane, which we call the delta-sigma plane, that is indexed by the norm of the estimator bias gradient and the variance of the estimator. The norm of the bias gradient is related to the maximum variation in the estimator bias function over a neighborhood of parameter space. Using a uniform Cramer-Rao (CR) bound on estimator variance, a delta-sigma tradeoff curve is specified that defines an “unachievable region” of the delta-sigma plane for a specified statistical model. In order to place an estimator on this plane for comparison with the delta-sigma tradeoff curve, the estimator variance, bias gradient, and bias gradient norm must be evaluated. We present a simple and accurate method for experimentally determining the bias gradient norm based on applying a bootstrap estimator to a sample mean constructed from the gradient of the log-likelihood. We demonstrate the methods developed in this paper for linear Gaussian and nonlinear Poisson inverse problems     

Estimation of average survival time of a process having an exponential lifetime model for time censored data

Censoring is almost mandatory, especially in dealing with costly sophisticated items in a life testing experiment. Assuming the survival times to be distributed exponentially with two parameters we have proposed an estimator using a preliminary test of significance for the estimation of average survival time of the time censored life data. The bias and mean square error of the proposed estimator have been studied. Based on some mathematical results and an empirical study, recommendations have been made regarding the use of proposed estimator.     

Penalized Partially Linear Models Using Sparse Representations With an Application to fMRI Time Series

In this paper, we consider modeling the nonparametric component in partially linear models (PLMs) using linear sparse representations, e.g., wavelet expansions. Two types of representations are investigated, namely, orthogonal bases (complete) and redundant overcomplete expansions. For bases, we introduce a regularized estimator of the nonparametric part. The important contribution here is that the nonparametric part can be parsimoniously estimated by choosing an appropriate penalty function for which the hard and soft thresholding estimators are special cases. This allows us to represent in an effective manner a broad class of signals, including stationary and/or nonstationary signals and avoids excessive bias in estimating the parametric component. We also give a fast estimation algorithm. The method is then generalized to handle the case of overcomplete representations. A large-scale simulation study is conducted to illustrate the finite sample properties of the estimator. The estimator is finally applied to real neurophysiological functional magnetic resonance imaging (MRI) data sets that are suspected to contain both smooth and transient drift features.     

Second-order parameter estimation

This work provides a general framework for the design of second-order blind estimators without adopting any approximation about the observation statistics or the a priori distribution of the parameters. The proposed solution is obtained minimizing the estimator variance subject to some constraints on the estimator bias. The resulting optimal estimator is found to depend on the observation fourth-order moments that can be calculated analytically from the known signal model. Unfortunately, in most cases, the performance of this estimator is severely limited by the residual bias inherent to nonlinear estimation problems. To overcome this limitation, the second-order minimum variance unbiased estimator is deduced from the general solution by assuming accurate prior information on the vector of parameters. This small-error approximation is adopted to design iterative estimators or trackers. It is shown that the associated variance constitutes the lower bound for the variance of any unbiased estimator based on the sample covariance matrix. The paper formulation is then applied to track the angle-of-arrival (AoA) of multiple digitally-modulated sources by means of a uniform linear array. The optimal second-order tracker is compared with the classical maximum likelihood (ML) blind methods that are shown to be quadratic in the observed data as well. Simulations have confirmed that the discrete nature of the transmitted symbols can be exploited to improve considerably the discrimination of near sources in medium-to-high SNR scenarios.     

Some quick and efficient methods for bearing-only target motionanalysis

Batch processing estimation methods for a target's trajectory, assumed to be linear and uniform, based only on the observation of its bearings, are presented. First, the observer is assumed to have constant velocity, so that one can only estimate the target's motion, up to a multiplicative constant. This motion is parameterized by three bearings at three judiciously chosen times, and some simple, quick, yet highly efficient estimators for them are proposed. The case of an observer moving nonuniformly is then considered. A quadratic estimator that is similar to the pseudolinear estimator but does not have bias is introduced. For the case in which the observer's motion can be decomposed into a finite number of constant velocity segments, two sets of quasi-sufficient statistics that permit considerable saving in computation with no significant loss of efficiency are also introduced. Expressions for the covariance matrix of the estimators and for their Cramer-Rao bounds are provided     

基于误差预处理技术的水下被动目标跟踪方法

针对伪线性估计器有偏性问题进行了研究,指出产生有偏性的原因是由于将测量方位误差引入了测量方程。针对潜艇测量方位的变化特点,提出了一种对测量方位分段拟合的处理方法,将处理后的测量方位用于伪线性估计器。通过仿真实验,验证了该方法不但可以大大减小有偏性,而且提高了目标要素的解算速度。     

Bias minimizing filter design for gradient-based image registration

Gradient-based image registration techniques represent a very popular class of approaches to registering pairs or sets of images. As the name suggests, these methods rely on image gradients to perform the task of registration. Very often, little attention is paid to the filters used to estimate image gradients. In this paper, we explore the relationship between such gradient filters and their effect on overall estimation performance in registering translated images. We propose a methodology for designing filters based on image content that minimize the estimator bias inherent to gradient-based image registration. We show that minimizing such bias improves the overall estimator performance in terms of mean square error (MSE) for high signal-to-noise ratio (SNR) scenarios. Finally, we propose a technique for designing such optimal gradient filters in the context of iterative multiscale image registration and verify their further improved performance.     

Sample Bias Estimation for Cloud-Free Aerosol Effects Over Global Oceans

Satellite-based methods for estimating the top-of-atmosphere shortwave direct radiative effect (SWRE) either use the spatial distribution of aerosol optical thickness (AOT) coupled with radiative transfer calculations or combine the AOT with broadband radiative energy data sets such as the Clouds and the Earth's Radiant Energy System (CERES). The first approach typically utilizes the AOT at a spatial resolution of from the Moderate Resolution Imaging Spectroradiometer (MODIS), and the second method relies on the same AOT, but it is convolved within the CERES footprint and has spatial resolutions that are greater than . Therefore, the SWRE may vary as a result of this difference in spatial resolution that we call sample bias. We correct for this sample bias using the AOT reported at the MODIS and the CERES product levels coupled with the radiative efficiency (SWRE per-unit optical depth) for 13 regions over the ocean as a function of season between December 2003 and November 2004 and demonstrate that the sample biases are seasonally and spatially dependent. Overall, nearly 75% of the pixels over the global oceans require a sample bias adjustment of some form. However, the adjustment is large , which is less than 7% of the time, primarily during the spring and summer months, in association with large dust aerosol concentrations with large optical depth gradients. If sample biases are not accounted for, they will globally reduce the SWRE by an average of 30% (4.1 versus ), although regionally, the adjustment could be larger (). We argue that these bias corrections are robust and simpler to use when compared with methods that employ narrow- to broadband relationships.     

A wavelet-based joint estimator of the parameters of long-rangedependence

A joint estimator is presented for the two parameters that define the long-range dependence phenomenon in the simplest case. The estimator is based on the coefficients of a discrete wavelet decomposition, improving a wavelet-based estimator of the scaling parameter (Abry and Veitch 1998), as well as extending it to include the associated power parameter. An important feature is its conceptual and practical simplicity, consisting essentially in measuring the slope and the intercept of a linear fit after a discrete wavelet transform is performed, a very fast (O(n)) operation. Under well-justified technical idealizations the estimator is shown to be unbiased and of minimum or close to minimum variance for the scale parameter, and asymptotically unbiased and efficient for the second parameter. Through theoretical arguments and numerical simulations it is shown that in practice, even for small data sets, the bias is very small and the variance close to optimal for both parameters. Closed-form expressions are given for the covariance matrix of the estimator as a function of data length, and are shown by simulation to be very accurate even when the technical idealizations are not satisfied. Comparisons are made against two maximum-likelihood estimators. In terms of robustness and computational cost the wavelet estimator is found to be clearly superior and statistically its performance is comparable. We apply the tool to the analysis of Ethernet teletraffic data, completing an earlier study on the scaling parameter alone     

A pooling procedure for life-test data

A preliminary-test shrinkage estimator for the scale parameter in pooling data from extreme-value distributions is proposed. The optimum values of shrinkage coefficients for the preliminary-test shrinkage estimator are obtained based on a regret function. The optimum values of shrinkage coefficients and the critical values for the preliminary test are tabulated for several sample sizes. For a mean square error criterion of goodness of estimation, the preliminary-test shrinkage estimator is preferable to the usual preliminary-test estimator     

Crop Acreage Estimation Using a Landsat-Based Estimator as an Auxiliary Variable

The problem of improving upon the ground survey estimates of crop acreages by utilizing Landsat data is addressed. Three estimators, called regression, ratio, and stratified ratio, are studied. for bias and variance, and their relative efficiencies are compared. The approach is to formulate analytically the estimation problem that utilizes ground survey data, as collected by the U. S. Department of Agriculture ture, and Landsat data, which provide complete coverage for an area of interest, and then to conduct simulation studies. It is shown over a wide range of parametric conditions that the regression estimator is the most efficient unless there is a low correlation between the actual and estimated crop acreages in the sampled area segments, in which case the ratio and stratified ratio estimators are better. Furthermore, it is seen that the regression estimator is potentially biased due to estimating the regression coefficient from the training sample segments. Estimation of the variance of the regression estimator is also investigated. Two variance estimators are considered, the large sample variance estimator and an alternative estimator suggested by Cochran. The large sample estimate of variance is found to be biased and inferior to the Cochran estimate for small sample sizes.     

Uniformly Improving the CramÉr-Rao Bound and Maximum-Likelihood Estimation

An important aspect of estimation theory is characterizing the best achievable performance in a given estimation problem, as well as determining estimators that achieve the optimal performance. The traditional CramÉr–Rao type bounds provide benchmarks on the variance of any estimator of a deterministic parameter vector under suitable regularity conditions, while requiring a-priori specification of a desired bias gradient. In applications, it is often not clear how to choose the required bias. A direct measure of the estimation error that takes both the variance and the bias into account is the mean squared error (MSE), which is the sum of the variance and the squared-norm of the bias. Here, we develop bounds on the MSE in estimating a deterministic parameter vector$ bf x_0$over all bias vectors that are linear in$ bf x_0$, which includes the traditional unbiased estimation as a special case. In some settings, it is possible to minimize the MSE over all linear bias vectors. More generally, direct minimization is not possible since the optimal solution depends on the unknown$ bf x_0$. Nonetheless, we show that in many cases, we can find bias vectors that result in an MSE bound that is smaller than the CramÉr–Rao lower bound (CRLB) for all values of$ bf x_0$. Furthermore, we explicitly construct estimators that achieve these bounds in cases where an efficient estimator exists, by performing a simple linear transformation on the standard maximum likelihood (ML) estimator. This leads to estimators that result in a smaller MSE than the ML approach for all possible values of$ bf x_0$.     

Commentary: failure-rate functions for doubly-truncated random variables

Navarro and Ruiz (see ibid., vol.45, p.685-90, 1996) express the nonparametric maximum likelihood estimator (NPMLE) of the distribution of a failure-time random variable as a function of the NPMLE of generalized failure-rate functions. These generalized failure-rate functions are equal to the probability density functions of a doubly-truncated failure-time random variable at the endpoints of the truncating interval. Readers can infer from this paper that this simple estimator can be applied to a doubly-truncated sample of failure times. This commentary explains why that estimator does not apply to the general setting in which the observed failure times are doubly-truncated with subject-specific truncating intervals. A doubly-truncated sample of times to brain tumor progression illustrates the deviation of that estimator from the NPMLE for these data.     

Intrinsic dimensionality estimation based on manifold assumption

Dimensionality reduction is an important tool and has been widely used in many fields of data mining and machine learning. Intrinsic dimension of data sets is a key parameter for dimensionality reduction. In this paper, a new intrinsic dimension estimation method based on geometrical relationship between manifold intrinsic dimension and data neighborhood geodesic distances is presented. The estimator is derived by manifold sampling assumption. On a densely sampled manifold, the number of samples that fall into a ball is equal to the volume times the density of the ball. The radius of the ball is calculated by graph distance which is approximation of geodesic distance on manifold. Then the intrinsic dimension is estimated on each sample. Experiments conducted on synthetic and real world data set show that the performance of our new method is robust and comparable to other works.     

基于多元回归KNN的油田缺失数据填充方法

为了提高油田开采的安全性和科学性,油田中装有各型数据传感器,但数据缺失导致传感器采集数据可用性显著降低。针对油田传感器大比例数据缺失填充问题,提出了一种基于多元回归KNN的缺失数据填充方法。该方法首先基于KNN利用传感器数据空间相关性预测缺失值,其次基于多元回归利用传感器数据时间相关性预测缺失值,最后将时空相关性预测结果通过样本决定系数进行整合。分别采用标准数据集和油田传感器数据集进行性能对比实验,结果验证了该方法对缺失数据填充的有效性和准确性。     

Minimum variance in biased estimation: bounds and asymptotically optimal estimators

We develop a uniform Cramer-Rao lower bound (UCRLB) on the total variance of any estimator of an unknown vector of parameters, with bias gradient matrix whose norm is bounded by a constant. We consider both the Frobenius norm and the spectral norm of the bias gradient matrix, leading to two corresponding lower bounds. We then develop optimal estimators that achieve these lower bounds. In the case in which the measurements are related to the unknown parameters through a linear Gaussian model, Tikhonov regularization is shown to achieve the UCRLB when the Frobenius norm is considered, and the shrunken estimator is shown to achieve the UCRLB when the spectral norm is considered. For more general models, the penalized maximum likelihood (PML) estimator with a suitable penalizing function is shown to asymptotically achieve the UCRLB. To establish the asymptotic optimality of the PML estimator, we first develop the asymptotic mean and variance of the PML estimator for any choice of penalizing function satisfying certain regularity constraints and then derive a general condition on the penalizing function under which the resulting PML estimator asymptotically achieves the UCRLB. This then implies that from all linear and nonlinear estimators with bias gradient whose norm is bounded by a constant, the proposed PML estimator asymptotically results in the smallest possible variance.     

Data record-based criteria for the selection of an auxiliary vector estimator of the MMSE/MVDR filter

When the auxiliary vector (AV) filter generation algorithm utilizes sample average estimated input data statistics, it provides a sequence of estimates of the ideal minimum mean-square error or minimum-variance distortionless-response filter for the given signal processing/receiver design application. Evidently, early nonasymptotic elements of the sequence offer favorable bias/variance balance characteristics and outperform in mean-square filter estimation error the unbiased sample matrix inversion (SMI) estimator as well as the (constraint) least-mean square, recursive least-squares, "multistage nested Wiener filter", and diagonally-loaded SMI filter estimators. Selecting the most successful (in some appropriate sense) AV filter estimator in the sequence for a given data record is a critical problem that has not been addressed so far. We deal exactly with this problem and we propose two data-driven selection criteria. The first criterion minimizes the cross-validated sample average variance of the AV filter output and can be applied to general filter estimation problems; the second criterion maximizes the estimated J-divergence of the AV filter output conditional distributions and is tailored to binary phase-shift-keying-type detection problems.     

Statistics of the RSS estimation algorithm for Gaussian measurementnoise

The large and small sample properties of the reduced sufficient statistics (RSS) estimator of Kulhavy (1990, 1992) are derived for the nonlinear additive white Gaussian noise measurement model. The RSS algorithm recursively propagates a set of sufficient statistics for a mixture density that approximates the true posterior density of a parameter vector. The joint probability density function for the weighting coefficients of the mixture density is derived for the case of additive white Gaussian noise. Through integration of this density, the estimator bias and mean-squared error are determined. The results are applied to a scalar phase estimation problem in which the sample-averaged statistics are compared with those derived from numerical integration of the density function. The asymptotic bias and variance of the RSS estimator are also derived and compared with simulation results     

Robust weighted likelihood estimation of exponential parameters

The problem of estimating the parameter of an exponential distribution when a proportion of the observations are outliers is quite important to reliability applications. The method of weighted likelihood is applied to this problem, and a robust estimator of the exponential parameter is proposed. Interestingly, the proposed estimator is an /spl alpha/-trimmed mean type estimator. The large-sample robustness properties of the new estimator are examined. Further, a Monte Carlo simulation study is conducted showing that the proposed estimator is, under a wide range of contaminated exponential models, more efficient than the usual maximum likelihood estimator in the sense of having a smaller risk, a measure combining bias & variability. An application of the method to a data set on the failure times of throttles is presented.     

Unbiased timing-error estimation in the presence of nonidealinterpolation

We propose a method to eliminate the bias term present in the timing-error estimator employed in digital receivers where the input signal is sampled by a fixed clock which is not synchronized to the transmitter clock. This bias error results from the nonideal interpolation that precedes the timing-error estimator. We show that it can be derived as a function of the previously estimated symbol timings. An unbiased timing-error estimate can then be obtained by subtracting this bias term from the output of the timing-error detector. Simulation results are included to show the performance improvement realizable by employing this method