Improved estimation of clutter properties in speckled imagery

This paper's aim is to evaluate the effectiveness of bootstrap methods in improving estimation of clutter properties in speckled imagery. Estimation is performed by standard maximum likelihood methods. We show that estimators obtained this way can be quite biased in finite samples, and develop bias correction schemes using bootstrap resampling. In particular, we propose a bootstrapping scheme which is an adaptation of that proposed by Efron (J. Amer. Statist. Assoc. 85 (1990) 79). The proposed bootstrap does not require the quantity of interest to have closed form, as does Efron's original proposal. The adaptation we suggest is particularly important since the maximum likelihood estimator of interest does not have a closed form. We show that this particular bootstrapping scheme outperforms alternative forms of bias reduction mechanisms, thus delivering more accurate inference. We also consider interval estimation using bootstrap methods, and show that a particular parametric bootstrap-based confidence interval is typically more reliable than both the asymptotic confidence interval and other bootstrap-based confidence intervals. An application to real data is presented and discussed.     

Resampling methods for ranked set samples

When measuring units are expensive or time consuming, while ranking them can be done easily, it is known that ranked set sampling (RSS) is preferred to simple random sampling (SRS). Available results for RSS are developed under specific parametric assumptions or are asymptotic in nature, with few results available for finite size samples when the underlying distribution of the observed data is unknown. We investigate the use of resampling techniques to draw inferences on population characteristics. To obtain standard error and confidence interval estimates we discuss and compare three methods of resampling a given ranked set sample. Chen et al. (2004. Ranked Set Sampling: Theory and Applications. Springer, New York) suggest a natural method to obtain bootstrap samples from each row of a RSS. We prove that this method is consistent for a location estimator. We propose two other methods that are designed to obtain more stratified resamples from the given sample. Algorithms are provided for these methods. We recommend a method that obtains a bootstrap RSS from the observations. We prove several properties of this method, including consistency for a location parameter. We define two types of L-estimators for RSS and obtain expressions for their exact moments. We discuss an application to obtain confidence intervals for the Winsorized mean of a RSS.     

Estimation comparison on busy period for a controllable M/G/1 system with bicriterion policy

We propose an estimator for the expected busy period (denoted by β) of a controllable M/G/1 queueing system in which the server applies a bicriterion 〈p, N〉 policy during his idle period. Using this estimator, we construct new confidence intervals for β, which are based on five bootstrap methods; standard bootstrap (SB), percentile bootstrap (PB), bootstrap pivotal (BP), bias-corrected percentile bootstrap (BCPB), bias-corrected and accelerated bootstrap (BCa). A numerical simulation study is conducted in order to demonstrate performance of the proposed estimator $\hat{\beta }$ and bootstrap confidence intervals for β. From the simulation results, we show that $\hat{\beta }$ is a consistent estimator for β, which agrees with the theoretical results. In addition, we also investigate the accuracy of the five bootstrap confidence intervals by calculating the coverage percentage and the relative coverage (defined as the ratio of coverage percentage to average length of confidence interval). Detailed discussions of simulation results for three queueing models are presented.     

Creating non-parametric bootstrap samples using Poisson frequencies

This article describes how, in the high-level software packages used by non-statisticians, approximate non-parametric bootstrap samples can be created and analyzed without physically creating new data sets, or resorting to complex programming. The comparable performance of this shortcut method, which uses Poisson rather than multinomial frequencies for the numbers of copies of each observation, is demonstrated theoretically by evaluating the bootstrap variance in an example where the classic estimator of the sampling variance of the statistic of interest has a known closed form. For sample sizes of 50 or more, bootstrap standard errors obtained by this shortcut method exceeded those obtained by the standard version by less than 1%. The proposed method is also evaluated in two worked examples, involving statistics whose sampling distribution is more complex. The second of these is also used to illustrate when one can and cannot use non-parametric bootstrap samples.     

Improved interval estimation of long run response from a dynamic linear model: A highest density region approach

This paper proposes a new method of interval estimation for the long run response (or elasticity) parameter from a general linear dynamic model. We employ the bias-corrected bootstrap, in which small sample biases associated with the parameter estimators are adjusted in two stages of the bootstrap. As a means of bias-correction, we use alternative analytic and bootstrap methods. To take atypical properties of the long run elasticity estimator into account, the highest density region (HDR) method is adopted for the construction of confidence intervals. From an extensive Monte Carlo experiment, we found that the HDR confidence interval based on indirect analytic bias-correction performs better than other alternatives, providing tighter intervals with excellent coverage properties. Two case studies (demand for oil and demand for beef) illustrate the results of the Monte Carlo experiment with respect to the superior performance of the confidence interval based on indirect analytic bias-correction.     

Confidence intervals of the hazard rate function for discrete distributions using mixtures

The statistical models and methods for lifetime data mainly deal with continuous nonnegative lifetime distributions. However, discrete lifetimes arise in various common situations where either the clock time is not the best scale for measuring lifetime or the lifetime is measured discretely. In most settings involving lifetime data, the population under study is not homogenous. Mixture models, in particular mixtures of discrete distributions, provide a natural answer to this problem. Nonparametric mixtures of power series distributions are considered, as for instance nonparametric mixtures of Poisson laws or nonparametric mixtures of geometric laws. The mixing distribution is estimated by nonparametric maximum likelihood (NPML). Next, the NPML estimator is used to build estimates and confidence intervals for the hazard rate function of the discrete lifetime distribution. To improve the performance of the confidence intervals, a bootstrap procedure is considered where the estimated mixture is used for resampling. Various bootstrap confidence intervals are investigated and compared to the confidence intervals obtained directly from the NPML estimates.     

Model based bootstrap methods for interval censored data

The performance of model based bootstrap methods for constructing point-wise confidence intervals around the survival function with interval censored data is investigated. It is shown that bootstrapping from the nonparametric maximum likelihood estimator of the survival function is inconsistent for the current status model. A model based smoothed bootstrap procedure is proposed and proved to be consistent. In fact, a general framework for proving the consistency of any model based bootstrap scheme in the current status model is established. In addition, simulation studies are conducted to illustrate the (in)-consistency of different bootstrap methods in mixed case interval censoring. The conclusions in the interval censoring model would extend more generally to estimators in regression models that exhibit non-standard rates of convergence.     

Weighted average least squares estimation with nonspherical disturbances and an application to the Hong Kong housing market

The recently proposed ‘weighted average least squares’ (WALS) estimator is a Bayesian combination of frequentist estimators. It has been shown that the WALS estimator possesses major advantages over standard Bayesian model averaging (BMA) estimators: the WALS estimator has bounded risk, allows a coherent treatment of ignorance and its computational effort is negligible. However, the sampling properties of the WALS estimator as compared to BMA estimators are heretofore unexamined. The WALS theory is further extended to allow for nonspherical disturbances, and the estimator is illustrated with data from the Hong Kong real estate market. Monte Carlo evidence shows that the WALS estimator performs significantly better than standard BMA and pretest alternatives.     

Variance Estimation in Censored Quantile Regression via Induced Smoothing

Statistical inference in censored quantile regression is challenging, partly due to the unsmoothness of the quantile score function. A new procedure is developed to estimate the variance of the Bang and Tsiatis inverse-censoring-probability weighted estimator for censored quantile regression by employing the idea of induced smoothing. The proposed variance estimator is shown to be asymptotically consistent. In addition, a numerical study suggests that the proposed procedure performs well in finite samples, and it is computationally more efficient than the commonly used bootstrap method.     

Half-life estimation based on the bias-corrected bootstrap: A highest density region approach

The half-life is defined as the number of periods required for the impulse response to a unit shock to a time series to dissipate by half. It is widely used as a measure of persistence, especially in international economics to quantify the degree of mean-reversion of the deviation from an international parity condition. Several studies have proposed bias-corrected point and interval estimation methods. However, they have found that the confidence intervals are rather uninformative with their upper bound being either extremely large or infinite. This is largely due to the distribution of the half-life estimator being heavily skewed and multi-modal. A bias-corrected bootstrap procedure for the estimation of half-life is proposed, adopting the highest density region (HDR) approach to point and interval estimation. The Monte Carlo simulation results reveal that the bias-corrected bootstrap HDR method provides an accurate point estimator, as well as tight confidence intervals with superior coverage properties to those of its alternatives. As an application, the proposed method is employed for half-life estimation of the real exchange rates of 17 industrialized countries. The results indicate much faster rates of mean-reversion than those reported in previous studies.     

Bootstrap prediction intervals for autoregressive time series

The calculation of interval forecasts for highly persistent autoregressive (AR) time series based on the bootstrap is considered. Three methods are considered for countering the small-sample bias of least-squares estimation for processes which have roots close to the unit circle: a bootstrap bias-corrected OLS estimator; the use of the Roy-Fuller estimator in place of OLS; and the use of the Andrews-Chen estimator in place of OLS. All three methods of bias correction yield superior results to the bootstrap in the absence of bias correction. Of the three correction methods, the bootstrap prediction intervals based on the Roy-Fuller estimator are generally superior to the other two. The small-sample performance of bootstrap prediction intervals based on the Roy-Fuller estimator are investigated when the order of the AR model is unknown, and has to be determined using an information criterion.     

Weighted bootstrap for neural model selection

This article proposes a weighted bootstrap procedure, which is an efficient bootstrap technique for neural model selection. Our primary interest in reducing computer effort is to not resample (in the original bootstrap procedure) uniformly from the original sample, but to modify this distribution in order to obtain variance reduction. The performance of the weighted bootstrap is demonstrated on two artificial data sets and one real dataset. Experimental results show that the weighted bootstrap procedure permits an approximately 2 to 1 reduction in replication size.     

Standard errors for bagged and random forest estimators

Bagging and random forests are widely used ensemble methods. Each forms an ensemble of models by randomly perturbing the fitting of a base learner. The standard errors estimation of the resultant regression function is considered. Three estimators are discussed. One, based on the jackknife, is applicable to bagged estimators and can be computed using the bagged ensemble. The two other estimators target the bootstrap standard error estimator, and require fitting multiple ensemble estimators, one for each bootstrap sample. It is shown that these bootstrap ensemble sizes can be small, which reduces the computation involved in forming the estimator. The estimators are studied using both simulated and real data.     

Optimal convex error estimators for classification

A cross-validation error estimator is obtained by repeatedly leaving out some data points, deriving classifiers on the remaining points, computing errors for these classifiers on the left-out points, and then averaging these errors. The 0.632 bootstrap estimator is obtained by averaging the errors of classifiers designed from points drawn with replacement and then taking a convex combination of this “zero bootstrap” error with the resubstitution error for the designed classifier. This gives a convex combination of the low-biased resubstitution and the high-biased zero bootstrap. Another convex error estimator suggested in the literature is the unweighted average of resubstitution and cross-validation. This paper treats the following question: Given a feature-label distribution and classification rule, what is the optimal convex combination of two error estimators, i.e. what are the optimal weights for the convex combination. This problem is considered by finding the weights to minimize the MSE of a convex estimator. It also considers optimality under the constraint that the resulting estimator be unbiased. Owing to the large amount of results coming from the various feature-label models and error estimators, a portion of the results are presented herein and the main body of results appears on a companion website. In the tabulated results, each table treats the classification rules considered for the model, various Bayes errors, and various sample sizes. Each table includes the optimal weights, mean errors and standard deviations for the relevant error measures, and the MSE and MAE for the optimal convex estimator. Many observations can be made by considering the full set of experiments. Some general trends are outlined in the paper. The general conclusion is that optimizing the weights of a convex estimator can provide substantial improvement, depending on the classification rule, data model, sample size and component estimators. Optimal convex bootstrap estimators are applied to feature-set ranking to illustrate their potential advantage over non-optimized convex estimators.     

Local bootstrap approaches for fractional differential parameter estimation in ARFIMA models

In this paper we investigate bootstrap techniques applied to the estimation of the fractional differential parameter in ARFIMA models, d. The novelty is the focus on the local bootstrap of the periodogram function. The approach is then applied to three different semiparametric estimators of d, known from the literature, based upon the periodogram function. By means of an extensive set of simulation experiments, the bias and mean square errors are quantified for each estimator and the efficacy of the local bootstrap is stated in terms of low bias, short confidence intervals, and low CPU times. Finally, a real data set is analyzed to demonstrate that the methodology may be quite effective in solving real problems.     

On the efficiency of bootstrap method into the analysis contingency table

The bootstrap method is a computer intensive statistical method that is widely used in performing nonparametric inference. Categorical data analysis, in particular the analysis of contingency tables, is commonly used in applied field. This work considers nonparametric bootstrap tests for the analysis of contingency tables. There are only a few research papers which exploit this field. The p-values of tests in contingency tables are discrete and should be uniformly distributed under the null hypothesis. The results of this article show that corresponding bootstrap versions work better than the standard tests. Properties of the proposed tests are illustrated and discussed using Monte Carlo simulations. This article concludes with an analytical example that examines the performance of the proposed tests and the confidence interval of the association coefficient.     

A Computationally Efficient, Consistent Bootstrap for Inference with Non-parametric DEA Estimators

We develop a tractable, consistent bootstrap algorithm for inference about Farrell?CDebreu efficiency scores estimated by non-parametric data envelopment analysis (DEA) methods. The algorithm allows for very general situations where the distribution of the inefficiencies in the input-output space may be heterogeneous. Computational efficiency and tractability are achieved by avoiding the complex double-smoothing procedure in the algorithm proposed by Kneip et al. (Econometric Theory 24:1663?C1697, 2008). In particular, we avoid technical difficulties in the earlier algorithm associated with smoothed estimates of a density with unknown, nonlinear, multivariate bounded support requiring complicated reflection methods. The new procedure described here is relatively simple and easy to implement: for particular values of a pair of smoothing parameters, the computational complexity is the same as the (inconsistent) naive bootstrap. The resulting computational speed allows the bootstrap to be iterated in order to optimize the smoothing parameters. From a practical viewpoint, only standard packages for computing DEA efficiency estimates, i.e., solving linear problems, are required for implementation. The performance of the method in finite samples is illustrated through some simulated examples.     

Interval estimation of binomial proportions based on weighted Polya posterior

Recently the interval estimation of binomial proportions is revisited in various literatures. This is mainly due to the erratic behavior of the coverage probability of the well-known Wald confidence interval. Various alternatives have been proposed. Among them, Agresti-Coull confidence interval has been recommended by Brown et al. [2001. Interval estimation for a binomial proportion. Statist. Sci. 16, 101-133] with other confidence intervals such as the Wilson interval and the equal tailed interval resulting from the natural noninformative Jefferys prior for a binomial proportion. However, it seems that Agresti-Coull interval is little bit wider than necessary when sample size is small, say n?40. In this note, an interval estimator is developed using weighted Polya posterior. It is shown that the confidence interval based on the weighted Polya posterior is essentially the Agresti-Coull interval with some improved features.     

A Fast Procedure for Calculating Importance Weights in Bootstrap Sampling

Importance sampling is an efficient strategy for reducing the variance of certain bootstrap estimates. It has found wide applications in bootstrap quantile estimation, proportional hazards regression, bootstrap confidence interval estimation, and other problems. Although estimation of the optimal sampling weights is a special case of convex programming, generic optimization methods are frustratingly slow on problems with large numbers of observations. For instance, interior point and adaptive barrier methods must cope with forming, storing, and inverting the Hessian of the objective function. In this paper, we present an efficient procedure for calculating the optimal importance weights and compare its performance to standard optimization methods on a representative data set. The procedure combines several potent ideas for large-scale optimization.