Asymptotic properties of least squares estimation with fuzzy observations

This paper deals with the asymptotic properties of the least squares estimators for fuzzy linear regression models with fuzzy triangular input-output and random error terms. The asymptotic normality and strong consistency of the fuzzy least squares estimator (FLSE) are investigated; a confidence region based on a class of FLSEs is proposed; the asymptotic relative efficiency of FLSEs with respect to the crisp least squares estimators is also provided and a numerical example is given. Some simulation results are also presented to illustrate the behavior of FLSEs.     

Estimating residual variance in random forest regression

Random forest, a data-mining technique which uses multiple classification or regression trees, is a popular algorithm used for prediction. Inference and goodness-of-fit assessment, however, may require an estimator of variability; in many applications the residual variance is of primary interest. This paper proposes two estimators of residual variance for random forest regression that take advantage of byproducts of the algorithm. The first estimator is based on the residual sum of squares from a random forest fit and uses a bootstrap bias correction. The second estimator is a difference-based estimator that uses proximity measures as weights. The estimators are evaluated through Monte Carlo simulations. Applications of the methods to the problem of assessing the relative variability of males and females on cognitive and achievement tests are discussed, and the methods are applied to estimate the residual variance in test scores for male and female students on the mathematics portion of the 2007 Arizona Instrument to Measure Standards.     

Robust variance estimation for random effects meta-analysis

In random effects meta-analysis, an overall effect is estimated using a weighted mean, with weights based on estimated marginal variances. The variance of the overall effect is often estimated using the inverse of the sum of the estimated weights, and inference about the overall effect is typically conducted using this ‘usual’ variance estimator, which is not robust to errors in the estimated marginal variances. In this paper, robust estimation for the asymptotic variance of a weighted overall effect estimate is explored by considering a robust variance estimator in comparison with the usual variance estimator and another less frequently used estimator, a weighted version of the sample variance. Three illustrative examples are presented to demonstrate and compare the three estimation methods. Furthermore, a simulation study is conducted to assess the robustness of the three variance estimators using estimated weights. The simulation results show that the robust variance estimator and the weighted sample variance estimator both estimate the variance of an overall effect more accurately than the usual variance estimator when the weights are imprecise due to the use of estimated marginal variances, as is typically the case in practice.Therefore, we argue that inference about an overall effect should be based on the robust variance estimator or the weighted sample variance, which provide protection against the practice of using estimated weights in meta-analytical inference.     

Computational methods for case-cohort studies

Computational methods, which can be implemented using standard Cox regression software, are given for fitting “exact” pseudolikehood estimates and robust and asymptotic variance estimators from case-cohort data. These methods are based on the computational approach of Therneau and Li [1999. Computing the Cox model for case cohort designs. Lifetime Data Anal. 5, 99-112] but will be less subject to small sample bias. Further, it is shown how to accommodate time-dependent covariates and estimate absolute risk. Extensions to stratified case-cohort sampled data are also provided. The methods are illustrated in analyses of case-cohort samples from a study of radiation exposure from fluoroscopy and breast cancer using SAS software.     

Asymptotically efficient estimation of the conditional expected shortfall

A procedure for efficient estimation of the trimmed mean of a random variable conditional on a set of covariates is proposed. For concreteness, the focus is on a financial application where the trimmed mean of interest corresponds to the conditional expected shortfall, which is known to be a coherent risk measure. The proposed class of estimators is based on representing the estimator as an integral of the conditional quantile function. Relative to the simple analog estimator that weights all conditional quantiles equally, asymptotic efficiency gains may be attained by giving different weights to the different conditional quantiles while penalizing excessive departures from uniform weighting. The approach presented here allows for either parametric or nonparametric modeling of the conditional quantiles and the weights, but is essentially nonparametric in spirit. The asymptotic properties of the proposed class of estimators are established. Their finite sample properties are illustrated through a set of Monte Carlo experiments and an empirical application¹.     

Consistency of pseudolikelihood estimation of fully visible Boltzmann machines

A Boltzmann machine is a classic model of neural computation, and a number of methods have been proposed for its estimation. Most methods are plagued by either very slow convergence or asymptotic bias in the resulting estimates. Here we consider estimation in the basic case of fully visible Boltzmann machines. We show that the old principle of pseudolikelihood estimation provides an estimator that is computationally very simple yet statistically consistent.     

Ergodicity of hidden Markov models

In this paper we study ergodic properties of hidden Markov models with a generalized observation structure. In particular sufficient conditions for the existence of a unique invariant measure for the pair filter-observation are given. Furthermore, necessary and sufficient conditions for the existence of a unique invariant measure of the triple state-observation-filter are provided in terms of asymptotic stability in probability of incorrectly initialized filters. We also study the asymptotic properties of the filter and of the state estimator based on the observations as well as on the knowledge of the initial state. Their connection with minimal and maximal invariant measures is also studied. Work partially supported by grants MIUR-PRIN 2001, PBZ KBN 016/P03/99 and IMPAN-BC Centre of Excellence     

On the performance of bias-reduction techniques for variance estimation in approximate Bayesian bootstrap imputation

Multiply imputed data sets can be created with the approximate Bayesian bootstrap (ABB) approach under the assumption of ignorable nonresponse. The theoretical development and inferential validity are predicated upon asymptotic properties; and biases are known to occur in small-to-moderate samples. There have been attempts to reduce the finite-sample bias for the multiple imputation variance estimator. In this note, we present an empirical study for evaluating the comparative performance of the two proposed bias-correction techniques and their impact on precision. The results suggest that to varying degrees, bias improvements are outweighed by efficiency losses for the variance estimator. We argue that the original ABB has better small-sample properties than the modified versions in terms of the integrated behavior of accuracy and precision, as measured by the root mean-square error.     

Bootstrap prediction intervals for autoregressive time series

The calculation of interval forecasts for highly persistent autoregressive (AR) time series based on the bootstrap is considered. Three methods are considered for countering the small-sample bias of least-squares estimation for processes which have roots close to the unit circle: a bootstrap bias-corrected OLS estimator; the use of the Roy-Fuller estimator in place of OLS; and the use of the Andrews-Chen estimator in place of OLS. All three methods of bias correction yield superior results to the bootstrap in the absence of bias correction. Of the three correction methods, the bootstrap prediction intervals based on the Roy-Fuller estimator are generally superior to the other two. The small-sample performance of bootstrap prediction intervals based on the Roy-Fuller estimator are investigated when the order of the AR model is unknown, and has to be determined using an information criterion.     

Monte Carlo methods for derivatives of options with discontinuous payoffs

Various Monte Carlo methods have been proposed to estimate the derivatives of contingent claims prices. The Monte Carlo approximate likelihood ratio estimator is studied. Recent convergence results are extended in order to show that the Monte Carlo approximate likelihood ratio derivative estimator is asymptotically equivalent, up to a second-order bias component, to an estimator based on a covariation approximation, the Monte Carlo Covariation estimator. Both converge slower than the Monte Carlo Malliavin derivative estimators. Theoretical convergence results are illustrated in a numerical experiment dealing with the risk management of digital options in a CEV model.     

Second-Order Performance Analysis and Unbiased Estimation for the Fitting of Concentric Circles

A number of methods, both algebraic and iterative, have been developed recently for the fitting of concentric circles. Previous studies focus on first-order analysis for performance evaluation, which is appropriate only when the observation noise is small so that the bias is insignificant compared to variance. Further studies indicate that the first-order analysis does not appear sufficient in explaining and predicting the performance of an estimator for the fitting problem, especially when the noise level becomes significant. This paper extends the previous study to perform the second-order analysis and evaluate the estimation bias of several concentric circle estimators. The second-order analysis exposes important characteristics of the estimators that cannot be seen from the first-order studies. The insights gained in the theoretical study have led to the development of a new estimator that is unbiased and performs best among the algebraic solutions. An adjusted maximum likelihood estimator is also proposed that can yield an unbiased estimate while maintaining the KCR bound performance.     

Statistical properties of error estimators in performance assessment of recognition systems

The problem of estimating the error probability of a given classification system is considered. Statistical properties of the empirical error count (C) and the average conditional error (R) estimators are studied. It is shown that in the large sample case the R estimator is unbiased and its variance is less than that of the C estimator. In contrast to conventional methods of Bayes error estimation the unbiasedness of the R estimator for a given classifier can be obtained only at the price of an additional set of classified samples. On small test sets the R estimator may be subject to a pessimistic bias caused by the averaging phenomenon characterizing the functioning of conditional error estimators.     

Improved Liu-type estimator in partial linear model

In this article, a Liu-type estimation is proposed for the vector-parameter in a partial linear model. This new estimator can be regarded as generalization of the restricted least-squares estimator, the restricted ridge estimator and the restricted Liu estimator. We also obtain the asymptotic distributional bias and risk of these estimators and we also discuss some properties of the new estimator. The selection of the tuning parameter in the proposed estimator is also presented. Finally, a simulation study is presented to explain the performance of the new estimator.     

Semiparametrically weighted robust estimation of regression models

A class of two-step robust regression estimators that achieve a high relative efficiency for data from light-tailed, heavy-tailed, and contaminated distributions irrespective of the sample size is proposed and studied. In particular, the least weighted squares (LWS) estimator is combined with data-adaptive weights, which are determined from the empirical distribution or quantile functions of regression residuals obtained from an initial robust fit. Just like many existing two-step robust methods, the LWS estimator with the proposed weights preserves robust properties of the initial robust estimate. However, contrary to the existing methods and despite the data-dependent weights, the first-order asymptotic behavior of LWS is fully independent of the initial estimate under mild conditions. Moreover, the proposed estimation method is asymptotically efficient if errors are normally distributed. A simulation study documents these theoretical properties in finite samples; in particular, the relative efficiency of LWS with the proposed weighting schemes can reach 85%-100% in samples of several tens of observations under various distributional models.     

Applications of the jackknife procedure in ridge regression

Three aspects of the application of the jackknife technique to ridge regression are considered, viz. as a bias estimator, as a variance estimator, and as an indicator of observations influence on parameter estimates. The ridge parameter is considered non-stochastic. The jackknifed ridge estimator is found to be a ridge estimator with a smaller value on the ridge parameter. Hence it has a smaller bias but a larger variance than the ridge estimator. The variance estimator is expected to be robust against heteroscedastic error variance as well as against outliers. A measure of observations influence on the estimates of regression parameters is proposed.     

Direct and two-step methods for closed-loop identification: a comparison of asymptotic and finite data set performance

The asymptotic and finite data behavior of some closed-loop identification methods are investigated. It is shown that, when the output power is limited, closed-loop identification can generally identify models with smaller variance than open-loop identification. Several variations on some two-step identification methods are compared with the direct identification method. High order FIR models are used as process models to avoid bias issues arising from inadequate model structures for the processes. Comparisons are, therefore, made based on the variance of the identified process models both for asymptotic situations and for finite data sets. Process model bias resulting from improper selection of the noise and sensitivity function models is also investigated. In this context, the results support the use of direct identification methods on closed-loop data.     

Some facts about the choice of the weighting matrices in Larimore type of subspace algorithms

In this paper the effect of some weighting matrices on the asymptotic variance of the estimates of linear discrete time state space systems estimated using subspace methods is investigated. The analysis deals with systems with white or without observed inputs and refers to the Larimore type of subspace procedures. The main result expresses the asymptotic variance of the system matrix estimates in canonical form as a function of some of the user choices, clarifying the question on how to choose them optimally. It is shown, that the CCA weighting scheme leads to optimal accuracy. The expressions for the asymptotic variance can be implemented more efficiently as compared to the ones previously published.     

Firing variability is higher than deduced from the empirical coefficient of variation

A convenient and often used summary measure to quantify the firing variability in neurons is the coefficient of variation (CV), defined as the standard deviation divided by the mean. It is therefore important to find an estimator that gives reliable results from experimental data, that is, the estimator should be unbiased and have low estimation variance. When the CV is evaluated in the standard way (empirical standard deviation of interspike intervals divided by their average), then the estimator is biased, underestimating the true CV, especially if the distribution of the interspike intervals is positively skewed. Moreover, the estimator has a large variance for commonly used distributions. The aim of this letter is to quantify the bias and propose alternative estimation methods. If the distribution is assumed known or can be determined from data, parametric estimators are proposed, which not only remove the bias but also decrease the estimation errors. If no distribution is assumed and the data are very positively skewed, we propose to correct the standard estimator. When defining the corrected estimator, we simply use that it is more stable to work on the log scale for positively skewed distributions. The estimators are evaluated through simulations and applied to experimental data from olfactory receptor neurons in rats.     

Additive estimators for probabilities of correct classification

Several methods for estimating a sample-based discriminant's probability of correct classification are compared with respect to bias, variance, robustness, and computation cost. “Smooth” modification of the counting estimator, or sample success proportion, is recommended to reduce bias and variance while retaining robustness. Also the “bootstrap” method of Efron⁽⁸⁾ can approximately correct an additive estimator's bias using an ancillary computer simulation. In contrast, bias reduction achieved by the popular “leave-one-out” modification of counting method is vitiated by corresponding increase in variance.     

自校正解耦信息融合Wiener状态估值器

对含未知噪声方差阵的多传感器系统,用现代时间序列分析方法.基于滑动平均(MA)新息模型的在线辨识和求解相关函数矩阵方程组,可得到估计噪声方差阵估值器,进而在按分量标量加权线性最小方差最优信息融合则下,提出了自校正解耦信息融合Wiener状态估值器.它的精度比每个局部自校正Wiener状态估值器精度高.它实现了状态分量的解耦局部Wiener估值器和解耦融合Wiener估值器.证明了它的收敛性,即若MA新息模型参数估计是一致的,则它将收敛于噪声统计已知时的最优解耦信息融合Wiener状态估值器,因而它具有渐近最优性.一个带3传感器的目标跟踪系统的仿真例子说明了其有效性.