Robust estimating equations and bias correction of correlation parameters for longitudinal data

The estimation of correlation parameters has received attention for both its own interest and improvement of the estimation efficiency of mean parameters by the generalized estimating equations (GEE) approach. Many of the well-established methods for the estimation of correlation parameters can be constructed under the GEE framework which is, however, sensitive to outliers. In this paper, we consider two ways of constructing robust estimating equations for achieving robust estimation of the correlation parameters. Furthermore, the estimators of the correlation parameters from the robustified GEE may be still biased as the expectation of the estimating equation is biased from zero when the underlying distribution is not symmetric. Therefore, bias-corrected robust estimators of correlation parameters are proposed. The performance of the proposed methods are investigated by simulation. The results show that the proposed robust and bias-corrected robust estimators can reduce the bias successfully. Two real data sets are analyzed for illustration.     

Second-Order Performance Analysis and Unbiased Estimation for the Fitting of Concentric Circles

A number of methods, both algebraic and iterative, have been developed recently for the fitting of concentric circles. Previous studies focus on first-order analysis for performance evaluation, which is appropriate only when the observation noise is small so that the bias is insignificant compared to variance. Further studies indicate that the first-order analysis does not appear sufficient in explaining and predicting the performance of an estimator for the fitting problem, especially when the noise level becomes significant. This paper extends the previous study to perform the second-order analysis and evaluate the estimation bias of several concentric circle estimators. The second-order analysis exposes important characteristics of the estimators that cannot be seen from the first-order studies. The insights gained in the theoretical study have led to the development of a new estimator that is unbiased and performs best among the algebraic solutions. An adjusted maximum likelihood estimator is also proposed that can yield an unbiased estimate while maintaining the KCR bound performance.     

Maximum correntropy-based pseudolinear Kalman filter for passive bearings-only target tracking

This paper proposes a new approach for solving the bearings-only target tracking (BoT) problem by introducing a maximum correntropy criterion to the pseudolinear Kalman filter (PLKF). PLKF has been a popular choice for solving BoT problems owing to the reduced computational complexity. However, the coupling between the measurement vector and pseudolinear noise causes bias in PLKF. To address this issue, a bias-compensated PLKF (BC-PLKF) under the assumption of Gaussian noisewas formulated. However, this assumptionmay not be valid in most practical cases. Therefore, a bias-compensated PLKF with maximum correntropy criterion is introduced, resulting in two new filters: maximum correntropy pseudolinear Kalman filter (MC-PLKF) and maximum correntropy bias-compensated pseudolinear Kalman filter (MC-BC-PLKF). To demonstrate the performance of the proposed estimators, a comparative analysis assuming large outliers in the process and measurement model of 2D BoT is conducted. These large outliers are modeled as non-Gaussian noises with diverse noise distributions that combine Gaussian and Laplacian noises. The simulation results are validated using root mean square error (RMSE), average RMSE (ARMSE), percentage of track loss and bias norm. Compared to PLKF and BC-PLKF, all the proposed maximum correntropy-based filters (MC-PLKF and MC-BC-PLKF) performed with superior estimation accuracy.     

Bias reduction in kernel binary regression

We consider Nadaraya-Watson type estimators for binary regression functions. We propose a method for improving the performance of such estimators by employing bias reduction techniques when estimating the constituent probability densities. Direct substitution of separately optimized density estimates into the regression function formula generates disappointing results in practice. However, adjusting the global smoothing parameter to optimize a performance criterion for the binary regression function itself is more promising. We focus on an implementation of this approach which uses a variable kernel technique to provide reduced bias density estimates, and where the global bandwidth is selected by an appropriately tailored leave-one-out (cross-validation) method. Theory and numerical experiments show that this form of bias reduction improves performance substantially when the underlying regression function is highly non-linear but is not beneficial when the underlying regression function is almost linear in form.     

Estimation of ordinal pattern probabilities in Gaussian processes with stationary increments

Analyzing the probabilities of ordinal patterns is a recent approach to quantifying the complexity of time series and detecting structural changes in the underlying dynamics. The present paper investigates statistical properties of estimators of ordinal pattern probabilities in discrete-time Gaussian processes with stationary increments. It shows that better estimators than the sample frequencies are available and establishes sufficient conditions under which these estimators are consistent and asymptotically normal. The results are applied to derive properties of the Zero Crossing estimator for the Hurst parameter in fractional Brownian motion. In a simulation study, the performance of the Zero Crossing estimator is compared to that of a similar “metric” estimator; furthermore, the Zero Crossing estimator is applied to the analysis of Nile River data.     

Locally and globally robust Penalized Trimmed Squares regression

Multiple outliers are frequently encountered in regression models used in business, economics, engineers and applied studies. The ordinary least squares (OLS) estimator fails even in the presence of a single outlying observation. To overcome this problem, a class of high breakdown robust estimators (insensitive to outliers up to 50% of the data sample) has been introduced as an alternative to the least squares regression. Among them the Penalized Trimmed Squares (PTS) is a reasonable high breakdown estimator. This estimator is defined by the minimization of an objective function where penalty cost for deleting an outlier is added, which serves as an upper bound on the residual error for any feasible regression line. Since the PTS does not require presetting the number of outliers to delete from the data set, it has better efficiency with respect to other estimators. However, small outliers remain influential causing bias to the regression line. In this work we present a new class of regression estimates called generalized PTS (GPTS). The new GPTS estimator is defined as the PTS but with penalties suitable for bounding the influence function of all observations. We show with some numerical examples and a Monte Carlo simulation study that the generalized PTS estimate has very good performance for both robust and efficiency properties.     

Bias from misspecification of the component variances in a normal mixture

Bias in parameter estimates can be substantial when heteroscedastic normal mixtures are misspecified as homoscedastic normal mixtures, and vice versa. We show through simulations that the maximum likelihood estimators under the false assumption of equal variances are inconsistent and bias in parameter estimates is appreciable and even substantial when the mixture components are not well-separated. Finite sample bias in parameter estimates is close to the asymptotic bias even for a sample size of 200 or less. When homoscedastic normal mixtures are misspecified as heteroscedastic normal mixtures, the maximum likelihood estimators are consistent. However, the maximum likelihood estimators under a correctly specified homoscedastic mixture model converge to the true parameter values faster than those under a misspecified heteroscedastic mixture model. The bias of the maximum likelihood estimators is less dependent on the lower bound imposed on the component variances to ensure that the likelihood is bounded under the false assumption of unequal variances when the sample size is 500 or more and the component distributions are well-separated. An example is given to demonstrate the effects of a misspecification of the component variances on estimates of the prevalence of hypertension using normal mixtures.     

Robust blind source separation by beta divergence

Blind source separation is aimed at recovering original independent signals when their linear mixtures are observed. Various methods for estimating a recovering matrix have been proposed and applied to data in many fields, such as biological signal processing, communication engineering, and financial market data analysis. One problem these methods have is that they are often too sensitive to outliers, and the existence of a few outliers might change the estimate drastically. In this article, we propose a robust method of blind source separation based on the beta divergence. Shift parameters are explicitly included in our model instead of the conventional way which assumes that original signals have zero mean. The estimator gives smaller weights to possible outliers so that their influence on the estimate is weakened. Simulation results show that the proposed estimator significantly improves the performance over the existing methods when outliers exist; it keeps equal performance otherwise.     

Bias in robust estimation caused by discontinuities and multiplestructures

When fitting models to data containing multiple structures, such as when fitting surface patches to data taken from a neighborhood that includes a range discontinuity, robust estimators must tolerate both gross outliers and pseudo outliers. Pseudo outliers are outliers to the structure of interest, but inliers to a different structure. They differ from gross outliers because of their coherence. Such data occurs frequently in computer vision problems, including motion estimation, model fitting, and range data analysis. The focus in this paper is the problem of fitting surfaces near discontinuities in range data. To characterize the performance of least median of the squares, least trimmed squares, M-estimators, Hough transforms, RANSAC, and MINPRAN on this type of data, the “pseudo outlier bias” metric is developed using techniques from the robust statistics literature, and it is used to study the error in robust fits caused by distributions modeling various types of discontinuities. The results show each robust estimator to be biased at small, but substantial, discontinuities. They also show the circumstances under which different estimators are most effective. Most importantly, the results imply present estimators should be used with care, and new estimators should be developed     

Robust regression with asymmetric heavy-tail noise distributions

In the presence of a heavy-tail noise distribution, regression becomes much more difficult. Traditional robust regression methods assume that the noise distribution is symmetric, and they downweight the influence of so-called outliers. When the noise distribution is asymmetric, these methods yield biased regression estimators. Motivated by data-mining problems for the insurance industry, we propose a new approach to robust regression tailored to deal with asymmetric noise distribution. The main idea is to learn most of the parameters of the model using conditional quantile estimators (which are biased but robust estimators of the regression) and to learn a few remaining parameters to combine and correct these estimators, to minimize the average squared error in an unbiased way. Theoretical analysis and experiments show the clear advantages of the approach. Results are on artificial data as well as insurance data, using both linear and neural network predictors.     

Robust exponential smoothing of multivariate time series

Multivariate time series may contain outliers of different types. In the presence of such outliers, applying standard multivariate time series techniques becomes unreliable. A robust version of multivariate exponential smoothing is proposed. The method is affine equivariant, and involves the selection of a smoothing parameter matrix by minimizing a robust loss function. It is shown that the robust method results in much better forecasts than the classic approach in the presence of outliers, and performs similarly when the data contain no outliers. Moreover, the robust procedure yields an estimator of the smoothing parameter less subject to downward bias. As a byproduct, a cleaned version of the time series is obtained, as is illustrated by means of a real data example.     

Asymptotically efficient estimators for the fittings of coupled circles and ellipses

Fitting a pair of coupled geometric objects to a number of coordinate points is a challenging and important problem in many applications including coordinate metrology, petroleum engineering and image processing. This paper derives two asymptotically efficient estimators, one for concentric circles fitting and the other for concentric ellipses fitting, based on the weighted equation error formulation and non-linear parameter transformation. The Kanatani–Cramér–Rao (KCR) lower bounds for the parameter estimates of the concentric circles and concentric ellipses under zero-mean Gaussian noise are provided to serve as the performance benchmark. Small-noise analysis shows that the proposed estimators reach the KCR lower bound performance asymptotically. The accuracy of the proposed estimators is corroborated by experiments with synthetic data and realistic images.     

Alpha稳定分布噪声下改进的频域预滤波方法

针对α琢稳定分布噪声和谐波信号在频域均表现为异常值这一特性,提出了一种新的频域预滤波估计方法。通过分数低阶共变将信号转换到频域,在频域利用自适应加权Myriad滤波器滤除尖锐脉冲,提取稳定分布噪声的共变谱,将稳定分布有色噪声转化为稳定白噪声,然后利用基于分数阶共变的Music方法估计信号的共变谱。仿真结果表明,在不需要知道稳定噪声任何统计特性的情况下,该方法在1<α<2的非线性度量空间,该算法取得了理想的效果。     

Gaussian-weighted Jensen–Shannon divergence as a robust fitness function for multi-model fitting

Model fitting is a fundamental component in computer vision for salient data selection, feature extraction and data parameterization. Conventional approaches such as the RANSAC family show limitations when dealing with data containing multiple models, high percentage of outliers or sample selection bias, commonly encountered in computer vision applications. In this paper, we present a novel model evaluation function based on Gaussian-weighted Jensen–Shannon divergence, and integrate into a particle swarm optimization (PSO) framework using ring topology. We avoid two problems from which most regression algorithms suffer, namely the requirements to specify inlier noise scale and the number of models. The novel evaluation method is generic and does not require any estimation of inlier noise. The continuous and meta-heuristic exploration facilitates estimation of each individual model while delivering the number of models automatically. Tests on datasets comprised of inlier noise and a large percentage of outliers (more than 90 % of the data) demonstrate that the proposed framework can efficiently estimate multiple models without prior information. Superior performance in terms of processing time and robustness to inlier noise is also demonstrated with respect to state of the art methods.     

Least trimmed euclidean deviations for robust leverage in regression estimates

Usually, in the regression models, the data are contaminated with unusually observations (outliers). For that reason the last 30 years have developed robust regression estimators. Among them some of the most famous are Least Trimmed Squares (LTS), MM, Penalized Trimmed Square (PTS) and others. Most of these methods, especially PTS, are based on initial leverage, concerning x outlying observations, of the data sample. However, often, multiple x-outliers pull the distance towards their value, causing leverage bias, and this is the masking problem.In this work we develop a new algorithm for robust leverage estimate based on Least Trimmed Euclidean Deviations (LTED). Extensive computational, Monte-Carlo simulations, with varying types of outliers and degrees of contamination, indicate that the LTED procedure identifies successfully the multiple outliers, and the resulting robust leverage improves significantly the PTS performance.     

Absolute penalty and shrinkage estimation in partially linear models

In the context of a partially linear regression model, shrinkage semiparametric estimation is considered based on the Stein-rule. In this framework, the coefficient vector is partitioned into two sub-vectors: the first sub-vector gives the coefficients of interest, i.e., main effects (for example, treatment effects), and the second sub-vector is for variables that may or may not need to be controlled. When estimating the first sub-vector, the best estimate may be obtained using either the full model that includes both sub-vectors, or the reduced model which leaves out the second sub-vector. It is demonstrated that shrinkage estimators which combine two semiparametric estimators computed for the full model and the reduced model outperform the semiparametric estimator for the full model. Using the semiparametric estimate for the reduced model is best when the second sub-vector is the null vector, but this estimator suffers seriously from bias otherwise. The relative dominance picture of suggested estimators is investigated. In particular, suitability of estimating the nonparametric component based on the B-spline basis function is explored. Further, the performance of the proposed estimators is compared with an absolute penalty estimator through Monte Carlo simulation. Lasso and adaptive lasso were implemented for simultaneous model selection and parameter estimation. A real data example is given to compare the proposed estimators with lasso and adaptive lasso estimators.     

Identification of multivariable stochastic linear systems viapolyspectral analysis given noisy input-output time-domain data

The paper considers the problem of identification of unknown parameters of multivariable, linear “errors-invariables” models. Attention is focused on frequency-domain approaches where the integrated polyspectrum (bispectrum or trispectrum) of the input and the integrated cross-polyspectrum, respectively, of the given time-domain input-output data are exploited. Two new classes of parametric frequency-domain approaches are proposed and analyzed. An integrated polyspectrum-based persistence of excitation condition on system input is defined. Both classes of the parameter estimators are shown to be strongly consistent in any measurement noise sequences with vanishing bispectra when integrated bispectrum-based approaches are used. The proposed parameter estimators are shown to be strongly consistent in Gaussian measurement noise when integrated trispectrum-based approaches are used. The input to the system need not be a linear process but must have nonvanishing bispectrum or trispectrum     

Tracking algorithm designed by the local asymptotic approach

The problem of sequential detection of parameter jumps in linear systems with constant noise level is discussed. The detection problem is analyzed by the asymptotic local approach, using the normalized output error sequence as the detection signal. For linear regression, ARMAX, and state-space models, a central limit theorem is proved, transforming the original problem into the problem of detecting an increase in the man of an asymptotically Gaussian distributed scalar process. The performance of the tracking algorithm, which consists of a parameter estimator with decreasing gain and a single Hinkley's detector, has been studied by simulations and compared to the performance of constant- and adaptive-gain parameter estimators. The proposed algorithm seems to be superior in performance, requiring only a little, generally negligible, additional computational effort. The algorithm provides the information about the jump times, and the time delay of jump detection seems to be unaffected by the measurement noise level, provided that this level is not affected by the change     

随机线性多变量系统参数估计的偏差校正法

在文［６］研究的基础上，把文［６］中提出的偏差补偿最小二乘法（ＢＥＬＳ）推广到多变量线性系统的辨识中。分析表明，只要有色噪声与系统的输入信号统计不相关，我国总可以通过对输入信号顶滤波的方法将已知零点嵌入到被辨识系统中，然后利用这些零点提供的信息消除噪声引起的辨识偏差。这种方法的特点是其辨识过程不依赖于有色噪声的模型。     

Local bootstrap approaches for fractional differential parameter estimation in ARFIMA models

In this paper we investigate bootstrap techniques applied to the estimation of the fractional differential parameter in ARFIMA models, d. The novelty is the focus on the local bootstrap of the periodogram function. The approach is then applied to three different semiparametric estimators of d, known from the literature, based upon the periodogram function. By means of an extensive set of simulation experiments, the bias and mean square errors are quantified for each estimator and the efficacy of the local bootstrap is stated in terms of low bias, short confidence intervals, and low CPU times. Finally, a real data set is analyzed to demonstrate that the methodology may be quite effective in solving real problems.