Robust estimation of the scale and of the autocovariance function of Gaussian short‐ and long‐range dependent processes

A desirable property of an autocovariance estimator is to be robust to the presence of additive outliers. It is well known that the sample autocovariance, being based on moments, does not have this property. Hence, the use of an autocovariance estimator which is robust to additive outliers can be very useful for time‐series modelling. In this article, the asymptotic properties of the robust scale and autocovariance estimators proposed by Rousseeuw and Croux (1993) and Ma and Genton (2000) are established for Gaussian processes, with either short‐ or long‐range dependence. It is shown in the short‐range dependence setting that this robust estimator is asymptotically normal at the rate , where n is the number of observations. An explicit expression of the asymptotic variance is also given and compared with the asymptotic variance of the classical autocovariance estimator. In the long‐range dependence setting, the limiting distribution displays the same behaviour as that of the classical autocovariance estimator, with a Gaussian limit and rate when the Hurst parameter H is less than 3/4 and with a non‐Gaussian limit (belonging to the second Wiener chaos) with rate depending on the Hurst parameter when H ∈ (3/4,1). Some Monte Carlo experiments are presented to illustrate our claims and the Nile River data are analysed as an application. The theoretical results and the empirical evidence strongly suggest using the robust estimators as an alternative to estimate the dependence structure of Gaussian processes.     

Robust Estimation in Vector Autoregressive Moving-Average Models

Bustos and Yohai proposed a class of robust estimates for autoregressive moving-average (ARMA) models based on residual autocovariances (RA estimates). In this paper an affine equivariant generalization of the RA estimates for vector ARMA processes is given. These estimates are asymptotically normal and, when the innovations have an elliptical distribution, their asymptotic covariance matrix differs only by a scalar factor from the covariance matrix corresponding to the maximum likelihood estimate. A Monte Carlo study confirms that the RA estimates are efficient under normal errors and robust when the sample contains outliers. A robust multivariate goodness-of-fit test based on the RA estimates is also obtained.     

Robust estimators under semi‐parametric partly linear autoregression: Asymptotic behaviour and bandwidth selection

Abstract. In this article, under a semi‐parametric partly linear autoregression model, a family of robust estimators for the autoregression parameter and the autoregression function is studied. The proposed estimators are based on a three‐step procedure, in which robust regression estimators and robust smoothing techniques are combined. Asymptotic results on the autoregression estimators are derived. Besides combining robust procedures with M‐smoothers, predicted values for the series and detection residuals, which allow to detect anomalous data, are introduced. Robust cross‐validation methods to select the smoothing parameter are presented as an alternative to the classical ones, which are sensitive to outlying observations. A Monte Carlo study is conducted to compare the performance of the proposed criteria. Finally, the asymptotic distribution of the autoregression parameter estimator is stated uniformly over the smoothing parameter.     

Highly Robust Estimation of the Autocovariance Function

In this paper, the problem of the robustness of the sample autocovariance function is addressed. We propose a new autocovariance estimator, based on a highly robust estimator of scale. Its robustness properties are studied by means of the influence function, and a new concept of temporal breakdown point. As the theoretical variance of the estimator does not have a closed form, we perform a simulation study. Situations with various size of outliers are tested. They confirm the robustness properties of the new estimator. An S-Plus function for the highly robust autocovariance estimator is made available on the Web at http://www-math.mit.edu/~yanyuan/Genton/Time/time.html. At the end, we analyze a time series of monthly interest rates of an Austrian bank.     

Testing equality of autocovariance operators for functional time series

We consider strictly stationary stochastic processes of Hilbert space-valued random variables and focus on fully functional tests for the equality of the lag-zero autocovariance operators of several independent functional time series. A moving block bootstrap (MBB)-based testing procedure is proposed which generates pseudo random elements that satisfy the null hypothesis of interest. It is based on directly bootstrapping the time series of tensor products which overcomessome common difficulties associated with applications of the bootstrap to related testing problems. The suggested methodology can be potentially applied to a broad range of test statistics of the hypotheses of interest. As an example, we establish validity for approximating the distribution under the null of a test statistic based on the Hilbert–Schmidt distance of the corresponding sample lag-zero autocovariance operators, and show consistency under the alternative. As a prerequisite, we prove a central limit theorem for the MBB procedure applied to the sample autocovariance operator which is of interest on its own. The finite sample size and power performance of the suggested MBB-based testing procedure is illustrated through simulations and an application to a real-life dataset is discussed.     

A JOINT EXPERIMENTAL DESIGN PROCEDURE FOR PARAMETER ESTIMATION

A joint experimental design procedure which aims at obtaining precise parameter estimates of a given mathematical model is proposed and illustrated by means of computer simulated experiments

This criterion is found both to modify the shape of the joint confidence hyperellipsoid and to reduce its volume significantly

Therefore, it behaves as a combination of two of the most representative techniques in use today, known as the minimum volume or the shape criteria. Numerical results are given which serve to display its salient features and to compare it with the above mentioned techniques.     

Parameter identification for state estimation—application to bioprocess software sensors

Together with some on-line measurements, a reliable process model is the key ingredient of a successful state observer design. In common practice, the model parameters are inferred from experimental data so as to minimize a model prediction error, e.g. so as to minimize an output least-squares criterion. In this procedure, no care is actually exercised to ensure that the unmeasured model states are sensitive to the measured states. In turn, if sensitivity is too low, the resulting state observer will probably generate poor estimates of the unmeasured states. To alleviate these problems, a new parameter identification procedure is proposed in this study, which is based on a cost function combining a conventional prediction error criterion with a state estimation sensitivity measure. Minimization of this combined cost function produces a model dedicated to state estimation purposes. A thorough analysis of the procedure is presented in the context of bioreactor modeling, including parameter identification, model validation and design of extended Kalman filters and full horizon observers.     

Nonlinear stochastic modeling to improve state estimation in process monitoring and control

State estimation from plant measurements plays an important role in advanced monitoring and control technologies, especially for chemical processes with nonlinear dynamics and significant levels of process and sensor noise. Several types of state estimators have been shown to provide high‐quality estimates that are robust to significant process disturbances and model errors. These estimators require a dynamic model of the process, including the statistics of the stochastic disturbances affecting the states and measurements. The goal of this article is to introduce a design method for nonlinear state estimation including the following steps: (i) nonlinear process model selection, (ii) stochastic disturbance model selection, (iii) covariance identification from operating data, and (iv) estimator selection and implementation. Results on the implementation of this design method in nonlinear examples (CSTR and large dimensional polymerization process) show that the linear time‐varying autocovariance least‐squares technique accurately estimates the noise covariances for the examples analyzed, providing a good set of such covariances for the state estimators implemented. On the estimation implementation, a case study of a chemical reactor demonstrates the better capabilities of MHE when compared with the extended Kalman filter. © 2010 American Institute of Chemical Engineers AIChE J, 2011     

Redescending estimators for data reconciliation and parameter estimation

Gross error detection is crucial for data reconciliation and parameter estimation, as gross errors can severely bias the estimates and the reconciled data. Robust estimators significantly reduce the effect of gross errors (or outliers) and yield less biased estimates. An important class of robust estimators are maximum likelihood estimators or M-estimators. These are commonly of two types, Huber estimators and Hampel estimators. The former significantly reduces the effect of large outliers whereas the latter nullifies their effect. In particular, these two estimators can be evaluated through the use of an influence function, which quantifies the effect of an observation on the estimated statistic. Here, the influence function must be bounded and finite for an estimator to be robust. For the Hampel estimators the influence function becomes zero for large outliers, nullifying their effect. On the other hand, Huber estimators do not reject large outliers; their influence function is simply bounded. As a result, we consider the three part redescending estimator of Hampel and compare its performance with a Huber estimator, the Fair function. A major advantage to redescending estimators is that it is easy to identify outliers without having to perform any exploratory data analysis on the residuals of regression. Instead, the outliers are simply the rejected observations. In this study, the redescending estimators are also tuned to the particular observed system data through an iterative procedure based on the Akaike information criterion, (AIC). This approach is not easily afforded by the Huber estimators and this can have a significant impact on the estimation. The resulting approach is incorporated within an efficient non-linear programming algorithm. Finally, all of these features are demonstrated on a number of process and literature examples for data reconciliation.     

Robust estimation for the covariance matrix of multi‐variate time series

In this article, we study the robust estimation for the covariance matrix of stationary multi‐variate time series. As a robust estimator, we propose to use a minimum density power divergence estimator (MDPDE) proposed by Basu et al. (1998) . Particularly, the MDPDE is designed to perform properly when the time series is Gaussian. As a special case, we consider the robust estimator for the autocovariance function of univariate stationary time series. It is shown that the MDPDE is strongly consistent and asymptotically normal under regularity conditions. Simulation results are provided for illustration.     

Parameter Estimation Under Failure-Censored Constant-Stress Life Testing Model: A Bayesian Approach

Abstract

This article compares likelihood and Bayesian estimations for partially accelerated constant-stress life test model under type II censoring assuming Pareto distribution of the second kind. Both maximum likelihood and Bayesian estimators of the model parameters are derived. The posterior means and posterior variances are obtained under the squared error loss function using Lindley's approximation procedure. The advantages of this proposed procedure are shown. Monte Carlo simulations are conducted under different samples sizes and different parameter values to assess and compare the proposed methods of estimation. A noninformative prior on the model parameters is used to make the comparison more meaningful. It has been observed that Lindley's method usually provides posterior variances and mean squared errors smaller than those of the maximum likelihood estimators. That is, Lindley's method produces improved estimates, which is an advantage of this method.     

Recursive versions Of the algorithm by krasker and welsch

Krasker and Welsch proposed an estimator related to GM-esti-mators for robust estimation in the classical linear model. In this paper we use standard methods for creating recursive algorithms to formulate a recursive version of their estimator. We also observe that a slight modification of the basic estimator by Krasker and Welsch has some symmetry properties which can be utilized to formulate an alternative recursive algorithm. The performance of the two algorithms for estimation of AR-parameters is studied in simulations. These indicate that both methods work well, though the second one seems to be slightly better. Hence the modified estimator introduced in this paper may be better suited for recursive versions.     

Robust estimates for arch processes

In this paper, we present two robust estimates for ARCH(p) models: τ ‐ and filtered τ‐estimates. These are defined by the minimization of conveniently robustified likelihood functions. The robustification is achieved by replacing the mean square error of the standardized observations with the square of a robust τ‐scale estimate in the reduced form of the Gaussian likelihood function. The robust filtering procedure avoids the propagation of the effect of one outlier on subsequent conditional variances. A Monte‐Carlo study shows that the maximum likelihood estimate practically collapses when there is only a small percentage of outlier contamination, while both robust estimates perform much better.     

A NOTE ON ASYMPTOTIC INFERENCE IN AUTOREGRESSIVE MODELS WITH ROOTS ON THE UNIT CIRCLE

Abstract. An estimation and inference procedure is proposed for parameters of the p th order autoregressive model with roots both on the unit circle and outside the unit circle. The procedure is motivated by the fact that the parameter estimates of the nonstationary part of the model have higher order consistency properties than the parameter estimates of the stationary part. The procedure allows the use of the known asymptotic distributional results of purely nonstationary models and purely stationary models. Only ordinary least squares routines are needed.     

PARAMETER ESTIMATION IN EXPONENTIAL MODELS

Abstract. In this paper the problems of parameter estimation and order determination of an exponential (EX) model are studied in the time domain. In order to estimate the parameters, the parameter equations of an EX model are given in terms of the autocorrelation function, which is similar to the Yule-Walker equations of an autoregressive moving-average model. Estimates of parameters are obtained with the aid of the parameter equations and theorems are proved relating the convergence rate and asymptotic distribution of the estimates. We present two kinds of methods for estimating the order and prove that the estimates of the order are consistent.     

Banded and tapered estimates for autocovariance matrices and the linear process bootstrap

We address the problem of estimating the autocovariance matrix of a stationary process. Under short range dependence assumptions, convergence rates are established for a gradually tapered version of the sample autocovariance matrix and for its inverse. The proposed estimator is formed by leaving the main diagonals of the sample autocovariance matrix intact while gradually down‐weighting off‐diagonal entries towards zero. In addition, we show the same convergence rates hold for a positive definite version of the estimator, and we introduce a new approach for selecting the banding parameter. The new matrix estimator is shown to perform well theoretically and in simulation studies. As an application, we introduce a new resampling scheme for stationary processes termed the linear process bootstrap (LPB). The LPB is shown to be asymptotically valid for the sample mean and related statistics. The effectiveness of the proposed methods are demonstrated in a simulation study.     

基于多模型外部分析和Greedy-KP1M的多工况过程监控

传统的基于多元统计过程监控方法都是假设过程处于单一工况下,而随着进料负荷、产品组分等过程参数的改变,生产过程的工况也随之改变,传统方法便不再适用。针对工业过程中的多工况监控问题,提出了一种基于多模型外部分析和Greedy-KP1M的多工况过程监控方法。首先针对传统外部分析方法描述能力不足的问题,用多模型局部建模代替单一模型来获得更好的描述能力,同时获得监控残差,通过对残差进行监控从而去除多工况的影响,进而将核单簇可能性聚类(KP1M)用于对残差的监控上。该方法拥有和支持向量数据描述(SVDD)相当的监控效果,但计算复杂度却远远小于SVDD。同时,采用Greedy方法提取特征样本,进一步降低了算法计算复杂度。最后将上述方法应用在TE模型和乙烯裂解炉的监控上,结果证明了该方法的有效性。     

A data‐driven multistage adaptive robust optimization framework for planning and scheduling under uncertainty

A novel data‐driven approach for optimization under uncertainty based on multistage adaptive robust optimization (ARO) and nonparametric kernel density M‐estimation is proposed. Different from conventional robust optimization methods, the proposed framework incorporates distributional information to avoid over‐conservatism. Robust kernel density estimation with Hampel loss function is employed to extract probability distributions from uncertainty data via a kernelized iteratively reweighted least squares algorithm. A data‐driven uncertainty set is proposed, where bounds of uncertain parameters are defined by quantile functions, to organically integrate the multistage ARO framework with uncertainty data. Based on this uncertainty set, we further develop an exact robust counterpart in its general form for solving the resulting data‐driven multistage ARO problem. To illustrate the applicability of the proposed framework, two typical applications in process operations are presented: The first one is on strategic planning of process networks, and the other one on short‐term scheduling of multipurpose batch processes. The proposed approach returns 23.9% higher net present value and 31.5% more profits than the conventional robust optimization method in planning and scheduling applications, respectively. © 2017 American Institute of Chemical Engineers AIChE J, 63: 4343–4369, 2017     

ORDERS AND INITIAL VALUES OF NON-STATIONARY MULTIVARIATE ARMA MODELS

Abstract. Akaike's stepwise canonical correlations procedure identifies each autoregressive (AR) order (assuming a lesser moving-average (MA) order) and gives initial parameter estimates of a stationary multivariate autoregressive moving-average model. We show that a similar procedure is valid for the non-stationary model when this can be approximated by a large-order AR-only model which has consistent least-squares estimators.
The procedure can also be adapted for all MA orders by fixing the overall maximum of any MA order less its corresponding AR order. We perform the canonical correlations procedure on a succession of values of this maximum and compare the results to obtain one or more prospective model structures.