Exactly/Nearly Unbiased Estimation of Autocovariances of a Univariate Time Series With Unknown Mean

This article proposes an exactly/nearly unbiased estimator of the autocovariance function of a univariate time series with unknown mean. The estimator is a linear function of the usual sample autocovariances computed using the observed demeaned data. The idea is to stack the usual sample autocovariances into a vector and show that the expectation of this vector is a linear combination of population autocovariances. A matrix that we label, A , collects the weights in these linear combinations. When the population autocovariances of high lags are zero (small), exactly (nearly) unbiased estimators of the remaining autocovariances can be obtained using the inverse of upper blocks of the A matrix. The A ‐matrix estimators are shown to be asymptotically equivalent to the usual sample autocovariance estimators. The A ‐matrix estimators can be used to construct estimators of the autocorrelation function that have less bias than the usual estimators. Simulations show that the A ‐matrix estimators can substantially reduce bias while not necessarily increasing mean square error. More powerful tests for the null hypothesis of white noise are obtained using the A ‐matrix estimators.     

Nonlinear spectral density estimation: thresholding the correlogram

Traditional kernel spectral density estimators are linear as a function of the sample autocovariance sequence. The purpose of this article is to propose and analyse two new spectral estimation methods that are based on the sample autocovariances in a nonlinear way. The rate of convergence of the new estimators is quantified, and practical issues such as bandwidth and/or threshold choice are addressed. The new estimators are also compared with traditional ones using flat‐top lag‐windows in a simulation experiment involving sparse time‐series models.     

An Estimating Method for Parametric Spectral Densities of Gaussian Time Series

An estimating method for spectral densities of Gaussian time series that belong to a parametric model is proposed. Spectral density estimators are evaluated by using average Kullback–Leibler divergence from the true spectral density to estimated spectral densities. In the classical approach, unknown spectral densities are estimated by replacing the unknown parameters by asymptotically efficient estimates. In the alternative method introduced in the present paper, spectral density estimates usually do not belong to the model. The alternative spectral density estimators asymptotically dominate the classical ones. The difference in average Kullback–Leibler divergence between them can be regarded as the mixture mean curvature of the model in the space of all spectral densities. The explicit expression for the proposed estimators of spectral densities of autoregressive processes is obtained. The accuracy of prediction can be improved by using predictors that correspond to the alternative spectral density estimators.     

Rate of convergence in the central limit theorem for parameter estimation in a causal,invertible ARMA(p,q) model

In this study we consider the estimators of the parameters of a stable ARMA(p, q) process. The autoregressive parameters are estimated by the instrumental variable technique while the moving average parameters are estimated using a derived autoregressive process. The estimators are shown to be asymptotically normal and their rate of convergence to normality is derived.     

A refined efficiency rate for ordinary least squares and generalized least squares estimators for a linear trend with autoregressive errors

When a straight line is fitted to time series data, generalized least squares (GLS) estimators of the trend slope and intercept are attractive as they are unbiased and of minimum variance. However, computing GLS estimators is laborious as their form depends on the autocovariances of the regression errors. On the other hand, ordinary least squares (OLS) estimators are easy to compute and do not involve the error autocovariance structure. It has been known for 50 years that OLS and GLS estimators have the same asymptotic variance when the errors are second‐order stationary. Hence, little precision is gained by using GLS estimators in stationary error settings. This article revisits this classical issue, deriving explicit expressions for the GLS estimators and their variances when the regression errors are drawn from an autoregressive process. These expressions are used to show that OLS methods are even more efficient than previously thought. Specifically, we show that the convergence rate of variance differences is one polynomial degree higher than that of least squares estimator variances. We also refine Grenander's (1954) variance ratio. An example is presented where our new rates cannot be improved upon. Simulations show that the results change little when the autoregressive parameters are estimated.     

DESIGN OF A COMPOSITION ESTIMATOR FOR INFERENTIAL CONTROL OF DISTILLATION COLUMNS

In distillation column control, secondary measurements such as temperatures and flows are widely used in order to infer product composition. This paper addresses the design of the linear static estimators using the secondary measurements for estimating product compositions of distillation columns. Based on the unified framework for the estimator design, the relationships among various static estimators are discussed in terms of the estimator structure. Il is shown that the projection estimator is equivalent to the regression estimators in the special cases. Since the projection estimator heavily depends on the measured inputs such as reflux flow and heat input to the reboiler due to its structural characteristic, the estimation performance is far more sensitive to measurement noise and nonlinearity of them, compared wiih the regression estimators based on the PCR or PLS method. It is also found that the use of the measured inputs leads to performance deterioration of both the projection and regression estimators because of their nonlinear effects on the product compositions especially in high-purity columns. Design guidelines for the PCR and PLS estimators are presented by analyzing the results of the simulation studies on a high-purity column example. The estimator based on the guidelines is robust to sensor noise and has a good predictive power     

Robust and powerful serial correlation tests with new robust estimates in ARX models

Abstract. We consider robust serial correlation tests in autoregressive models with exogenous variables (ARX). Since the least squares estimators are not robust when outliers are present, a new family of estimators is introduced, called residual autocovariances for ARX (RA‐ARX). They provide resistant estimators that are less sensible to abnormal observations in the output variable of the dynamic model. Such ‘bad’ observations could be due to unexpected phenomena such as economic crisis or equipment failure in engineering, among others. We show that the new robust estimators are consistent and we can consider robust and powerful tests of serial correlation in ARX models based on these estimators. The new one‐sided tests of serial correlation are obtained in extending Hong's (1996) approach in a framework resistant to outliers. They are based on a weighted sum of robust squared residual autocorrelations and on any robust and n^1/2‐consistent estimators. Our approach generalizes Li's (1988) test statistic, that can be interpreted as a test using the truncated uniform kernel. However, many kernels deliver a higher power. This is confirmed in a simulation study, where we investigate the finite sample properties of the new robust serial correlation tests in comparison to some commonly used robust and non‐robust tests.     

YULE-WALKER ESTIMATES FOR CONTINUOUS-TIME AUTOREGRESSIVE MODELS

Abstract. I consider continuous-time autoregressive processes of order p and develop estimators of the model parameters based on Yule-Walker type equations. For continuously recorded data, it is shown that these estimators are least squares estimators and have the same asymptotic distribution as maximum likelihood estimators.
In practice, though, data can only be observed discretely. For discrete data, I consider approximations to the continuous-time estimators. It is shown that some of these discrete-time estimators are asymptotically biased. Alternative estimators based on the autocovariance function are suggested. These are asymptotically unbiased and are a fast alternative to the maximum likelihood estimators described by Jones. They may also be used as starting values for maximum likelihood estimation.     

The Effect of the Estimation on Goodness‐of‐Fit Tests in Time Series Models

Abstract. We analyze, by simulation, the finite‐sample properties of goodness‐of‐fit tests based on residual autocorrelation coefficients (simple and partial) obtained using different estimators frequently used in the analysis of autoregressive moving‐average time‐series models. The estimators considered are unconditional least squares, maximum likelihood and conditional least squares. The results suggest that although the tests based on these estimators are asymptotically equivalent for particular models and parameter values, their sampling properties for samples of the size commonly found in economic applications can differ substantially, because of differences in both finite‐sample estimation efficiencies and residual regeneration methods.     

Redescending estimators for data reconciliation and parameter estimation

Gross error detection is crucial for data reconciliation and parameter estimation, as gross errors can severely bias the estimates and the reconciled data. Robust estimators significantly reduce the effect of gross errors (or outliers) and yield less biased estimates. An important class of robust estimators are maximum likelihood estimators or M-estimators. These are commonly of two types, Huber estimators and Hampel estimators. The former significantly reduces the effect of large outliers whereas the latter nullifies their effect. In particular, these two estimators can be evaluated through the use of an influence function, which quantifies the effect of an observation on the estimated statistic. Here, the influence function must be bounded and finite for an estimator to be robust. For the Hampel estimators the influence function becomes zero for large outliers, nullifying their effect. On the other hand, Huber estimators do not reject large outliers; their influence function is simply bounded. As a result, we consider the three part redescending estimator of Hampel and compare its performance with a Huber estimator, the Fair function. A major advantage to redescending estimators is that it is easy to identify outliers without having to perform any exploratory data analysis on the residuals of regression. Instead, the outliers are simply the rejected observations. In this study, the redescending estimators are also tuned to the particular observed system data through an iterative procedure based on the Akaike information criterion, (AIC). This approach is not easily afforded by the Huber estimators and this can have a significant impact on the estimation. The resulting approach is incorporated within an efficient non-linear programming algorithm. Finally, all of these features are demonstrated on a number of process and literature examples for data reconciliation.     

ASYMPTOTIC PROPERTIES OF SOME PRELIMINARY ESTIMATORS FOR AUTOREGRESSIVE MOVING AVERAGE TIME SERIES MODELS

Abstract. Some simple preliminary estimators for the coefficients of mixed autoregressive moving average time series models are considered. As the first step the estimators require the fitting of a long autoregression to the data. The first two methods of the paper are non-iterative and generally inefficient. The estimators are Yule-Walker type modifications of the least squares estimators of the coefficients in auxiliary linear regression models derived, respectively, for the coefficients of the long autoregression and for the coefficients of the corresponding long moving average approximation of the model. Both of these estimators are shown to be strongly consistent and their asymptotic distributions are derived. The asymptotic distributions are used in studying the loss in efficiency and in constructing the third estimator of the paper which is an asymptotically efficient two-step estimator. A numerical illustration of the third estimator with real data is given.     

Applications of linear estimation theory to chemical processes: a feasibility study

The feasibility of using linear recursive least-squares estimators for on-line state and parameter estimation is studied with reference to two examples. With the first example, a binary distillation column, it is shown that the estimators are insensitive to the statistical approximations involved and provide a powerful means of dealing with bias, drift, or scatter in instrument readings. In the second example, a reactor with decaying catalyst, it is shown that drastic model simplification is possible and model parameters can be successfully estimated from indirect measurements. Formulae for growing memory and limited memory filters for measurement noise are derived in the appendix.     

Rank‐based estimation for autoregressive moving average time series models

Abstract. We establish asymptotic normality and consistency for rank‐based estimators of autoregressive‐moving average model parameters. The estimators are obtained by minimizing a rank‐based residual dispersion function similar to the one given by L.A. Jaeckel [Ann. Math. Stat. Vol. 43 (1972) 1449–1458]. These estimators can have the same asymptotic efficiency as maximum likelihood estimators and are robust. The quality of the asymptotic approximations for finite samples is studied via simulation.     

Unit root bootstrap tests under infinite variance

This article presents a family of new tests for unit roots based on M‐estimators. Their robustness makes them very appealing when working with distributions that have infinite variance or heavy tails. These tests are completely automatic regardless of the complex distributions of this kind of estimators because the critical values are approximated using bootstrap, no additional parameter has to be estimated and the results obtained are very good in small samples. An exhaustive Monte Carlo study shows the high performance of these tests compared with others proposed in the literature when the variance is infinite.     

Distributed state estimation from infrequent and delayed measurements in large-scale processes

Effective control and monitoring of a process usually require frequent and delay-free measurements of important process output variables. However, these measurements are often either not available or available infrequently with significant time delays. This article presents a method that allows for improving the performance of distributed state estimators implemented on large-scale manufacturing processes. The method uses a sample state augmentation approach that permits using delayed measurements in distributed state estimation. The method can be used with any state estimator, including unscented Kalman filters, extended Kalman filters, and moving horizon state estimators. The method optimally handles the tradeoff between computational time and estimation accuracy in distributed state estimation implemented using a computer with parallel processors. Its implementation and performance are shown using a few simulated examples.     

SEMI‐PARAMETRIC ESTIMATION OF LINEAR COINTEGRATING MODELS WITH NONLINEAR CONTEMPORANEOUS ENDOGENEITY

This article considers linear cointegrating models with unknown nonlinear short‐run contemporaneous endogeneity. Two estimators are proposed to estimate the linear cointegrating parameter after the nonlinear endogenous component is estimated by local linear regression approach. Both the proposed estimators are shown to have the same mixed normal limiting distribution with zero mean and smaller asymptotic variance than the fully modified ordinary least squares and instrumental variables estimators. Monte Carlo simulations are used to evaluate the finite sample performance of our proposed estimators, and an empirical application is also included.     

CROSS-VALIDATORY CHOICE OF A SPECTRUM ESTIMATE AND ITS CONNECTIONS WITH AIC

Abstract. This paper concerns the use of a generalized version of the cross-validated log likelihood criterion (CVLL) for selecting a spectrum estimator from an arbitrary class of candidate estimators. It is shown that CVLL is asymptotically equivalent to the expected Kullback-Leibler information of the candidate estimator. The Akaike information criterion (AIC) is also asymptotically equivalent to Kullback-Leibler information, but the applicability of AIC is limited to parametric estimators. Thus CVLL can be viewed as a cross-validatory generalization of AIC. Monte Carlo results show that CVLL is able to provide an effective choice from a class of candidates which simultaneously includes autoregressive and classical smoothed periodogram estimators. To save computation time, CVLL can be evaluated only for the classical estimators while the computationally more efficient AIC is evaluated for the parametric estimators. The criterion values are all directly comparable in this case. As an additional computation-saving device, a non-cross-validatory version of CVLL for classical estimators is proposed and studied.     

MULTIPLICATIVE EXPONENTIAL MODELS FOR STATIONARY TIME SERIES

Abstract. A class of models for one dimensional time series is presented. The spectrum of such a model is obtained by raising the spectrum of a known parameterized model to an exponent, allowed to attain arbitrary real values. For a moving average model this for example means that the roots of the moving average operator are allowed to have any real order. This method adds a further flexibility to the model which for example allows us to model long memory time series using only a few parameters. The exponent is parameterized in a special way to make the estimation of the parameter determining the exponent asymptotically independent of the estimation of the other model-parameters. The asymptotic distribution of the estimators is derived. The idea is also used for multiplicative models with an exponent for each seasonal factor. In this case the estimators are only approximately independent for a large season length. Finally an application of the model is given using the Beveridge wheat price index.     

Robust estimators under semi‐parametric partly linear autoregression: Asymptotic behaviour and bandwidth selection

Abstract. In this article, under a semi‐parametric partly linear autoregression model, a family of robust estimators for the autoregression parameter and the autoregression function is studied. The proposed estimators are based on a three‐step procedure, in which robust regression estimators and robust smoothing techniques are combined. Asymptotic results on the autoregression estimators are derived. Besides combining robust procedures with M‐smoothers, predicted values for the series and detection residuals, which allow to detect anomalous data, are introduced. Robust cross‐validation methods to select the smoothing parameter are presented as an alternative to the classical ones, which are sensitive to outlying observations. A Monte Carlo study is conducted to compare the performance of the proposed criteria. Finally, the asymptotic distribution of the autoregression parameter estimator is stated uniformly over the smoothing parameter.     

Drying Kinetics of Silica Gel: Statistical Discrimination Using Nonlinearity Measures

The objective of the present work was to analyze the drying kinetics of silica gel based on an experimental study performed in a thin‐layer dryer and a statistical discrimination of the main drying kinetic equations. Most semi‐empirical drying kinetics equations presented in the literature are nonlinear; thus, care should be taken when estimating parameters, since in some situations the estimators may not be appropriate. There are procedures available to validate the statistical properties of the least squares estimators of nonlinear models. In this study, five semi‐empirical drying kinetics equations were discriminated using measures of curvature and bias. The results showed that the Overhults equation is the best one to describe the drying kinetics of silica gel.