YULE-WALKER ESTIMATES FOR CONTINUOUS-TIME AUTOREGRESSIVE MODELS

Abstract. I consider continuous-time autoregressive processes of order p and develop estimators of the model parameters based on Yule-Walker type equations. For continuously recorded data, it is shown that these estimators are least squares estimators and have the same asymptotic distribution as maximum likelihood estimators.
In practice, though, data can only be observed discretely. For discrete data, I consider approximations to the continuous-time estimators. It is shown that some of these discrete-time estimators are asymptotically biased. Alternative estimators based on the autocovariance function are suggested. These are asymptotically unbiased and are a fast alternative to the maximum likelihood estimators described by Jones. They may also be used as starting values for maximum likelihood estimation.     

A refined efficiency rate for ordinary least squares and generalized least squares estimators for a linear trend with autoregressive errors

When a straight line is fitted to time series data, generalized least squares (GLS) estimators of the trend slope and intercept are attractive as they are unbiased and of minimum variance. However, computing GLS estimators is laborious as their form depends on the autocovariances of the regression errors. On the other hand, ordinary least squares (OLS) estimators are easy to compute and do not involve the error autocovariance structure. It has been known for 50 years that OLS and GLS estimators have the same asymptotic variance when the errors are second‐order stationary. Hence, little precision is gained by using GLS estimators in stationary error settings. This article revisits this classical issue, deriving explicit expressions for the GLS estimators and their variances when the regression errors are drawn from an autoregressive process. These expressions are used to show that OLS methods are even more efficient than previously thought. Specifically, we show that the convergence rate of variance differences is one polynomial degree higher than that of least squares estimator variances. We also refine Grenander's (1954) variance ratio. An example is presented where our new rates cannot be improved upon. Simulations show that the results change little when the autoregressive parameters are estimated.     

Least tail‐trimmed squares for infinite variance autoregressions

We develop a robust least squares estimator for autoregressions with possibly heavy tailed errors. Robustness to heavy tails is ensured by negligibly trimming the squared error according to extreme values of the error and regressors. Tail‐trimming ensures asymptotic normality and super‐‐convergence with a rate comparable to the highest achieved amongst M‐estimators for stationary data. Moreover, tail‐trimming ensures robustness to heavy tails in both small and large samples. By comparison, existing robust estimators are not as robust in small samples, have a slower rate of convergence when the variance is infinite, or are not asymptotically normal. We present a consistent estimator of the covariance matrix and treat classic inference without knowledge of the rate of convergence. A simulation study demonstrates the sharpness and approximate normality of the estimator, and we apply the estimator to financial returns data. Finally, tail‐trimming can be easily extended beyond least squares estimation for a linear stationary AR model. We discuss extensions to quasi‐maximum likelihood for GARCH, weighted least squares for a possibly non‐stationary random coefficient autoregression, and empirical likelihood for robust confidence region estimation, in each case for models with possibly heavy tailed errors.     

The Effect of the Estimation on Goodness‐of‐Fit Tests in Time Series Models

Abstract. We analyze, by simulation, the finite‐sample properties of goodness‐of‐fit tests based on residual autocorrelation coefficients (simple and partial) obtained using different estimators frequently used in the analysis of autoregressive moving‐average time‐series models. The estimators considered are unconditional least squares, maximum likelihood and conditional least squares. The results suggest that although the tests based on these estimators are asymptotically equivalent for particular models and parameter values, their sampling properties for samples of the size commonly found in economic applications can differ substantially, because of differences in both finite‐sample estimation efficiencies and residual regeneration methods.     

ALTERNATIVE ESTIMATORS AND UNIT ROOT TESTS FOR THE AUTOREGRESSIVE PROCESS

Abstract. We compare several estimators for the second-order autoregressive process and compare the associated tests for a unit root. Monte Carlo results are reported for the ordinary least squares estimator, the simple symmetric least squares estimator and the weighted symmetric least squares estimator. The weighted symmetric least squares estimator of the autoregressive parameters generally has smaller mean square error than that of the ordinary least squares estimator, particularly when one root is close to one in absolute value. For the second-order model with known zero intercept, the one-sided ordinary least squares test for a unit root is more powerful than the symmetric tests. For the model with an estimated intercept, the one-sided weighted symmetric least squares test is the most powerful test.     

基于岭估计的化工过程测量置信水平研究(英文)

Ordinary least squares （OLS） algorithm is widely applied in process measurement, because the sensor model used to estimate unknown parameters can be approximated through multivariate linear model. However, with few or noisy data or multi-collinearity, unbiased OLS leads to large variance. Biased estimators, especially ridge es-timator, have been introduced to improve OLS by trading bias for variance. Ridge estimator is feasible as an esti-mator with smaller variance. At the same confidence level, with additive noise as the normal random variable, the less variance one estimator has, the shorter the two-sided symmetric confidence interval is. However, this finding is limited to the unbiased estimator and few studies analyze and compare the confidence levels between ridge estima-tor and OLS. This paper derives the matrix of ridge parameters under necessary and sufficient conditions based on which ridge estimator is superior to OLS in terms of mean squares error matrix, rather than mean squares error. Then the confidence levels between ridge estimator and OLS are compared under the condition of OLS fixed sym-metric confidence interval, rather than the criteria for evaluating the validity of different unbiased estimators. We conclude that the confidence level of ridge estimator can not be directly compared with that of OLS based on the criteria available for unbiased estimators, which is verified by a simulation and a laboratory scale experiment on a single parameter measurement.     

ESTIMATION OF THE MULTIVARIATE AUTOREGRESSIVE MOVING AVERAGE HAVING PARAMETER RESTRICTIONS AND AN APPLICATION TO ROTATIONAL SAMPLING

Abstract. The vector autoregressive moving average model with nonlinear parametric restrictions is considered. A simple and easy-to-compute Newton-Raphson estimator is proposed that approximates the restricted maximum likelihood estimator which takes full advantage of the information contained in the restrictions. In the case when there are no parametric restrictions, our Newton-Raphson estimator is equivalent to the estimator proposed by Reinsel et al. (Maximum likelihood estimators in the multivariate autoregressive moving-average model from a generalized least squares view point. J. Time Ser. Anal. 13 (1992), 133–45). The Newton-Raphson estimation procedure also extends to the vector ARMAX model. Application of our Newton-Raphson estimation method in rotational sampling problems is discussed. Simulation results are presented for two different restricted models to illustrate the estimation procedure and compare its performance with that of two alternative procedures that ignore the parametric restrictions.     

Efficient use of higher-lag autocorrelations for estimating autoregressive processes

The Yule–Walker estimator is commonly used in time-series analysis, as a simple way to estimate the coefficients of an autoregressive process. Under strong assumptions on the noise process, this estimator possesses the same asymptotic properties as the Gaussian maximum likelihood estimator. However, when the noise is a weak one, other estimators based on higher-order empirical autocorrelations can provide substantial efficiency gains. This is illustrated by means of a first-order autoregressive process with a Markov-switching white noise. We show how to optimally choose a linear combination of a set of estimators based on empirical autocorrelations. The asymptotic variance of the optimal estimator is derived. Empirical experiments based on simulations show that the new estimator performs well on the illustrative model.     

On Residual Variance Estimation in Autoregressive Models

In this paper we consider time series models belonging to the autoregressive (AR) family and deal with the estimation of the residual variance. This is important because estimates of the variance are involved in, for example, confidence sets for the parameters of the model, estimation of the spectrum, expressions for the estimated error of prediction and sample quantities used to make inferences about the order of the model. We consider the asymptotic biases for moment and least squares estimators of the residual variance, and compare them with known results when available and with those for maximum likelihood estimators under normality. Simulation results are presented for finite samples     

Linear Trend with Fractionally Integrated Errors

We consider the estimation of linear trend for a time series in the presence of additive long-memory noise with memory parameter d ∈[0, 1.5). Although no parametric model is assumed for the noise, our assumptions include as special cases the random walk with drift as well as linear trend with stationary invertible autoregressive moving-average errors. Moreover, our assumptions include a wide variety of trend-stationary and difference-stationary situations. We consider three different trend estimators: the ordinary least squares estimator based on the original series, the sample mean of the first differences and a class of weighted (tapered) means of the first differences. We present expressions for the asymptotic variances of these estimators in the form of one-dimensional integrals. We also establish the asymptotic normality of the tapered means for d ∈[0, 1.5) −{0.5} and of the ordinary least squares estimator for d ∈ (0.5, 1.5). We point out connections with existing theory and present applications of the methodology.     

ESTIMATION OF COEFFICIENTS OF TIME SERIES REGRESSION WITH A NONSTATIONARY ERROR PROCESS

Abstract. We treat a problem of estimating unknown coefficients of a time series regression when the variance of the error changes with time, i.e. when a process which the error term obeys is nonstationary. First, we show the weak consistency of the ordinary least squares estimator for the coefficients of a polynomial regression under some assumptions on the covariance structure of the error process. Next, we propose a nonparametric method for estimating the variance of the error process and a weighted least squares estimator of the regression coefficients, which is constructed by using the estimator of the variance. We investigate statistical properties of our proposed estimator in the following way. We consider the prediction of a future value of a linear trend by using our proposed estimator and evaluate its prediction error. By simulation studies, we compare the prediction error of the predictor constructed by using our proposed estimator with the prediction errors obtained for other estimators including the ordinary least squares estimator when the variance of the error process increases with time and the sample sizes are small. As a result, our proposed estimator seems to be reasonable.     

TESTING FOR A UNIT ROOT IN AN AR(1) TIME SERIES USING IRREGULARLY OBSERVED DATA

Abstract. For an AR(1) model having a unit root with nonconsecutively observed or missing data we consider the ordinary least squares estimator, the one-step Newton-Raphson estimator and an ordinary least squares type estimator which is a simple approximation of the Newton-Raphson estimator. It is shown that the limiting distributions of these estimators of the unit root are the same as those of the regression estimators as tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Statist. Assoc. 74 (1979), 427–31) for the complete data situation. Simulation results show that our proposed unit root tests perform very well for small samples.     

ASYMPTOTIC PROPERTIES OF SOME PRELIMINARY ESTIMATORS FOR AUTOREGRESSIVE MOVING AVERAGE TIME SERIES MODELS

Abstract. Some simple preliminary estimators for the coefficients of mixed autoregressive moving average time series models are considered. As the first step the estimators require the fitting of a long autoregression to the data. The first two methods of the paper are non-iterative and generally inefficient. The estimators are Yule-Walker type modifications of the least squares estimators of the coefficients in auxiliary linear regression models derived, respectively, for the coefficients of the long autoregression and for the coefficients of the corresponding long moving average approximation of the model. Both of these estimators are shown to be strongly consistent and their asymptotic distributions are derived. The asymptotic distributions are used in studying the loss in efficiency and in constructing the third estimator of the paper which is an asymptotically efficient two-step estimator. A numerical illustration of the third estimator with real data is given.     

ESTIMATION FOR REGRESSIVE AND AUTOREGRESSIVE MODELS WITH NON-NEGATIVE RESIDUAL ERRORS

Abstract. The parameter estimation problems for regressive and autoregressive models are investigated. A new procedure is proposed which differs from the least squares method. Theorems relating to the rate of almost sure convergence of the new estimators are formulated. Some simulation results are also shown. With these convergent rates and simulation results a clear comparison of the new estimator with the least squares estimator is obtained.     

Quasi‐Maximum Likelihood Estimation for a Class of Continuous‐time Long‐memory Processes

Tsai and Chan (2003) has recently introduced the Continuous‐time Auto‐Regressive Fractionally Integrated Moving‐Average (CARFIMA) models useful for studying long‐memory data. We consider the estimation of the CARFIMA models with discrete‐time data by maximizing the Whittle likelihood. We show that the quasi‐maximum likelihood estimator is asymptotically normal and efficient. Finite‐sample properties of the quasi‐maximum likelihood estimator and those of the exact maximum likelihood estimator are compared by simulations. Simulations suggest that for finite samples, the quasi‐maximum likelihood estimator of the Hurst parameter is less biased but more variable than the exact maximum likelihood estimator. We illustrate the method with a real application.     

MAXIMUM LIKELIHOOD ESTIMATION FOR AUTOREGRESSIVE PROCESSES DISTURBED BY A MOVING AVERAGE

Abstract. Maximum likelihood estimation for stationary autoregressive processes when the signal is subject to a moving-average sampling error is discussed. A modified maximum likelihood estimator is proposed. An algorithm for computing derivatives of the modified likelihood is suggested. Maximum likelihood estimators of the parameter vector are shown to be strongly consistent and to have a multivariate normal limiting distribution. A Monte Carlo simulation shows that the modified maximum likelihood estimator performs better than other available estimators. US current labour force data are analysed as an example.     

A study on misspecified nonstationary autoregressive time series with a unit root

The effects of order misspecification in nonstationary autoregressive time series estimations are investigated. The true process is assumed to be stationary if differenced. The ordinary least squares estimator is shown to be weakly convergent and its probability limit is derived. Expressions for the dominating terms of the prediction error and of the prediction mean squared error are derived. Using the expressions and Monte Carlo simulations, we compare prediction errors in the misspecified models based on the observation series and those based on the differenced series.     

Maximum likelihood estimation of higher‐order integer‐valued autoregressive processes

Abstract. In this article, we extend the earlier work of Freeland and McCabe [Journal of time Series Analysis (2004) Vol. 25, pp. 701–722] and develop a general framework for maximum likelihood (ML) analysis of higher‐order integer‐valued autoregressive processes. Our exposition includes the case where the innovation sequence has a Poisson distribution and the thinning is binomial. A recursive representation of the transition probability of the model is proposed. Based on this transition probability, we derive expressions for the score function and the Fisher information matrix, which form the basis for ML estimation and inference. Similar to the results in Freeland and McCabe (2004) , we show that the score function and the Fisher information matrix can be neatly represented as conditional expectations. Using the INAR(2) specification with binomial thinning and Poisson innovations, we examine both the asymptotic efficiency and finite sample properties of the ML estimator in relation to the widely used conditional least squares (CLS) and Yule–Walker (YW) estimators. We conclude that, if the Poisson assumption can be justified, there are substantial gains to be had from using ML especially when the thinning parameters are large.     

Redescending estimators for data reconciliation and parameter estimation

Gross error detection is crucial for data reconciliation and parameter estimation, as gross errors can severely bias the estimates and the reconciled data. Robust estimators significantly reduce the effect of gross errors (or outliers) and yield less biased estimates. An important class of robust estimators are maximum likelihood estimators or M-estimators. These are commonly of two types, Huber estimators and Hampel estimators. The former significantly reduces the effect of large outliers whereas the latter nullifies their effect. In particular, these two estimators can be evaluated through the use of an influence function, which quantifies the effect of an observation on the estimated statistic. Here, the influence function must be bounded and finite for an estimator to be robust. For the Hampel estimators the influence function becomes zero for large outliers, nullifying their effect. On the other hand, Huber estimators do not reject large outliers; their influence function is simply bounded. As a result, we consider the three part redescending estimator of Hampel and compare its performance with a Huber estimator, the Fair function. A major advantage to redescending estimators is that it is easy to identify outliers without having to perform any exploratory data analysis on the residuals of regression. Instead, the outliers are simply the rejected observations. In this study, the redescending estimators are also tuned to the particular observed system data through an iterative procedure based on the Akaike information criterion, (AIC). This approach is not easily afforded by the Huber estimators and this can have a significant impact on the estimation. The resulting approach is incorporated within an efficient non-linear programming algorithm. Finally, all of these features are demonstrated on a number of process and literature examples for data reconciliation.