BOOTSTRAPPING STATIONARY AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. In this paper we develop an asymptotic theory for application of the bootstrap to stationary stochastic processes of autoregressive moving-average (ARMA) type, with known order ( p, q ). We give a proof of the asymptotic validity of the bootstrap proposal applied to M estimators for the unknown parameter vector of the process. For this purpose we derive an asymptotic expansion for M estimators in ARMA models and construct an estimate for the unknown distribution function of the residuals which in principle are not observable. A small simulation study is also included.     

ESTIMATION OF AUTOREGRESSIVE MOVING-AVERAGE MODELS VIA HIGH-ORDER AUTOREGRESSIVE APPROXIMATIONS

Abstract. In this paper the problem of estimating autoregressive moving-average (ARMA) models is dealt with by first estimating a high-order autoregressive (AR) approximation and then using the AR estimate to form the ARMA estimate. We show how to obtain an efficient ARMA estimate by allowing the order of the AR estimate to tend to infinity as the number of observations tends to infinity. This approach is closely related to the work of Durbin. By transforming the approach into the frequency domain, we can view it as an L ²-norm model approximation of the relative error of the spectral factors. It can also be seen as replacing the periodogram estimate in the Whittle approach by a high-order AR spectral density estimate. Since L ²-norm approximation is a difficult task, we replace it by a modification of a recent model approximation technique called balanced model reduction. By an example, we show that this technique gives almost efficient ARMA estimates without the use of numerical optimization routines.     

RECURSIVE GENERALIZED M ESTIMATES FOR AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. Outliers in time series seriously affect conventional parameter estimates. In this paper a robust recursive estimation procedure for the parameters of auto-regressve moving-average models with additive outliers is proposed. Using 'cleaned' residuals from an initial robust fit of an autoregression of high order as input, bounded influence regression is applied recursively. The proposal follows certain ideas of Hannan and Rissanen, who suggested a three-stage procedure for order and parameter estimation in a conventional setting.
A Monte Carlo study is performed to investigate the robustness properties of the proposed class of estimates and to compare them with various other suggestions, including least squares, M estimates, residual autocovariance and truncated residual autocovariance estimates. The results show that the recursive generalized M estimates compare favourably with them. Finally, possible modifications to master even vigourous situations are suggested.     

A NOTE ON SQUARE ROOT FILTERING FOR VECTOR AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. A simplified version of the square root Kalman filter is obtained for a vector autoregressive moving-average (VARMA) model. The algorithm is computationally more efficient that the standard square root algorithm and its output can be used to compute the likelihood of a VARMA model accurately.     

FORECASTING OF MULTIVARIATE PERIODIC AUTOREGRESSIVE MOVING-AVERAGE PROCESSES

Abstract. Minimum mean square error forecasting of multivariate autoregressive moving-average processes with periodically varying parameters and orders is considered. General expressions are obtained for the forecasts, their errors and the covariance matrices of the forecast errors. Recursive evaluations of these quantities are shown to follow from the conditional expectation approach. Prediction ellipsoids and intervals for future values of the process are given. Update equations for the forecasts are obtained. The general results are illustrated and verified for a particular case of the process. A simulated example is given.     

A LINEAR ESTIMATION PROCEDURE FOR THE PARAMETERS OF AUTOREGRESSIVE MOVING-AVERAGE PROCESSES

Abstract. A linear estimation procedure for the parameters of autoregressive moving-average processes is proposed. The basic idea is to write the spectrum for the moving-average part as a linear function of a properly selected set of parameters and to use Chiu's weighted least-squares procedure to reduce the problem to a weighted linear least-squares problem. The proposed procedure finds estimates by solving systems of linear equations and does not need optimization programs. An one-step estimate is also suggested. It is shown that the estimates are asymptotically equal to the commonly used 'approximate' maximum likelihood estimate described in the paper. For Gaussian processes, the estimates obtained by the proposed procedures are asymptotically efficient.     

ESTIMATION OF AUTOREGRESSIVE MOVING-AVERAGE ORDER GIVEN AN INFINITE NUMBER OF MODELS AND APPROXIMATION OF SPECTRAL DENSITIES

Abstract. A modification of the minimum Akaike information criterion (AIC) procedure (and of related procedures like the Bayesian information criterion (BIC)) for order estimation in autoregressive moving-average (ARMA) models is introduced. This procedure has the advantage that consistency for the order estimators obtained via this procedure can be established without restricting attention to only a finite number of models. The behaviour of these newly introduced order estimators is also analysed for the case when the data-generating process is not an ARMA process (transfer function/spectral density approximation). Furthermore, the behaviour of the order estimators obtained via minimization of BIC (or of related criteria) is investigated for a non-ARMA data-generating process.     

THE EFFECT OF AGGREGATION ON PREDICTION IN AUTOREGRESSIVE INTEGRATED MOVING-AVERAGE MODELS

Abstract. Let x _t be a time series generated by an autoregressive integrated moving-average process ARIMA( p, d, q ). The non-overlapping aggregate series also follows an ARIMA process. Thus, the prediction of the aggregated observations could be done by either the disaggregate model or the aggregate model. We derive the efficiency of the predictors for two important disaggregate models, ARIMA(0, 1, 1) and ARIMA(0, 2, 2), when the models are assumed known. When the models are not known we estimate the efficiency through simulation with the models being selected using Akaike's information criterion.     

A GOODNESS-OF-FIT TEST FOR AUTOREGRESSIVE MOVING-AVERAGE MODELS BASED ON THE STANDARDIZED SAMPLE SPECTRAL DISTRIBUTION OF THE RESIDUALS

Abstract. A stochastic process derived from the standardized sample spectral density of the residuals of a causal and invertible ARMA( p, q ) model is introduced to construct a goodness-of-fit procedure. The test statistics considered have a proper limiting distribution which is free of unknown parameters and which, unlike some well-known goodness-of-fit statistics based on the residuals, does not depend on the sample size.     

A GENERALIZED LEAST-SQUARES APPROACH FOR ESTIMATION OF AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. In this paper we present a generalized least-squares approach for estimating autoregressive moving-average (ARMA) models. Simulation results based on different model structures with varying numbers of observations are used to contrast the performance of our procedure with that of maximum likelihood estimates. Existing software packages can be utilized to derive these estimates.     

ESTIMATION FOR REGRESSIVE AND AUTOREGRESSIVE MODELS WITH NON-NEGATIVE RESIDUAL ERRORS

Abstract. The parameter estimation problems for regressive and autoregressive models are investigated. A new procedure is proposed which differs from the least squares method. Theorems relating to the rate of almost sure convergence of the new estimators are formulated. Some simulation results are also shown. With these convergent rates and simulation results a clear comparison of the new estimator with the least squares estimator is obtained.     

ON THE ASYMPTOTIC DISTRIBUTION OF THE GENERALIZED PARTIAL AUTOCORRELATION FUNCTION IN AUTOREGRESSIVE MOVING-AVERAGE PROCESSES

Abstract. It has been conjectured and illustrated that the estimate of the generalized partial autocorrelation function (GPAC), which has been used for the identification of autoregressive moving-average (ARMA) models, has a thick-tailed asymptotic distribution. The purpose of this paper is to investigate the asymptotic behaviour of the GPAC in detail. It will be shown that the GPAC can be represented as a ratio of two functions, known as the θ function and the Λ function, each of which itself has a useful pattern for ARMA model identification. We shall show the consistencies of the extended Yule-Walker estimates of the three functions and present their asymptotic distributions.     

THE PREDICTIVE PERFORMANCE OF THREE AUTOREGRESSIVE MOVING-AVERAGE MODELS:A MONTE CARLO INVESTIGATION

Abstract. The relative accuracy of point and interval forecasts from three related autoregressive moving-average (ARMA) models—multivariate, univariate, and transfer function—is evaluated in this study. It is found that the multivariate models produce the most accurate one- and three-step-ahead point forecasts of nonindependent series. However, the most accurate point forecasts of independent series are generated by the univariate models. Compared with the multivariate models, the transfer function predictions are relatively unreliable, but with the appropriate restrictions they are superior to the univariate forecasts in certain cases. Interval forecasts from the correctly specified models are reliable indicators of forecast dispersion.     

A STUDY OF THE APPLICATION OF STATE-DEPENDENT MODELS IN NON-LINEAR TIME SERIES ANALYSIS

Abstract. The theory of state-dependent models was developed by Priestley (1980), and a few simple applications were given in Priestley (1981). In this paper, an extensive study of the application of state-dependent models to a wide variety of non-linear time series data is carried out. Both real and simulated data are used in the study, and the problems encountered are highlighted. The method is demonstrated to be successful in practice in many cases, and suggestions for the further development of the algorithm are also given.     

A NOTE ON ASYMPTOTIC INFERENCE IN AUTOREGRESSIVE MODELS WITH ROOTS ON THE UNIT CIRCLE

Abstract. An estimation and inference procedure is proposed for parameters of the p th order autoregressive model with roots both on the unit circle and outside the unit circle. The procedure is motivated by the fact that the parameter estimates of the nonstationary part of the model have higher order consistency properties than the parameter estimates of the stationary part. The procedure allows the use of the known asymptotic distributional results of purely nonstationary models and purely stationary models. Only ordinary least squares routines are needed.     

ORDER IDENTIFICATION STATISTICS IN STATIONARY AUTOREGRESSIVE MOVING-AVERAGE MODELS:VECTOR AUTOCORRELATIONS AND THE BOOTSTRAP

Abstract. In this paper we consider the vector autocorrelation approach for identifying ARMA ( p, q ) models and use a bootstrap procedure in order to evaluate the distribution of the corresponding sample statistics by means of a resampling scheme for the residuals when p and q are unknown. The asymptotic validity of the bootstrap procedure applied to the vector autocorrelation estimates is established. Some simulations and examples demonstrating the appropriateness of the proposed bootstrap procedure in comparison with large-sample Gaussian approximations are included.     

REVERSED RESIDUALS IN AUTOREGRESSIVE TIME SERIES ANALYSIS

Abstract. Both linear and non-linear time series can have directional features which can be used to enhance the modelling and investigation of linear or non-linear autoregressive statistical models. For this purpose, reversed p th-order residuals are introduced. Cross-correlations of residuals and squared reversed residuals allow extensions of current model identification ideas. Quadratic types of partial autocorrelation functions are introduced to assess dependence associated with non-linear models which nevertheless have linear autoregressive correlation structures. The use of these residuals and their cross-correlation functions is exemplified empirically on some deseasonalized river flow data for which a first-order autoregressive model is a satisfactory second-order fit. Parallel theoretical computations are undertaken for the non-linear first-order random coefficient autoregressive model and comparisons are made. While the data are shown to be strongly non-linear, their correlational signatures are found to be convincingly different from those of a first-order autoregressive model with random coefficients.     

ON THE PARTIAL SUMS OF RESIDUALS IN AUTOREGRESSIVE AND MOVING AVERAGE MODELS

Abstract. The limiting process of partial sums of residuals in stationary and invertible autoregressive moving-average models is studied. It is shown that the partial sums converge to a standard Brownian motion under the assumptions that estimators of unknown parameters are root- n consistent and that innovations are independent and identically distributed random variables with zero mean and finite variance or, more generally, are martingale differences with moment restrictions specified in Theorem 1. Applications for goodness-of-fit and change-point problems are considered. The use of residuals for constructing nonparametric density estimation is discussed.     

ON THE SELECTION OF SUBSET AUTOREGRESSIVE TIME SERIES MODELS

Abstract. The estimation of subset autoregressive time series models has been a difficult problem because of the large number of possible alternative models involved. However, with the advent of model selection criteria based on the maximum likelihood, subset model fitting has become feasible. Using an efficient technique for evaluating the residual variance of all possible subset models, a method is proposed for the fitting of subset autoregressive models. The application of the method is illustrated by means of real and simulated data.