Bootstrap predictive inference for ARIMA processes

Abstract.  In this study, we propose a new bootstrap strategy to obtain prediction intervals for autoregressive integrated moving-average processes. Its main advantage over other bootstrap methods previously proposed for autoregressive integrated processes is that variability due to parameter estimation can be incorporated into prediction intervals without requiring the backward representation of the process. Consequently, the procedure is very flexible and can be extended to processes even if their backward representation is not available. Furthermore, its implementation is very simple. The asymptotic properties of the bootstrap prediction densities are obtained. Extensive finite-sample Monte Carlo experiments are carried out to compare the performance of the proposed strategy vs. alternative procedures. The behaviour of our proposal equals or outperforms the alternatives in most of the cases. Furthermore, our bootstrap strategy is also applied for the first time to obtain the prediction density of processes with moving-average components.     

BOOTSTRAPPING STATIONARY AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. In this paper we develop an asymptotic theory for application of the bootstrap to stationary stochastic processes of autoregressive moving-average (ARMA) type, with known order ( p, q ). We give a proof of the asymptotic validity of the bootstrap proposal applied to M estimators for the unknown parameter vector of the process. For this purpose we derive an asymptotic expansion for M estimators in ARMA models and construct an estimate for the unknown distribution function of the residuals which in principle are not observable. A small simulation study is also included.     

Oracle M‐Estimation for Time Series Models

We propose a thresholding M‐estimator for multivariate time series. Our proposed estimator has the oracle property that its large‐sample properties are the same as of the classical M‐estimator obtained under the a priori information that the zero parameters were known. We study the consistency of the standard block bootstrap, the centred block bootstrap and the empirical likelihood block bootstrap distributions of the proposed M‐estimator. We develop automatic selection procedures for the thresholding parameter and for the block length of the bootstrap methods. We present the results of a simulation study of the proposed methods for a sparse vector autoregressive VAR(2) time series model. The analysis of two real‐world data sets illustrate applications of the methods in practice.     

Bootstrapping the Local Periodogram of Locally Stationary Processes

Abstract. Locally stationary processes are non‐stationary stochastic processes the second‐order structure of which varies smoothly over time. In this paper, we develop a method to bootstrap the local periodogram of a locally stationary process. Our method generates pseudo local periodogram ordinates by combining a parametric time and non‐parametric frequency domain bootstrap approach. We first fit locally a time varying autoregressive model so as to capture the essential characteristics of the underlying process. A locally calculated non‐parametric correction in the frequency domain is then used so as to improve upon the locally parametric autoregressive fit. As an application, we investigate theoretically the asymptotic properties of the bootstrap method proposed applied to the class of local spectral means, local ratio statistics and local spectral density estimators. Some simulations demonstrate the ability of our method to give accurate estimates of the quantities of interest in finite sample situations and an application to a real‐life data‐set is presented.     

Testing for parameter stability in nonlinear autoregressive models

In this article we develop testing procedures for the detection of structural changes in nonlinear autoregressive processes. For the detection procedure, we model the regression function by a single layer feedforward neural network. We show that CUSUM‐type tests based on cumulative sums of estimated residuals, that have been intensively studied for linear regression, can be extended to this case. The limit distribution under the null hypothesis is obtained, which is needed to construct asymptotic tests. For a large class of alternatives, it is shown that the tests have asymptotic power one. In this case, we obtain a consistent change‐point estimator which is related to the test statistics. Power and size are further investigated in a small simulation study with a particular emphasis on situations where the model is misspecified, i.e. the data is not generated by a neural network but some other regression function. As illustration, an application on the Nile data set as well as S&P log‐returns is given.     

Properties of the nonparametric autoregressive bootstrap

For nonparametric autoregression, we investigate a model based bootstrap procedure (`autoregressive bootstrap') that mimics the complete dependence structure of the original time series. We give consistency results for uniform bootstrap confidence bands of the autoregression function based on kernel estimates of the autoregression function. This result is achieved by global strong approximations of the kernel estimates for the resample and for the original sample. Furthermore, it is obtained that the autoregressive bootstrap also yields asymptotically correct approximations for distributions of parametric statistics, for which regression-type bootstrap-techniques like the wild bootstrap do not work. For this purpose, we prove geometric ergodicity and absolute regularity of the nonparametric autoregressive bootstrap process. We propose some particular estimators of the autoregression function and of the density of the innovations such that the mixing coefficients of the autoregressive bootstrap process can be bounded uniformly by some exponentially decaying sequence. This is achieved by using well-established coupling techniques. Moreover, by using some `decoupling' argument, we show that the stationary density of the bootstrap process converges to that of the original process. The paper may serve as a template for proving similar consistency results for other bootstrap techniques such as the Markov bootstrap.     

ON BOOTSTRAP PREDICTIVE INFERENCE FOR AUTOREGRESSIVE PROCESSES

Abstract. In this paper we consider bootstrap-based predictive inference for autoregressive processes of order p. We consider both unconditional inference and inference conditional on the last p observed values. We make two contributions. Our first contribution is to point out the best way to apply the bootstrap to unconditional predictive inference when the process is Gaussian. Now, it may be argued that predictive inference for autoregressive processes of order p should be carried out conditional on the last p observed values. When the process is Gaussian, a bootstrap predictive inference conditional on the last p observed values is conveniently computed by 'running' the same autoregressive process backwards in time. This procedure is inappropriate for non-Gaussian autoregressive processes. Our second (and more important) contribution is to present a method (which is not computationally burdensome) for the computation of a bootstrap predictive inference for a non-Gaussian autoregressive process of order p conditional on the last p observed values.     

Bootstrap Unit‐Root Tests: Comparison and Extensions

Abstract. In this article, we study and compare the properties of several bootstrap unit‐root tests recently proposed in the literature. The tests are Dickey–Fuller (DF) or Augmented DF, based either on residuals from an autoregression and the use of the block bootstrap or on first‐differenced data and the use of the stationary bootstrap or sieve bootstrap. We extend the analysis by interchanging the data transformations (differences vs. residuals), the types of bootstrap and the presence or absence of a correction for autocorrelation in the tests. We show that two sieve bootstrap tests based on residuals remain asymptotically valid. In contrast to the literature which focuses on a comparison of the bootstrap tests with an asymptotic test, we compare the bootstrap tests among themselves using response surfaces for their size and power in a simulation study. This study leads to the following conclusions: (i) augmented DF tests are always preferred to standard DF tests; (ii) the sieve bootstrap performs better than the block bootstrap; (iii) difference‐based tests appear to have slightly better size properties, but residual‐based tests appear more powerful.     

High Moment Partial Sum Processes of Residuals in ARMA Models and their Applications

Abstract. In this article, we study high moment partial sum processes based on residuals of a stationary autoregressive moving average (ARMA) model with known or unknown mean parameter. We show that they can be approximated in probability by the analogous processes which are obtained from the i.i.d. errors of the ARMA model. However, if a unknown mean parameter is used, there will be an additional term that depends on model parameters and a mean estimator. When properly normalized, this additional term will vanish. Thus the processes converge weakly to the same Gaussian processes as if the residuals were i.i.d. Applications to change‐point problems and goodness‐of‐fit are considered, in particular, cumulative sum statistics for testing ARMA model structure changes and the Jarque–Bera omnibus statistic for testing normality of the unobservable error distribution of an ARMA model.     

Residual Empirical Processes and Weighted Sums for Time‐Varying Processes with Applications to Testing for Homoscedasticity

In the context of heteroscedastic time‐varying autoregressive (AR)‐process we study the estimation of the error/innovation distributions. Our study reveals that the non‐parametric estimation of the AR parameter functions has a negligible asymptotic effect on the estimation of the empirical distribution of the residuals even though the AR parameter functions are estimated non‐parametrically. The derivation of these results involves the study of both function‐indexed sequential residual empirical processes and weighted sum processes. Exponential inequalities and weak convergence results are derived. As an application of our results we discuss testing for the constancy of the variance function, which in special cases corresponds to testing for stationarity.     

A GENERALIZED BLOCK BOOTSTRAP FOR SEASONAL TIME SERIES

When time‐series data contain a periodic/seasonal component, the usual block bootstrap procedures are not directly applicable. We propose a modification of the block bootstrap – the generalized seasonal block bootstrap (GSBB) – and show its asymptotic consistency without undue restrictions on the relative size of the period and block size. Notably, it is exactly such restrictions that limit the applicability of other proposals of block bootstrap methods for time series with periodicities. The finite‐sample performance of the GSBB is also illustrated by means of a small simulation experiment.     

A Sieve Bootstrap For The Test Of A Unit Root

Abstract. In this paper, we consider a sieve bootstrap for the test of a unit root in models driven by general linear processes. The given model is first approximated by a finite autoregressive integrated process of order increasing with the sample size, and then the method of bootstrap is applied for the approximated autoregression to obtain the critical values for the usual unit root tests. The resulting tests, which may simply be viewed as the bootstrapped versions of Augmented Dickey–Fuller (ADF) unit root tests by Said and Dickey (1984 ), are shown to be consistent under very general conditions. The asymptotic validity of the bootstrap ADF unit root tests is thus established. Our conditions are significantly weaker than those used by Said and Dickey. Simulations show that bootstrap provides substantial improvements on finite sample sizes of the tests.     

Stationary bootstrapping for non‐parametric estimator of nonlinear autoregressive model

We consider stationary bootstrap approximation of the non‐parametric kernel estimator in a general kth‐order nonlinear autoregressive model under the conditions ensuring that the nonlinear autoregressive process is a geometrically Harris ergodic stationary Markov process. We show that the stationary bootstrap procedure properly estimates the distribution of the non‐parametric kernel estimator. A simulation study is provided to illustrate the theory and to construct confidence intervals, which compares the proposed method favorably with some other bootstrap methods.     

A Note on Bootstrapping M-Estimators in ARMA Models

Kreiss and Franke (Bootstrapping stationary autoregressive moving-average models. J. Time Ser. Anal. 13 (1992), 297–317) proposed bootstrapping a linear approximation to the M-estimator in autoregressive moving-average (ARMA) models. In this paper, it is argued that it may be better to apply the bootstrap principle directly to the M-estimator itself. A number of simulation results are presented to compare the two procedures for estimating the sampling distribution of an M-estimator. The theoretical asymptotic validity of the standard bootstrap applied to the M-estimator is established.     

Bootstrapping unit root tests for integrated processes

Abstract. In this paper, we consider two bootstrap algorithms for testing unit roots under the condition that the observed process is unit root integrated. The first method consists of generating the resampled data after fitting an autoregressive model to the first differences of the observations. The second method consists of applying the stationary bootstrap to the first differences. Both procedures are shown to give methods that approach the correct asymptotic distribution under the null hypothesis of a unit root. We also present a Monte-Carlo study comparing the two methods for some ARIMA models.     

On the Consistency of Bootstrap Testing for a Parameter on the Boundary of the Parameter Space

It is well known that with a parameter on the boundary of the parameter space, such as in the classic cases of testing for a zero location parameter or no autoregressive conditional heteroskedasticity (ARCH) effects, the classic nonparametric bootstrap – based on unrestricted parameter estimates – leads to inconsistent testing. In contrast, we show here that for the two aforementioned cases, a nonparametric bootstrap test based on parameter estimates obtained under the null – referred to as ‘restricted bootstrap’ – is indeed consistent. While the restricted bootstrap is simple to implement in practice, novel theoretical arguments are required in order to establish consistency. In particular, since the bootstrap is analysed both under the null hypothesis and under the alternative, non‐standard asymptotic expansions are required to deal with parameters on the boundary. Detailed proofs of the asymptotic validity of the restricted bootstrap are given and, for the leading case of testing for no ARCH, a Monte Carlo study demonstrates that the bootstrap quasi‐likelihood ratio statistic performs extremely well in terms of empirical size and power for even remarkably small samples, outperforming the standard and bootstrap Lagrange multiplier tests as well as the asymptotic quasi‐likelihood ratio test.     

Bootstrap Tests for an Autoregressive Unit Root in the Presence of Weakly Dependent Errors

This paper examines bootstrap tests of the null hypothesis of an autoregressive unit root in models that may include a linear rend and/or an intercept and which are driven by innovations that belong to the class of stationary and invertible linear processes. Our approach makes use of a sieve bootstrap procedure based on residual resampling from autoregressive approximations, the order of which increases with the sample size at a suitable rate. We show that the sieve bootstrap provides asymptotically valid tests of the unit-root hypothesis and demonstrate the small-sample effectiveness of the method by means of simulation.     

Some Nonparametric Tests for Change-Point Detection Based on the ℙ-ℙ and ℚ-ℚ Plot Processes

Abstract

We propose nonparametric procedures for testing change-point by using the ?-? and ?-? plots processes. The limiting distributions of the proposed statistics are characterized under the null hypothesis of no change and also under contiguous alternatives. We give an estimator of the change-point coefficient and obtain its strong consistency. We introduce the bootstrapped version of ?-? and ?-? processes, requiring the estimation of quantile density, and obtain their limiting laws. Finally, we propose and investigate the exchangeable bootstrap of the empirical ?-? plot and ?-? plot processes which avoids the problem of the estimation of quantile density, which is of its own interest. These results are used for calculating p-values of the proposed test statistics. Emphasis is placed on the explanation of the strong approximation methodology.     

A note on non‐parametric testing for Gaussian innovations in AR–ARCH models

In this paper, we consider autoregressive models with conditional autoregressive variance, including the case of homoscedastic AR models and the case of ARCH models. Our aim is to test the hypothesis of normality for the innovations in a completely non‐parametric way, that is, without imposing parametric assumptions on the conditional mean and volatility functions. To this end, the Cramér–von Mises test based on the empirical distribution function of non‐parametrically estimated residuals is shown to be asymptotically distribution‐free. We demonstrate its good performance for finite sample sizes in a small simulation study. AMS 2010 Classification: Primary 62 M10, Secondary 62 G10     

Change‐Point Detection in Autoregressive Models with no Moment Assumptions

In this paper we consider the problem of detecting a change in the parameters of an autoregressive process where the moments of the innovation process do not necessarily exist. An empirical likelihood ratio test for the existence of a change point is proposed and its asymptotic properties are studied. In contrast to other works on change‐point tests using empirical likelihood, we do not assume knowledge of the location of the change point. In particular, we prove that the maximizer of the empirical likelihood is a consistent estimator for the parameters of the autoregressive model in the case of no change point and derive the limiting distribution of the corresponding test statistic under the null hypothesis. We also establish consistency of the new test. A nice feature of the method is the fact that the resulting test is asymptotically distribution‐free and does not require an estimate of the long‐run variance. The asymptotic properties of the test are investigated by means of a small simulation study, which demonstrates good finite‐sample properties of the proposed method.