Properties of the nonparametric autoregressive bootstrap

For nonparametric autoregression, we investigate a model based bootstrap procedure (`autoregressive bootstrap') that mimics the complete dependence structure of the original time series. We give consistency results for uniform bootstrap confidence bands of the autoregression function based on kernel estimates of the autoregression function. This result is achieved by global strong approximations of the kernel estimates for the resample and for the original sample. Furthermore, it is obtained that the autoregressive bootstrap also yields asymptotically correct approximations for distributions of parametric statistics, for which regression-type bootstrap-techniques like the wild bootstrap do not work. For this purpose, we prove geometric ergodicity and absolute regularity of the nonparametric autoregressive bootstrap process. We propose some particular estimators of the autoregression function and of the density of the innovations such that the mixing coefficients of the autoregressive bootstrap process can be bounded uniformly by some exponentially decaying sequence. This is achieved by using well-established coupling techniques. Moreover, by using some `decoupling' argument, we show that the stationary density of the bootstrap process converges to that of the original process. The paper may serve as a template for proving similar consistency results for other bootstrap techniques such as the Markov bootstrap.     

Large Sample Properties of Parameter Estimates for Periodic ARMA Models

This paper studies the asymptotic properties of parameter estimates for causal and invertible periodic autoregressive moving-average (PARMA) time series models. A general limit result for PARMA parameter estimates with a moving-average component is derived. The paper presents examples that explicitly identify the limiting covariance matrix for parameter estimates from a general periodic autoregression (PAR), a first-order periodic moving average (PMA(1)), and the mixed PARMA(1,1) model. Some comparisons and contrasts to univariate and vector autoregressive moving-average sequences are made.     

ESTIMATION OF THE ORDER OF A MOVING AVERAGE MODEL FROM AUTOREGRESSIVE AND WINDOW ESTIMATES OF THE INVERSE CORRELATION FUNCTION

Abstract. The autoregressive and window estimates of the inverse correlation function are used for estimating the order of a finite moving average process by using criteria similar to the FPEα criterion of Bhansali and Downham (1977). The asymptotic distribution of the estimates is derived. Their finite sample behaviour is examined by means of a simulation study.     

Stationary bootstrapping for non‐parametric estimator of nonlinear autoregressive model

We consider stationary bootstrap approximation of the non‐parametric kernel estimator in a general kth‐order nonlinear autoregressive model under the conditions ensuring that the nonlinear autoregressive process is a geometrically Harris ergodic stationary Markov process. We show that the stationary bootstrap procedure properly estimates the distribution of the non‐parametric kernel estimator. A simulation study is provided to illustrate the theory and to construct confidence intervals, which compares the proposed method favorably with some other bootstrap methods.     

SIMULTANEOUS CONFIDENCE BANDS FOR THE SPECTRAL ESTIMATE OF TWO-CHANNEL AUTOREGRESSIVE PROCESSES

Abstract. In this paper we derive simultaneous confidence bands for the maximum entropy method spectral estimate of two-channel autoregressive (AR) processes by using the asymptotic theory of the estimation of periodic AR processes.     

Empirical likelihood inference for random coefficient INAR(p) process

In this article, we study the empirical likelihood (EL) method for the pth‐order random coefficient integer‐valued autoregressive process. In particular, the limiting distribution of the log EL ratio statistic is established and the confidence regions for the parameter of interest are derived. Also a simulation study is conducted for the evaluation of the developed approach.     

CONFIDENCE REGIONS FOR PARAMETERS IN THE AR(1) MODEL

Abstract. The construction of approximate joint and marginal confidence regions for parameters in the first-order autoregressive time series model is discussed. These regions are based on the large sample distributions of the likelihood ratio (and approximations to it), of the maximum likelihood estimates and of the score statistics. All these approaches are illustrated using a well-known example from Box and Jenkins (Time Series Analysis:Forecasting and Control, revised edn. San Francisco:Holden Day, 1976) and some simulated series. In addition, a simulation study is provided for comparing the coverage properties of the various procedures.     

ASYMPTOTIC INFERENCE FOR NON-INVERTIBLE MOVING-AVERAGE TIME SERIES

Abstract. This paper is concerned with statistical inference of nonstationary and non-invertible autoregressive moving-average (ARMA) processes. It makes use of the fact that a derived process of an ARMA( p, q ) model follows an AR( q ) model with an autoregressive (AR) operator equivalent to the moving-average (MA) part of the original ARMA model. Asymptotic distributions of least squares estimates of MA parameters based on a constructed derived process are obtained as corresponding analogs of a nonstationary AR process. Extensions to the nearly non-invertible models are considered and the limiting distributions are obtained as functionals of stochastic integrals of Brownian motions and Ornstein-Uhlenbeck processes. For application, a two-stage procedure is proposed for testing unit roots in the MA polynomial. Examples are given to illustrate the application.     

A UNIFIED APPROACH TO CONFIDENCE BOUNDS FOR THE AUTOREGRESSIVE SPECTRAL ESTIMATOR

Abstract. Confidence bounds for the spectral density of a stationary time series are derived. A unified method begins with the autoregressive spectral estimate and produces both confidence intervals at single frequencies chosen a priori and a simultaneous confidence band for multiple a posteriori comparisons. The crux is optimization of a quadratic form subject to the constraint imposed by the F -statistic. An approximate method that may produce tighter bounds is described. The former methods are demonstrated on the Waldmeier annual mean sunspot numbers.     

ESTIMATION OF AUTOREGRESSIVE PARAMETERS AND ORDER SELECTION FOR ARMA MODELS

Abstract. A quick algorithm for obtaining estimates of autoregressive parameters for autoregressive moving-average model is presented. The algorithm is recursive in the orders, and can be used for model selection by providing a criterion and a two-way table of certain partial covariances. Consistency and asymptotic normality of the estimates are shown.     

ON THE ESTIMATION OF THE INVERSE CORRELATION FUNCTION

Two new methods for estimating the inverse covariance and inverse correlation functions of a time series are proposed. One of them is based on an orthogonality property, the other is suggested by interpolation considerations. The two methods are shown to be asymptotically equivalent, and their asymptotic distribution is derived. The asymptotic distribution turns out to be the same as that of the autoregressive estimates of the inverse correlations. The problem of choosing an estimation method in practice is discussed.     

Rate of convergence in the central limit theorem for parameter estimation in a causal,invertible ARMA(p,q) model

In this study we consider the estimators of the parameters of a stable ARMA(p, q) process. The autoregressive parameters are estimated by the instrumental variable technique while the moving average parameters are estimated using a derived autoregressive process. The estimators are shown to be asymptotically normal and their rate of convergence to normality is derived.     

On Residual Variance Estimation in Autoregressive Models

In this paper we consider time series models belonging to the autoregressive (AR) family and deal with the estimation of the residual variance. This is important because estimates of the variance are involved in, for example, confidence sets for the parameters of the model, estimation of the spectrum, expressions for the estimated error of prediction and sample quantities used to make inferences about the order of the model. We consider the asymptotic biases for moment and least squares estimators of the residual variance, and compare them with known results when available and with those for maximum likelihood estimators under normality. Simulation results are presented for finite samples     

On the Optimal Segment Length for Parameter Estimates for Locally Stationary Time Series

We discuss the behaviour of parameter estimates when stationary time series models are fitted locally to non-stationary processes which have an evolutionary spectral representation. A particular example is the estimation for an autoregressive process with time-varying coefficients by local Yule–Walker estimates. The bias and the mean squared error for the parameter estimates are calculated and the optimal length of the data segment is determined.     

THE ESTIMATION OF RANDOM COEFFICIENT AUTOREGRESSIVE MODELS. I

Abstract. This paper is concerned with autoregressive models in which the coefficients are assumed to be not constant but subject to random perturbations so that we are considering a class of random coefficient autoregressive models. By means of a two stage regression procedure estimates of the unknown parameters of these models are obtained. The estimates are shown to be strongly consistent and to satisfy a central limit theorem. A number of Monte Carlo experiments was carried out to illustrate the estimation procedure and their results are reported.     

A BAYESIAN APPROACH TO ESTIMATING AND FORECASTING ADDITIVE NONPARAMETRIC AUTOREGRESSIVE MODELS

Abstract. We present a Bayesian approach for estimating nonparametrically an additive autoregressive model with the regression curve estimates cubic smoothing splines. Our approach is robust to innovation outliers; it can handle missing observations and produce multistep ahead forecasts. The computation is carried out using Markov chain Monte Carlo and requires O( nM ) operations where n is the sample size and M is the number of Markov chain iterations. This makes it the first exact algorithm for spline smoothing of an additive autoregressive model which can handle large data sets. The properties of the estimates and forecasts are studied empirically using simulated and real data sets.     

EXPERIENCES WITH THE BRILLINGER SPECTRAL ESTIMATOR APPLIED TO SIMULATED IRREGULARLY OBSERVED PROCESSES

Abstract. Shannon interpolation is used to assign values from a readily simulated discrete time process to the times of a point process, simulated by Ogata's thinning technique. The result is a set of unequally spaced samples from a hypothetical continuous time process with spectrum equal to that of the discrete time process for frequencies |ω| ≤π/Δ and identically equal to zero for |ω| > π/Δ, where Δ is the discrete time step. The spectra are theoretically known both for the sampled process and for the sampling point process. We calculate Brillinger spectral estimates for examples of a process with autoregressive spectrum, sampled at the times of a Hawkes Self Exciting Point Process. The success of the Brillinger estimator is demonstrated but it is shown to have an inherently high variance. An approximate confidence interval is discussed.     

Consistent estimation and order selection for nonstationary autoregressive processes with stable innovations

Abstract. A possibly nonstationary autoregressive process, of unknown finite order, with possibly infinite‐variance innovations is studied. The ordinary least squares autoregressive parameter estimates are shown to be consistent, and their rate of convergence, which depends on the index of stability, α, is established. We also establish consistency of lag‐order selection criteria in the nonstationary case. A small experiment illustrates the relative performance of different lag‐length selection criteria in finite samples.     

ON THE STRUCTURE OF THE LIKELIHOOD FUNCTION OF AUTOREGRESSIVE AND MOVING AVERAGE MODELS

Abstract. In finite order normal moving average models the maximum likelihood estimates always exist. For finite order normal autoregressive models sufficient conditions for the existence of maximum likelihood estimates is given. Some cases not satisfying the conditions are studied.     

SYMMETRIC STABLE SEQUENCES WITH MISSING OBSERVATIONS

Abstract. A procedure for evaluating optimal linear estimates of missing values in the minimum dispersion sense is proposed for stationary symmetric stable processes. Analytical expressions for the estimates are obtained for the autoregressive moving-average process and it is shown that the finite variance setting results are special cases. Cases of one and more than one missing value are considered.