Bootstrapping confidence intervals for the change‐point of time series

Abstract. We study an at‐most‐one‐change time‐series model with an abrupt change in the mean and dependent errors that fulfil certain mixing conditions. We obtain confidence intervals for the unknown change‐point via bootstrapping methods. Precisely, we use a block bootstrap of the estimated centred error sequence. Then, we reconstruct a sequence with a change in the mean using the same estimators as before. The difference between the change‐point estimator of the resampled sequence and the one of the original sequence can be used as an approximation of the difference between the real change‐point and its estimator. This enables us to construct confidence intervals using the empirical distribution of the resampled time series. A simulation study shows that the resampled confidence intervals are usually closer to their target levels and at the same time smaller than the asymptotic intervals.     

Bootstrapping Time Series Regressions with Integrated Processes

This paper studies the bootstrap procedures for time series regressions with integrated processes. Both estimation and hypothesis testing are studied. It is shown that the suggested bootstrap approximations to the distribution of the least squares estimator and the regression test statistic are asymptotically valid. A Monte Carlo experiment is conducted to evaluate the finite sample performance of these bootstrap procedures. The simulation results indicate that the bootstrap method provides reasonably good approximation to the distribution of the least squares estimator, and gives proper size and satisfactory power.     

Oracle M‐Estimation for Time Series Models

We propose a thresholding M‐estimator for multivariate time series. Our proposed estimator has the oracle property that its large‐sample properties are the same as of the classical M‐estimator obtained under the a priori information that the zero parameters were known. We study the consistency of the standard block bootstrap, the centred block bootstrap and the empirical likelihood block bootstrap distributions of the proposed M‐estimator. We develop automatic selection procedures for the thresholding parameter and for the block length of the bootstrap methods. We present the results of a simulation study of the proposed methods for a sparse vector autoregressive VAR(2) time series model. The analysis of two real‐world data sets illustrate applications of the methods in practice.     

A Note on Bootstrapping M-Estimators in ARMA Models

Kreiss and Franke (Bootstrapping stationary autoregressive moving-average models. J. Time Ser. Anal. 13 (1992), 297–317) proposed bootstrapping a linear approximation to the M-estimator in autoregressive moving-average (ARMA) models. In this paper, it is argued that it may be better to apply the bootstrap principle directly to the M-estimator itself. A number of simulation results are presented to compare the two procedures for estimating the sampling distribution of an M-estimator. The theoretical asymptotic validity of the standard bootstrap applied to the M-estimator is established.     

Bootstrapping Sequential Change-Point Tests

Abstract

In this article we propose some bootstrapping methods to obtain critical values for sequential change-point tests. We consider a change in the mean with i.i.d. errors. Theoretical results show the asymptotic validity of the proposed bootstrap procedures. A simulation study compares the bootstrap and the asymptotic tests and shows that the studentized bootstrap test behave generally better than asymptotic tests if measured by α- resp. β-errors and its run length.     

Stationary bootstrapping for non‐parametric estimator of nonlinear autoregressive model

We consider stationary bootstrap approximation of the non‐parametric kernel estimator in a general kth‐order nonlinear autoregressive model under the conditions ensuring that the nonlinear autoregressive process is a geometrically Harris ergodic stationary Markov process. We show that the stationary bootstrap procedure properly estimates the distribution of the non‐parametric kernel estimator. A simulation study is provided to illustrate the theory and to construct confidence intervals, which compares the proposed method favorably with some other bootstrap methods.     

Bootstrap Approximation to Prediction MSE for State–Space Models with Estimated Parameters

Abstract. We propose simple parametric and nonparametric bootstrap methods for estimating the prediction mean square error (PMSE) of state vector predictors that use estimated model parameters. As is well known, substituting the model parameters by their estimates in the theoretical PMSE expression that assumes known parameter values results in underestimation of the true PMSE. The parametric method consists of generating parametrically a large number of bootstrap series from the model fitted to the original series, re‐estimating the model parameters for each series using the same method as used for the original series and then estimating the separate components of the PMSE. The nonparametric method generates the series by bootstrapping the standardized innovations estimated for the original series. The bootstrap methods are compared with other methods considered in the literature in a simulation study that also examines the robustness of the various methods to non‐normality of the model error terms. Application of the bootstrap method to a model fitted to employment ratios in the USA that contains 18 unknown parameters, estimated by a three‐step procedure yields unbiased PMSE estimators.     

Bias reduction via resampling for estimation following sequential tests

It is well known that maximum likelihood (ML) estimation results in biased estimates when estimating parameters following a sequential test. Existing bias correction methods rely on explicit calculations of the bias that are often difficult to derive. We suggest a simple alternative to the existing methods. The new approach relies on approximating the bias of the estimate using a bootstrap method. It requires bootstrapping the sequential testing process by resampling observations from a distribution based on the ML estimate. Each bootstrap process will give a new ML estimate, and the corresponding bootstrap mean can be used to calibrate the estimate. An advantage of the new method over the existing methods is that the same procedure can be used under different stopping rules and different study designs. Simulation results suggest that this method performs competitively with existing methods.     

BOOTSTRAP FOR RANDOM COEFFICIENT AUTOREGRESSIVE MODELS

In this paper, we deal with autoregressive processes with random coefficients. We propose a least‐squares estimator for the fourth‐order moments of both the innovation and disturbance noises and state its consistency. The main theme of the paper is the development of bootstrap procedures for the autoregressive parameter. We show how to obtain approximative residuals for the process even though the standard method for autoregressive processes does not work in this context since one then would obtain convoluted residuals of the innovation and disturbance noises. These ideas lead to a modification of the classical residual bootstrap for autoregressive processes. The consistency of the bootstrap procedure is established. Further, the estimators proposed in the first part are used to form two wild bootstrap modifications. Finally, the performances of the three bootstrap procedures are explored by a simulation study and compared with each other.     

Biomass estimation in batch biotechnological processes by Bayesian Gaussian process regression

This paper proposes a biomass concentration estimator for a batch biotechnological process based on Bayesian regression with Gaussian process. On the basis of experimental data, a two-stage bootstrap technique has been developed for the estimator design. In the first stage, the biomass data set was augmented with virtual filtered measurements, and in the second stage, the biomass estimator design was completed. The method provides information on the confidence level of the estimates, and the biomass estimator performances are illustrated for the Bacillus thuringiensis δ-endotoxins production process.     

A decision theoretic approach to change point estimation for binomial CUSUM control charts

Detecting when the process has changed is a classical problem in sequential analysis and is an important practical issue in statistical process control. This article is concerned about the binomial cumulative sum (CUSUM) control chart, which is extensively applied to industrial process control, health care, public health surveillance, and other fields. For the binomial CUSUM, a maximum likelihood estimator has been proposed to estimate the change point. In our article, following a decision theoretic approach, we develop a new estimator that aims to improve the existing methods. For interval estimation, we propose a parametric bootstrap procedure to construct the confidence set of the change point. We compare our proposed method with the maximum likelihood estimator and Page's last zero estimator in terms of mean squared error by simulations. We find that the proposed method gives more unbiased and robust results than the existing procedures under various parameter designs. We analyze jewelry manufacturing data for illustration.     

Banded and tapered estimates for autocovariance matrices and the linear process bootstrap

We address the problem of estimating the autocovariance matrix of a stationary process. Under short range dependence assumptions, convergence rates are established for a gradually tapered version of the sample autocovariance matrix and for its inverse. The proposed estimator is formed by leaving the main diagonals of the sample autocovariance matrix intact while gradually down‐weighting off‐diagonal entries towards zero. In addition, we show the same convergence rates hold for a positive definite version of the estimator, and we introduce a new approach for selecting the banding parameter. The new matrix estimator is shown to perform well theoretically and in simulation studies. As an application, we introduce a new resampling scheme for stationary processes termed the linear process bootstrap (LPB). The LPB is shown to be asymptotically valid for the sample mean and related statistics. The effectiveness of the proposed methods are demonstrated in a simulation study.     

Testing equality of autocovariance operators for functional time series

We consider strictly stationary stochastic processes of Hilbert space-valued random variables and focus on fully functional tests for the equality of the lag-zero autocovariance operators of several independent functional time series. A moving block bootstrap (MBB)-based testing procedure is proposed which generates pseudo random elements that satisfy the null hypothesis of interest. It is based on directly bootstrapping the time series of tensor products which overcomessome common difficulties associated with applications of the bootstrap to related testing problems. The suggested methodology can be potentially applied to a broad range of test statistics of the hypotheses of interest. As an example, we establish validity for approximating the distribution under the null of a test statistic based on the Hilbert–Schmidt distance of the corresponding sample lag-zero autocovariance operators, and show consistency under the alternative. As a prerequisite, we prove a central limit theorem for the MBB procedure applied to the sample autocovariance operator which is of interest on its own. The finite sample size and power performance of the suggested MBB-based testing procedure is illustrated through simulations and an application to a real-life dataset is discussed.     

Inference for the Fourth‐Order Innovation Cumulant in Linear Time Series

The rescaled fourth‐order cumulant of the unobserved innovations of linear time series is an important parameter in statistical inference. This article deals with the problem of estimating this parameter. An existing nonparametric estimator is first discussed, and its asymptotic properties are derived. It is shown how the autocorrelation structure of the underlying process affects the behaviour of the estimator. Based on our findings and on an important invariance property of the parameter of interest with respect to linear filtering, a pre‐whitening‐based nonparametric estimator of the same parameter is proposed. The estimator is obtained using the filtered time series only; that is, an inversion of the pre‐whitening procedure is not required. The asymptotic properties of the new estimator are investigated, and its superiority is established for large classes of stochastic processes. It is shown that for the particular estimation problem considered, pre‐whitening can reduce the variance and the bias of the estimator. The finite sample performance of both estimators is investigated by means of simulations. The new estimator allows for a simple modification of the multiplicative frequency domain bootstrap, which extends its considerable range of validity. Furthermore, the problem of testing hypotheses about the rescaled fourth‐order cumulant of the unobserved innovations is also considered. In this context, a simple test for Gaussianity is proposed. Some real‐life data applications are presented.     

A Nonparametric Test for Weak Dependence Against Strong Cycles and its Bootstrap Analogue

Abstract. We examine a test for the hypothesis of weak dependence against strong cyclical components. We show that the limiting distribution of the test is a Gumbel distribution, denoted G(·). However, since G(·) may be a poor approximation to the finite sample distribution, being the rate of the convergence logarithmic [see Hall Journal of Applied Probability (1979) , Vol. 16, pp. 433–439], inferences based on G(·) may not be very reliable for moderate sample sizes. On the other hand, in a related context, Hall [Probability Theory and Related Fields (1991) , Vol. 89, pp. 447–455] showed that the level of accuracy of the bootstrap is significantly better. For that reason, we describe an approach to bootstrapping the test based on Efron's [Annals of Statistics (1979) , Vol. 7, pp. 1–26] resampling scheme of the data. We show that the bootstrap principle is consistent under very mild regularity conditions.     

Valid Edgeworth Expansions of Some Estimators and Bootstrap Confidence Intervals in First-order Autoregression

We derive the third-order valid Edgeworth expansions for the standardized and the Studentized versions of some estimators in first-order autoregression without Gaussianity. As a special case of a Gaussian process, the validity of the expansion obtained by Ochi (Asymptotic expansions for the distribution of an estimator in the first-order autoregressive process. Journal of Time Ser. Anal. 4 (1983), 57–67) is demonstrated. By applying the second-order Edgeworth expansion to the bootstrap procedure, we construct the confidence intervals for the autoregressive coefficient.     

Improved generalized method of moments estimators for weakly dependent observations

This article introduces a new class of generalized method of moments estimators for weakly dependent observations with auxiliary information. The estimators are based on a tapered version of blocking techniques similar to the tapered block bootstrap introduced by Paparoditis and Politis (2001) , and can efficiently incorporate auxiliary information via a set of weights obtained by the generalized empirical likelihood estimator. Simulations show that the proposed estimators perform well in finite samples, and can be less biased and more precise than other asymptotically equivalent estimators.     

Bootstrapping a weighted linear estimator of the ARCH parameters

Abstract.  A standard assumption while deriving the asymptotic distribution of the quasi maximum likelihood estimator in ARCH models is that all ARCH parameters must be strictly positive. This assumption is also crucial in deriving the limit distribution of appropriate linear estimators (LE). We propose a weighted linear estimator (WLE) of the ARCH parameters in the classical ARCH model and show that its limit distribution is multivariate normal even when some of the ARCH coefficients are zero. The asymptotic dispersion matrix involves unknown quantities. We consider appropriate bootstrapped version of this WLE and prove that it is asymptotically valid in the sense that the bootstrapped distribution (given the data) is a consistent estimate (in probability) of the distribution of the WLE. Although we do not show theoretically that the bootstrap outperforms the normal approximation, our simulations demonstrate that it yields better approximations than the limiting normal.     

Generalized Resampling Scheme With Application to Spectral Density Matrix in Almost Periodically Correlated Class of Time Series

The aim of this article is to introduce new resampling scheme for nonstationary time series, called generalized resampling scheme (GRS). The proposed procedure is a generalization of well known in the literature subsampling procedure and is simply related to existing block bootstrap techniques. To document the usefulness of GRS, we consider the example of model with almost periodic phenomena in mean and variance function, where the consistency of the proposed procedure was examined. Finally, we prove the consistency of GRS for the spectral density matrix for nonstationary, multivariate almost periodically correlated time series. We consider both zero mean and non‐zero mean case. The consistency holds under general assumptions concerning moment and α‐mixing conditions for multivariate almost periodically correlated time series. Proving the consistency in this case poses a difficulty since the estimator of the spectral density matrix can be interpreted as a sum of random matrixes whose dependence grow with the sample size.     

An improved independent component regression modeling and quantitative calibration procedure

An improved independent component regression (M‐ICR) algorithm is proposed by constructing joint latent variable (LV) based regressors, and a quantitative statistical analysis procedure is designed using a bootstrap technique for model validation and performance evaluation. First, the drawbacks of the conventional regression modeling algorithms are analyzed. Then the proposed M‐ICR algorithm is formulated for regressor design. It constructs a dual‐objective optimization criterion function, simultaneously incorporating quality‐relevance and independence into the feature extraction procedure. This ties together the ideas of partial‐least squares (PLS), and independent component regression (ICR) under the same mathematical umbrella. By adjusting the controllable suboptimization objective weights, it adds insight into the different roles of quality‐relevant and independent characteristics in calibration modeling, and, thus, provides possibilities to combine the advantages of PLS and ICR. Furthermore, a quantitative statistical analysis procedure based on a bootstrapping technique is designed to identify the effects of LVs, determine a better model rank and overcome ill‐conditioning caused by model over‐parameterization. A confidence interval on quality prediction is also approximated. The performance of the proposed method is demonstrated using both numerical and real world data. © 2009 American Institute of Chemical Engineers AIChE J, 2010