BOOTSTRAP FOR RANDOM COEFFICIENT AUTOREGRESSIVE MODELS

In this paper, we deal with autoregressive processes with random coefficients. We propose a least‐squares estimator for the fourth‐order moments of both the innovation and disturbance noises and state its consistency. The main theme of the paper is the development of bootstrap procedures for the autoregressive parameter. We show how to obtain approximative residuals for the process even though the standard method for autoregressive processes does not work in this context since one then would obtain convoluted residuals of the innovation and disturbance noises. These ideas lead to a modification of the classical residual bootstrap for autoregressive processes. The consistency of the bootstrap procedure is established. Further, the estimators proposed in the first part are used to form two wild bootstrap modifications. Finally, the performances of the three bootstrap procedures are explored by a simulation study and compared with each other.     

APPROXIMATE DISTRIBUTION OF PARAMETER ESTIMATORS FOR FIRST-ORDER AUTOREGRESSIVE MODELS

Abstract. Formulae for the exact bias and mean square error for the least squares for forward-backward least squares estimators are obtained based on the explicit expressions for the moment-generating and characteristic functions of quadratic form in the first-order autoregressive process. Asymptotic expressions for their cumulants and the maximum likelihood estimator are given. Approximations of the distributions of the above estimators are proposed based on the Ornstein-Ulenbeck process. A simple computational procedure for the exact distribution is developed, and some numerical comparisons are given which support the overall good accuracy of the approximation and confirm that the maximum likelihood estimator performs better than the others.     

THE ESTIMATION AND APPLICATION OF LONG MEMORY TIME SERIES MODELS

Abstract. The definitions of fractional Gaussian noise and integrated (or fractionally differenced) series are generalized, and it is shown that the two concepts are equivalent. A new estimator of the long memory parameter in these models is proposed, based on the simple linear regression of the log periodogram on a deterministic regressor. The estimator is the ordinary least squares estimator of the slope parameter in this regression, formed using only the lowest frequency ordinates of the log periodogram. Its asymptotic distribution is derived, from which it is evident that the conventional interpretation of these least squares statistics is justified in large samples. Using synthetic data the asymptotic theory proves to be reliable in samples of 50 observations or more. For three postwar monthly economic time series, the estimated integrated series model provides more reliable out-of-sample forecasts than do more conventional procedures.     

Sequential estimation for the parameters of a stationary auto regressive model

This paper considers the problem of sequential point estimation and fixed accuracy confidence set procedures of autoregressive parameters in a ρ-th order stationary autoregressive model. The sequential estimator proposed here is based on the least squares estimator and is shown to be risk efficient as the cost of estimation error tends to infinity. Furthermore, the proposed procedure for fixed-width confidence set is shown to be both asymptotically consistent and asymptotically efficient as the width approaches zero.     

ALTERNATIVE ESTIMATORS AND UNIT ROOT TESTS FOR THE AUTOREGRESSIVE PROCESS

Abstract. We compare several estimators for the second-order autoregressive process and compare the associated tests for a unit root. Monte Carlo results are reported for the ordinary least squares estimator, the simple symmetric least squares estimator and the weighted symmetric least squares estimator. The weighted symmetric least squares estimator of the autoregressive parameters generally has smaller mean square error than that of the ordinary least squares estimator, particularly when one root is close to one in absolute value. For the second-order model with known zero intercept, the one-sided ordinary least squares test for a unit root is more powerful than the symmetric tests. For the model with an estimated intercept, the one-sided weighted symmetric least squares test is the most powerful test.     

ESTIMATION OF COEFFICIENTS OF TIME SERIES REGRESSION WITH A NONSTATIONARY ERROR PROCESS

Abstract. We treat a problem of estimating unknown coefficients of a time series regression when the variance of the error changes with time, i.e. when a process which the error term obeys is nonstationary. First, we show the weak consistency of the ordinary least squares estimator for the coefficients of a polynomial regression under some assumptions on the covariance structure of the error process. Next, we propose a nonparametric method for estimating the variance of the error process and a weighted least squares estimator of the regression coefficients, which is constructed by using the estimator of the variance. We investigate statistical properties of our proposed estimator in the following way. We consider the prediction of a future value of a linear trend by using our proposed estimator and evaluate its prediction error. By simulation studies, we compare the prediction error of the predictor constructed by using our proposed estimator with the prediction errors obtained for other estimators including the ordinary least squares estimator when the variance of the error process increases with time and the sample sizes are small. As a result, our proposed estimator seems to be reasonable.     

Stationary bootstrapping for non‐parametric estimator of nonlinear autoregressive model

We consider stationary bootstrap approximation of the non‐parametric kernel estimator in a general kth‐order nonlinear autoregressive model under the conditions ensuring that the nonlinear autoregressive process is a geometrically Harris ergodic stationary Markov process. We show that the stationary bootstrap procedure properly estimates the distribution of the non‐parametric kernel estimator. A simulation study is provided to illustrate the theory and to construct confidence intervals, which compares the proposed method favorably with some other bootstrap methods.     

基于岭估计的化工过程测量置信水平研究(英文)

Ordinary least squares （OLS） algorithm is widely applied in process measurement, because the sensor model used to estimate unknown parameters can be approximated through multivariate linear model. However, with few or noisy data or multi-collinearity, unbiased OLS leads to large variance. Biased estimators, especially ridge es-timator, have been introduced to improve OLS by trading bias for variance. Ridge estimator is feasible as an esti-mator with smaller variance. At the same confidence level, with additive noise as the normal random variable, the less variance one estimator has, the shorter the two-sided symmetric confidence interval is. However, this finding is limited to the unbiased estimator and few studies analyze and compare the confidence levels between ridge estima-tor and OLS. This paper derives the matrix of ridge parameters under necessary and sufficient conditions based on which ridge estimator is superior to OLS in terms of mean squares error matrix, rather than mean squares error. Then the confidence levels between ridge estimator and OLS are compared under the condition of OLS fixed sym-metric confidence interval, rather than the criteria for evaluating the validity of different unbiased estimators. We conclude that the confidence level of ridge estimator can not be directly compared with that of OLS based on the criteria available for unbiased estimators, which is verified by a simulation and a laboratory scale experiment on a single parameter measurement.     

ESTIMATING THE ARCH PARAMETERS BY SOLVING LINEAR EQUATIONS

Abstract. This paper discusses the asymptotics of two-stage least squares estimator of the parameters of ARCH models. The estimator is easy to obtain since it involves solving two sets of linear equations. At the same time, the estimator has the same asymptotic efficiency as that of the widely used quasi-maximum likelihood estimator. Simulation results show that, even for small sample size, the performance of our estimator compared to the quasi-maximum likelihood estimator is better.     

Explosive Random‐Coefficient AR(1) Processes and Related Asymptotics for Least‐Squares Estimation

Abstract. Large sample properties of the least‐squares and weighted least‐squares estimates of the autoregressive parameter of the explosive random‐coefficient AR(1) process are discussed. It is shown that, contrary to the standard AR(1) case, the least‐squares estimator is inconsistent whereas the weighted least‐squares estimator is consistent and asymptotically normal even when the error process is not necessarily Gaussian. Conditional asymptotics on the event that a certain limiting random variable is non‐zero is also discussed.     

Oracle M‐Estimation for Time Series Models

We propose a thresholding M‐estimator for multivariate time series. Our proposed estimator has the oracle property that its large‐sample properties are the same as of the classical M‐estimator obtained under the a priori information that the zero parameters were known. We study the consistency of the standard block bootstrap, the centred block bootstrap and the empirical likelihood block bootstrap distributions of the proposed M‐estimator. We develop automatic selection procedures for the thresholding parameter and for the block length of the bootstrap methods. We present the results of a simulation study of the proposed methods for a sparse vector autoregressive VAR(2) time series model. The analysis of two real‐world data sets illustrate applications of the methods in practice.     

LEAST SQUARES ESTIMATION OF A SHIFT IN LINEAR PROCESSES

Abstract. This paper considers a mean shift with an unknown shift point in a linear process and estimates the unknown shift point (change point) by the method of least squares. Pre-shift and post-shift means are estimated concurrently with the change point. The consistency and the rate of convergence for the estimated change point are established. The asymptotic distribution for the change point estimator is obtained when the magnitude of shift is small. It is shown that serial correlation affects the variance of the change point estimator via the sum of the coefficients of the linear process. When the underlying process is autoregressive moving average, a mean shift causes overestimation of its order. A simple procedure is suggested to mitigate the bias in order estimation.     

Polynomial Trend Regression With Long‐memory Errors

Abstract. For a time series generated by polynomial trend with stationary long‐memory errors, the ordinary least squares estimator (OLSE) of the trend coefficients is asymptotically normal, provided the error process is linear. The asymptotic distribution may no longer be normal, if the error is in the form of a long‐memory linear process passing through certain nonlinear transformations. However, one hardly has sufficient information about the transformation to determine which type of limiting distribution the OLSE converges to and to apply the correct distribution so as to construct valid confidence intervals for the coefficients based on the OLSE. The present paper proposes a modified least squares estimator to bypass this drawback. It is shown that the asymptotic normality can be assured for the modified estimator with mild trade‐off of efficiency even when the error is nonlinear and the original limit for the OLSE is non‐normal. The estimator performs fairly well when applied to various simulated series and two temperature data sets concerning global warming.     

On Truncatd sequential estimation of the drifting parametermean in the first order autoregressive models

This paper considers the problem of sequential point estimation of the drifting parameter mean in the first order autoregression process. The truncated sequential procedure proposed here is based on the least squares estimator and is shown to ensure the preassigned mean square accuracy of the estimates. The uniform in parameter asymptotic normality of the sequential estimator is established.     

ON THE RELATIONSHIP BETWEEN GENERALIZED LEAST SQUARES AND GAUSSIAN ESTIMATION OF VECTOR ARMA MODELS

Abstract. This paper investigates theoretical aspects of the relationship between the generalized least squares and Gaussian estimation schemes for vector autoregressive moving-average models. The asymptotic convergence of the generalized least squares estimator to the Gaussian estimator is established and an alternative numerical method for implementing the generalized least squares scheme is proposed. Finally, some simulation results are presented to illustrate the theory.     

Asymptotic theory for certain regression models with long memory errors

The asymptotic distribution of a weighted linear combination of a linear long memory series is shown to be normal for certain weights. This result can be used to derive the limiting distribution of the least squares estimators for polynomial trends and of the periodogram at fixed Fourier frequencies. A closed form expression for the asymptotic relative bias of the tapered periodogram at fixed Fourier frequencies is also obtained. A weighted least squares estimator, which is asymptotically efficient for polynomial trend regressors, is shown to be asymptotically normal.     

A decision theoretic approach to change point estimation for binomial CUSUM control charts

Detecting when the process has changed is a classical problem in sequential analysis and is an important practical issue in statistical process control. This article is concerned about the binomial cumulative sum (CUSUM) control chart, which is extensively applied to industrial process control, health care, public health surveillance, and other fields. For the binomial CUSUM, a maximum likelihood estimator has been proposed to estimate the change point. In our article, following a decision theoretic approach, we develop a new estimator that aims to improve the existing methods. For interval estimation, we propose a parametric bootstrap procedure to construct the confidence set of the change point. We compare our proposed method with the maximum likelihood estimator and Page's last zero estimator in terms of mean squared error by simulations. We find that the proposed method gives more unbiased and robust results than the existing procedures under various parameter designs. We analyze jewelry manufacturing data for illustration.     

ESTIMATION FOR REGRESSIVE AND AUTOREGRESSIVE MODELS WITH NON-NEGATIVE RESIDUAL ERRORS

Abstract. The parameter estimation problems for regressive and autoregressive models are investigated. A new procedure is proposed which differs from the least squares method. Theorems relating to the rate of almost sure convergence of the new estimators are formulated. Some simulation results are also shown. With these convergent rates and simulation results a clear comparison of the new estimator with the least squares estimator is obtained.     

ESTIMATION OF THE MULTIVARIATE AUTOREGRESSIVE MOVING AVERAGE HAVING PARAMETER RESTRICTIONS AND AN APPLICATION TO ROTATIONAL SAMPLING

Abstract. The vector autoregressive moving average model with nonlinear parametric restrictions is considered. A simple and easy-to-compute Newton-Raphson estimator is proposed that approximates the restricted maximum likelihood estimator which takes full advantage of the information contained in the restrictions. In the case when there are no parametric restrictions, our Newton-Raphson estimator is equivalent to the estimator proposed by Reinsel et al. (Maximum likelihood estimators in the multivariate autoregressive moving-average model from a generalized least squares view point. J. Time Ser. Anal. 13 (1992), 133–45). The Newton-Raphson estimation procedure also extends to the vector ARMAX model. Application of our Newton-Raphson estimation method in rotational sampling problems is discussed. Simulation results are presented for two different restricted models to illustrate the estimation procedure and compare its performance with that of two alternative procedures that ignore the parametric restrictions.     

TESTING FOR A UNIT ROOT IN AN AR(1) TIME SERIES USING IRREGULARLY OBSERVED DATA

Abstract. For an AR(1) model having a unit root with nonconsecutively observed or missing data we consider the ordinary least squares estimator, the one-step Newton-Raphson estimator and an ordinary least squares type estimator which is a simple approximation of the Newton-Raphson estimator. It is shown that the limiting distributions of these estimators of the unit root are the same as those of the regression estimators as tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Statist. Assoc. 74 (1979), 427–31) for the complete data situation. Simulation results show that our proposed unit root tests perform very well for small samples.