Using least squares to generate forecasts in regressions with serial correlation

Abstract. The topic of serial correlation in regression models has attracted a great deal of research in the last 50 years. Most of these studies have assumed that the structure of the error covariance matrix Ω was known or could be consistently estimated from the data. In this article, we describe a new procedure for generating forecasts for regression models with serial correlation based on ordinary least squares and on an approximate representation of the form of the autocorrelation. We prove that the predictors from this specification are asymtotically efficient under some regularity conditions. In addition, we show that there is not much to be gained in trying to identify the correct form of the serial correlation since efficient forecasts can be generated using autoregressive approximations of the autocorrelation. A large simulation study is also used to compare the finite sample predictive efficiencies of this new estimator vis‐à‐vis estimators based on ordinary least squares and generalized least squares.     

Effects of outliers on the identification and estimation of GARCH models

Abstract. This paper analyses how outliers affect the identification of conditional heteroscedasticity and the estimation of generalized autoregressive conditionally heteroscedastic (GARCH) models. First, we derive the asymptotic biases of the sample autocorrelations of squared observations generated by stationary processes and show that the properties of some conditional homoscedasticity tests can be distorted. Second, we obtain the asymptotic and finite sample biases of the ordinary least squares (OLS) estimator of ARCH(p) models. The finite sample results are extended to generalized least squares (GLS), maximum likelihood (ML) and quasi‐maximum likelihood (QML) estimators of ARCH(p) and GARCH(1,1) models. Finally, we show that the estimated asymptotic standard deviations are biased estimates of the sample standard deviations.     

Least tail‐trimmed squares for infinite variance autoregressions

We develop a robust least squares estimator for autoregressions with possibly heavy tailed errors. Robustness to heavy tails is ensured by negligibly trimming the squared error according to extreme values of the error and regressors. Tail‐trimming ensures asymptotic normality and super‐‐convergence with a rate comparable to the highest achieved amongst M‐estimators for stationary data. Moreover, tail‐trimming ensures robustness to heavy tails in both small and large samples. By comparison, existing robust estimators are not as robust in small samples, have a slower rate of convergence when the variance is infinite, or are not asymptotically normal. We present a consistent estimator of the covariance matrix and treat classic inference without knowledge of the rate of convergence. A simulation study demonstrates the sharpness and approximate normality of the estimator, and we apply the estimator to financial returns data. Finally, tail‐trimming can be easily extended beyond least squares estimation for a linear stationary AR model. We discuss extensions to quasi‐maximum likelihood for GARCH, weighted least squares for a possibly non‐stationary random coefficient autoregression, and empirical likelihood for robust confidence region estimation, in each case for models with possibly heavy tailed errors.     

On Residual Variance Estimation in Autoregressive Models

In this paper we consider time series models belonging to the autoregressive (AR) family and deal with the estimation of the residual variance. This is important because estimates of the variance are involved in, for example, confidence sets for the parameters of the model, estimation of the spectrum, expressions for the estimated error of prediction and sample quantities used to make inferences about the order of the model. We consider the asymptotic biases for moment and least squares estimators of the residual variance, and compare them with known results when available and with those for maximum likelihood estimators under normality. Simulation results are presented for finite samples     

Residuals‐based tests for the null of no‐cointegration: an Analytical comparison

Abstract. This article studies the asymptotic distribution of five residuals‐based tests for the null of no‐cointegration under a local alternative when the tests are computed using both ordinary least squares (OLS) and generalized least squares (GLS)‐detrended variables. The local asymptotic power of the tests is shown to be a function of Brownian motion and Ornstein–Uhlenbeck processes, depending on a single nuisance parameter, which is determined by the correlation at frequency zero of the errors of the cointegration regression with the shocks to the right‐hand side variables. The tests are compared in terms of power in large and small samples. It is shown that, while no significant improvement can be achieved by using unit root tests other than the OLS detrended t‐test originally proposed by Engle and Granger (1987), the power of GLS residuals tests can be higher than the power of system tests for some values of the nuisance parameter.     

ESTIMATION OF COEFFICIENTS OF TIME SERIES REGRESSION WITH A NONSTATIONARY ERROR PROCESS

Abstract. We treat a problem of estimating unknown coefficients of a time series regression when the variance of the error changes with time, i.e. when a process which the error term obeys is nonstationary. First, we show the weak consistency of the ordinary least squares estimator for the coefficients of a polynomial regression under some assumptions on the covariance structure of the error process. Next, we propose a nonparametric method for estimating the variance of the error process and a weighted least squares estimator of the regression coefficients, which is constructed by using the estimator of the variance. We investigate statistical properties of our proposed estimator in the following way. We consider the prediction of a future value of a linear trend by using our proposed estimator and evaluate its prediction error. By simulation studies, we compare the prediction error of the predictor constructed by using our proposed estimator with the prediction errors obtained for other estimators including the ordinary least squares estimator when the variance of the error process increases with time and the sample sizes are small. As a result, our proposed estimator seems to be reasonable.     

Postmodel selection estimators of variance function for nonlinear autoregression

We consider a problem of estimating a conditional variance function of an autoregressive process. A finite collection of parametric models for conditional density is studied when both regression and variance are modelled by parametric functions. The proposed estimators are defined as the maximum likelihood estimators in the models chosen by penalized selection criteria. Consistency properties of the resulting estimator of the variance when the conditional density belongs to one of the parametric models are studied as well as its behaviour under mis‐specification. The autoregressive process does not need to be stationary but only existence of a stationary distribution and ergodicity is required. Analogous results for the pseudolikelihood method are also discussed. A simulation study shows promising behaviour of the proposed estimator in the case of heavy‐tailed errors in comparison with local linear smoothers.     

Linear Trend with Fractionally Integrated Errors

We consider the estimation of linear trend for a time series in the presence of additive long-memory noise with memory parameter d ∈[0, 1.5). Although no parametric model is assumed for the noise, our assumptions include as special cases the random walk with drift as well as linear trend with stationary invertible autoregressive moving-average errors. Moreover, our assumptions include a wide variety of trend-stationary and difference-stationary situations. We consider three different trend estimators: the ordinary least squares estimator based on the original series, the sample mean of the first differences and a class of weighted (tapered) means of the first differences. We present expressions for the asymptotic variances of these estimators in the form of one-dimensional integrals. We also establish the asymptotic normality of the tapered means for d ∈[0, 1.5) −{0.5} and of the ordinary least squares estimator for d ∈ (0.5, 1.5). We point out connections with existing theory and present applications of the methodology.     

基于岭估计的化工过程测量置信水平研究(英文)

Ordinary least squares （OLS） algorithm is widely applied in process measurement, because the sensor model used to estimate unknown parameters can be approximated through multivariate linear model. However, with few or noisy data or multi-collinearity, unbiased OLS leads to large variance. Biased estimators, especially ridge es-timator, have been introduced to improve OLS by trading bias for variance. Ridge estimator is feasible as an esti-mator with smaller variance. At the same confidence level, with additive noise as the normal random variable, the less variance one estimator has, the shorter the two-sided symmetric confidence interval is. However, this finding is limited to the unbiased estimator and few studies analyze and compare the confidence levels between ridge estima-tor and OLS. This paper derives the matrix of ridge parameters under necessary and sufficient conditions based on which ridge estimator is superior to OLS in terms of mean squares error matrix, rather than mean squares error. Then the confidence levels between ridge estimator and OLS are compared under the condition of OLS fixed sym-metric confidence interval, rather than the criteria for evaluating the validity of different unbiased estimators. We conclude that the confidence level of ridge estimator can not be directly compared with that of OLS based on the criteria available for unbiased estimators, which is verified by a simulation and a laboratory scale experiment on a single parameter measurement.     

基于极小极大估计器的偏最小二乘方法及应用

针对多输入多输出化工过程中,自变量间、因变量间均存在较强的相关性,提出了偏最小二乘回归(PLSR)与极小极大估计器相结合的PLS-Minimax算法。该算法先对样本数据进行多因变量的PLSR,以消除变量间的复共线性,建立较为稳健的模型;然后基于多变量残差的协方差矩阵,采用极小极大准则,估计收缩系数矩阵,以修正回归系数矩阵,改善模型的预报性能。将PLS-Minimax算法实际应用于聚合反应过程的建模,效果良好。与已有方法相比,其所建模型的预报精度有显著提高。     

APPROXIMATE DISTRIBUTION OF PARAMETER ESTIMATORS FOR FIRST-ORDER AUTOREGRESSIVE MODELS

Abstract. Formulae for the exact bias and mean square error for the least squares for forward-backward least squares estimators are obtained based on the explicit expressions for the moment-generating and characteristic functions of quadratic form in the first-order autoregressive process. Asymptotic expressions for their cumulants and the maximum likelihood estimator are given. Approximations of the distributions of the above estimators are proposed based on the Ornstein-Ulenbeck process. A simple computational procedure for the exact distribution is developed, and some numerical comparisons are given which support the overall good accuracy of the approximation and confirm that the maximum likelihood estimator performs better than the others.     

A study on misspecified nonstationary autoregressive time series with a unit root

The effects of order misspecification in nonstationary autoregressive time series estimations are investigated. The true process is assumed to be stationary if differenced. The ordinary least squares estimator is shown to be weakly convergent and its probability limit is derived. Expressions for the dominating terms of the prediction error and of the prediction mean squared error are derived. Using the expressions and Monte Carlo simulations, we compare prediction errors in the misspecified models based on the observation series and those based on the differenced series.     

ALTERNATIVE ESTIMATORS AND UNIT ROOT TESTS FOR THE AUTOREGRESSIVE PROCESS

Abstract. We compare several estimators for the second-order autoregressive process and compare the associated tests for a unit root. Monte Carlo results are reported for the ordinary least squares estimator, the simple symmetric least squares estimator and the weighted symmetric least squares estimator. The weighted symmetric least squares estimator of the autoregressive parameters generally has smaller mean square error than that of the ordinary least squares estimator, particularly when one root is close to one in absolute value. For the second-order model with known zero intercept, the one-sided ordinary least squares test for a unit root is more powerful than the symmetric tests. For the model with an estimated intercept, the one-sided weighted symmetric least squares test is the most powerful test.     

Asymptotic Properties of Weighted Least Squares Estimation in Weak PARMA Models

The aim of this work is to investigate the asymptotic properties of weighted least squares (WLS) estimation for causal and invertible periodic autoregressive moving average (PARMA) models with uncorrelated but dependent errors. Under mild assumptions, it is shown that the WLS estimators of PARMA models are strongly consistent and asymptotically normal. It extends Thm 3.1 of Basawa and Lund (2001) on least squares estimation of PARMA models with independent errors. It is seen that the asymptotic covariance matrix of the WLS estimators obtained under dependent errors is generally different from that obtained with independent errors. The impact can be dramatic on the standard inference methods based on independent errors when the latter are dependent. Examples and simulation results illustrate the practical relevance of our findings. An application to financial data is also presented.     

Asymptotics of Quantiles and Rank Scores in Nonlinear Time Series

This paper extends the concept of regression and autoregression quantiles and rank scores to a very general nonlinear time series model. The asymptotic linearizations of these nonlinear quantiles are then used to obtain the limiting distributions of a class of L-estimators of the parameters. In particular, the limiting distributions of the least absolute deviation estimator and trimmed estimators are obtained. These estimators turn out to be asymptotically more efficient than the widely used conditional least squares estimator for heavy-tailed error distributions. The results are applicable to linear and nonlinear regression and autoregressive models including self-exciting threshold autoregressive models with known threshold.     

Large sample properties of parameter least squares estimates for time-varying arma models

Abstract.  This paper considers estimation of ARMA models with time-varying coefficients. The ARMA parameters belong to d different regimes. The changes in regime occur at irregular time intervals. Consistency and asymptotic normality of least squares and quasi-generalized least squares estimators are shown.     

Long-Memory Errors in Time Series Regressions with a Unit Root

This paper is concerned with estimation and inference in univariate time series regression with a unit root when the error sequence exhibits long-range temporal dependence. We consider generating mechanisms for the unit root process which include models with or without a drift term and we study the limit behavior of least squares statistics in regression models without drift and trend, with drift but no time trend, and with drift and time trend. We derive the limit distribution and rate of convergence of the ordinary least squares (OLS) estimator of the unit root, the intercept and the time trend in the three regression models and for the two different data-generating processes. The limiting distributions for the OLS estimator differ from those obtained under the hypothesis of weakly dependent errors not only in terms of the limiting process involved but also in terms of functional form. Further, we characterize the asymptotic behavior of both the t statistics for testing the unit root hypothesis and the t statistic for the intercept and time trend coefficients. We find that t ratios either diverge to infinity or collapse to zero. The limiting behavior of Phillips's Z _α and Z _t semiparametric corrections is also analyzed and found to be similar to that of standard Dickey– Fuller tests. Our results indicate that misspecification of the temporal dependence features of the error sequence produces major effects on the asymptotic distribution of estimators and t ratios and suggest that alternative approaches might be more suited to testing for a unit root in time series regression.     

Akaike’s information criterion correction for the least‐squares autoregressive spectral estimator

In this article we propose a new correction for the penalty term of the Akaike’s information criterion (AIC), when it is used in the context of order selection for an autoregressive fit of the spectral density of a stationary time series. The classical AIC penalty term may be viewed as an approximation of an appropriate target quantity. Simulations show that the quality of this approximation strongly depends on the type of autoregressive estimator used, as well as on the discrepancy used. Therefore here we consider the least squares autoregressive estimator and the Whittle discrepancy only. In this context we propose a closed formula correction of the AIC penalty term. We also develop asymptotic theory which justifies this proposal: an asymptotically valid second‐order expansion of a stochastic approximation of the target quantity. This expansion assumes a non‐parametric framework: it does not assume gaussianity of the process and only requires its spectral density to be smooth enough. Simulations show that, as compared to previously introduced corrections, this new correction performs similarly to finite sample information criterion, while they both outperform AIC corrected and AIC.     

Inference on Multivariate Heteroscedastic Time Varying Random Coefficient Models

In this article, we introduce the general setting of a multivariate time series autoregressive model with stochastic time‐varying coefficients and time‐varying conditional variance of the error process. This allows modelling VAR dynamics for non‐stationary time series and estimation of time‐varying parameter processes by the well‐known rolling regression estimation techniques. We establish consistency, convergence rates, and asymptotic normality for kernel estimators of the paths of coefficient processes and provide pointwise valid standard errors. The method is applied to a popular seven‐variable dataset to analyse evidence of time variation in empirical objects of interest for the DSGE (dynamic stochastic general equilibrium) literature.     

TESTING FOR A UNIT ROOT IN AN AR(1) TIME SERIES USING IRREGULARLY OBSERVED DATA

Abstract. For an AR(1) model having a unit root with nonconsecutively observed or missing data we consider the ordinary least squares estimator, the one-step Newton-Raphson estimator and an ordinary least squares type estimator which is a simple approximation of the Newton-Raphson estimator. It is shown that the limiting distributions of these estimators of the unit root are the same as those of the regression estimators as tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Statist. Assoc. 74 (1979), 427–31) for the complete data situation. Simulation results show that our proposed unit root tests perform very well for small samples.