ESTIMATION OF COEFFICIENTS OF TIME SERIES REGRESSION WITH A NONSTATIONARY ERROR PROCESS

Abstract. We treat a problem of estimating unknown coefficients of a time series regression when the variance of the error changes with time, i.e. when a process which the error term obeys is nonstationary. First, we show the weak consistency of the ordinary least squares estimator for the coefficients of a polynomial regression under some assumptions on the covariance structure of the error process. Next, we propose a nonparametric method for estimating the variance of the error process and a weighted least squares estimator of the regression coefficients, which is constructed by using the estimator of the variance. We investigate statistical properties of our proposed estimator in the following way. We consider the prediction of a future value of a linear trend by using our proposed estimator and evaluate its prediction error. By simulation studies, we compare the prediction error of the predictor constructed by using our proposed estimator with the prediction errors obtained for other estimators including the ordinary least squares estimator when the variance of the error process increases with time and the sample sizes are small. As a result, our proposed estimator seems to be reasonable.     

Long-Memory Errors in Time Series Regressions with a Unit Root

This paper is concerned with estimation and inference in univariate time series regression with a unit root when the error sequence exhibits long-range temporal dependence. We consider generating mechanisms for the unit root process which include models with or without a drift term and we study the limit behavior of least squares statistics in regression models without drift and trend, with drift but no time trend, and with drift and time trend. We derive the limit distribution and rate of convergence of the ordinary least squares (OLS) estimator of the unit root, the intercept and the time trend in the three regression models and for the two different data-generating processes. The limiting distributions for the OLS estimator differ from those obtained under the hypothesis of weakly dependent errors not only in terms of the limiting process involved but also in terms of functional form. Further, we characterize the asymptotic behavior of both the t statistics for testing the unit root hypothesis and the t statistic for the intercept and time trend coefficients. We find that t ratios either diverge to infinity or collapse to zero. The limiting behavior of Phillips's Z _α and Z _t semiparametric corrections is also analyzed and found to be similar to that of standard Dickey– Fuller tests. Our results indicate that misspecification of the temporal dependence features of the error sequence produces major effects on the asymptotic distribution of estimators and t ratios and suggest that alternative approaches might be more suited to testing for a unit root in time series regression.     

Asymptotic theory for certain regression models with long memory errors

The asymptotic distribution of a weighted linear combination of a linear long memory series is shown to be normal for certain weights. This result can be used to derive the limiting distribution of the least squares estimators for polynomial trends and of the periodogram at fixed Fourier frequencies. A closed form expression for the asymptotic relative bias of the tapered periodogram at fixed Fourier frequencies is also obtained. A weighted least squares estimator, which is asymptotically efficient for polynomial trend regressors, is shown to be asymptotically normal.     

ON THE EFFICIENCY OF THE SAMPLE MEAN IN LONG-MEMORY NOISE

Abstract. When estimating the unknown mean of a stationary time series, the best linear unbiased estimator is often a significantly better estimator than the ordinary least squares estimates _n. The relative efficiency of these two estimators is investigated for time series whose spectrum behaves like a power at the origin (e.g., fractional Gaussian noise and fractional ARIMA).     

TESTING FOR A UNIT ROOT IN AN AR(1) TIME SERIES USING IRREGULARLY OBSERVED DATA

Abstract. For an AR(1) model having a unit root with nonconsecutively observed or missing data we consider the ordinary least squares estimator, the one-step Newton-Raphson estimator and an ordinary least squares type estimator which is a simple approximation of the Newton-Raphson estimator. It is shown that the limiting distributions of these estimators of the unit root are the same as those of the regression estimators as tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Statist. Assoc. 74 (1979), 427–31) for the complete data situation. Simulation results show that our proposed unit root tests perform very well for small samples.     

Efficient use of higher-lag autocorrelations for estimating autoregressive processes

The Yule–Walker estimator is commonly used in time-series analysis, as a simple way to estimate the coefficients of an autoregressive process. Under strong assumptions on the noise process, this estimator possesses the same asymptotic properties as the Gaussian maximum likelihood estimator. However, when the noise is a weak one, other estimators based on higher-order empirical autocorrelations can provide substantial efficiency gains. This is illustrated by means of a first-order autoregressive process with a Markov-switching white noise. We show how to optimally choose a linear combination of a set of estimators based on empirical autocorrelations. The asymptotic variance of the optimal estimator is derived. Empirical experiments based on simulations show that the new estimator performs well on the illustrative model.     

A Test of Linearity for Functional Autoregressive Models

We propose a new test for linearity in time series. We consider an asymptotically stationary functional AR( p ) model on ℜ ^d of the form
X _n = f ( X _{n −1}, ..., X _{n − p} ) + ξ _n ( n ∈ N).
The testing procedure is based on a suitably normalized sum of quadratic deviations between two different estimates of the function f evaluated at q distinct points of ℜ ^dp . The estimators are f^ _n , a recursive version of the non-parametric kernel estimator of f , and  _n , a least squares estimator well suited to the linear case. The main result states that the test statistic has a χ² limit distribution under the null hypothesis. A similar result is derived under the alternative hypothesis for the test statistic corrupted by a non-linear term. Our simulations indicate that our asymptotic results hold for moderate sample sizes when the testing procedure is used carefully     

Polynomial Trend Regression With Long‐memory Errors

Abstract. For a time series generated by polynomial trend with stationary long‐memory errors, the ordinary least squares estimator (OLSE) of the trend coefficients is asymptotically normal, provided the error process is linear. The asymptotic distribution may no longer be normal, if the error is in the form of a long‐memory linear process passing through certain nonlinear transformations. However, one hardly has sufficient information about the transformation to determine which type of limiting distribution the OLSE converges to and to apply the correct distribution so as to construct valid confidence intervals for the coefficients based on the OLSE. The present paper proposes a modified least squares estimator to bypass this drawback. It is shown that the asymptotic normality can be assured for the modified estimator with mild trade‐off of efficiency even when the error is nonlinear and the original limit for the OLSE is non‐normal. The estimator performs fairly well when applied to various simulated series and two temperature data sets concerning global warming.     

THE ESTIMATION AND APPLICATION OF LONG MEMORY TIME SERIES MODELS

Abstract. The definitions of fractional Gaussian noise and integrated (or fractionally differenced) series are generalized, and it is shown that the two concepts are equivalent. A new estimator of the long memory parameter in these models is proposed, based on the simple linear regression of the log periodogram on a deterministic regressor. The estimator is the ordinary least squares estimator of the slope parameter in this regression, formed using only the lowest frequency ordinates of the log periodogram. Its asymptotic distribution is derived, from which it is evident that the conventional interpretation of these least squares statistics is justified in large samples. Using synthetic data the asymptotic theory proves to be reliable in samples of 50 observations or more. For three postwar monthly economic time series, the estimated integrated series model provides more reliable out-of-sample forecasts than do more conventional procedures.     

Broadband Semiparametric Estimation of the Memory Parameter of a Long-Memory Time Series Using Fractional Exponential Models

We consider a fractional exponential, or FEXP estimator of the memory parameter of a stationary Gaussian long-memory time series. The estimator is constructed by fitting a FEXP model of slowly increasing dimension to the log periodogram at all Fourier frequencies by ordinary least squares, and retaining the corresponding estimated memory parameter. We do not assume that the data were necessarily generated by a FEXP model, or by any other finite-parameter model. We do, however, impose a global differentiability assumption on the spectral density except at the origin. Because of this, and its use of all Fourier frequencies, we refer to the FEXP estimator as a broadband semiparametric estimator. We demonstrate the consistency of the FEXP estimator, and obtain expressions for its asymptotic bias and variance. If the true spectral density is sufficiently smooth, the FEXP estimator can strongly outperform existing semiparametric estimators, such as the Geweke–Porter-Hudak (GPH) and Gaussian semiparametric estimators (GSE), attaining an asymptotic mean squared error proportional to (log n )/ n , where n is the sample size. In a simulation study, we demonstrate the merits of using a finite-sample correction to the asymptotic variance, and we also explore the possibility of automatically selecting the dimension of the exponential model using Mallows' C_L criterion.     

A refined efficiency rate for ordinary least squares and generalized least squares estimators for a linear trend with autoregressive errors

When a straight line is fitted to time series data, generalized least squares (GLS) estimators of the trend slope and intercept are attractive as they are unbiased and of minimum variance. However, computing GLS estimators is laborious as their form depends on the autocovariances of the regression errors. On the other hand, ordinary least squares (OLS) estimators are easy to compute and do not involve the error autocovariance structure. It has been known for 50 years that OLS and GLS estimators have the same asymptotic variance when the errors are second‐order stationary. Hence, little precision is gained by using GLS estimators in stationary error settings. This article revisits this classical issue, deriving explicit expressions for the GLS estimators and their variances when the regression errors are drawn from an autoregressive process. These expressions are used to show that OLS methods are even more efficient than previously thought. Specifically, we show that the convergence rate of variance differences is one polynomial degree higher than that of least squares estimator variances. We also refine Grenander's (1954) variance ratio. An example is presented where our new rates cannot be improved upon. Simulations show that the results change little when the autoregressive parameters are estimated.     

ALTERNATIVE ESTIMATORS AND UNIT ROOT TESTS FOR THE AUTOREGRESSIVE PROCESS

Abstract. We compare several estimators for the second-order autoregressive process and compare the associated tests for a unit root. Monte Carlo results are reported for the ordinary least squares estimator, the simple symmetric least squares estimator and the weighted symmetric least squares estimator. The weighted symmetric least squares estimator of the autoregressive parameters generally has smaller mean square error than that of the ordinary least squares estimator, particularly when one root is close to one in absolute value. For the second-order model with known zero intercept, the one-sided ordinary least squares test for a unit root is more powerful than the symmetric tests. For the model with an estimated intercept, the one-sided weighted symmetric least squares test is the most powerful test.     

ASYMPTOTIC PROPERTIES OF SOME PRELIMINARY ESTIMATORS FOR AUTOREGRESSIVE MOVING AVERAGE TIME SERIES MODELS

Abstract. Some simple preliminary estimators for the coefficients of mixed autoregressive moving average time series models are considered. As the first step the estimators require the fitting of a long autoregression to the data. The first two methods of the paper are non-iterative and generally inefficient. The estimators are Yule-Walker type modifications of the least squares estimators of the coefficients in auxiliary linear regression models derived, respectively, for the coefficients of the long autoregression and for the coefficients of the corresponding long moving average approximation of the model. Both of these estimators are shown to be strongly consistent and their asymptotic distributions are derived. The asymptotic distributions are used in studying the loss in efficiency and in constructing the third estimator of the paper which is an asymptotically efficient two-step estimator. A numerical illustration of the third estimator with real data is given.     

Fractional cointegration in the presence of linear trends

Abstract. We consider bivariate regressions of nonstationary fractionally integrated variables dominated by linear time trends. The asymptotic behaviour of the ordinary least square (OLS) estimators in this case allows limiting normality to arise at a faster rate of convergence than if the individual series were detrended, increasing in this way the power of the tests for fractional cointegration. We also show that the limiting distribution of the t‐ratio of the slope coefficient depends upon the presence or not of a deterministic trend in the conditional regressor. We introduce the concept of local fractional trend to explain the apparently diverging asymptotic theories that apply when a trend is either present or absent in our set‐up.     

Asymptotics of Quantiles and Rank Scores in Nonlinear Time Series

This paper extends the concept of regression and autoregression quantiles and rank scores to a very general nonlinear time series model. The asymptotic linearizations of these nonlinear quantiles are then used to obtain the limiting distributions of a class of L-estimators of the parameters. In particular, the limiting distributions of the least absolute deviation estimator and trimmed estimators are obtained. These estimators turn out to be asymptotically more efficient than the widely used conditional least squares estimator for heavy-tailed error distributions. The results are applicable to linear and nonlinear regression and autoregressive models including self-exciting threshold autoregressive models with known threshold.     

Simple Regressions with Linear Time Trends

Simple regressions of two trend stationary time series are considered. As the linear trend dominates the stochastic components the rates of convergence and the limiting distributions of ordinary least squares statistics are exactly the same as in the case of cointegrated regressions with drifts. In particular, asymptotic standard normal t statistics are readily available. Hence, asymptotic inference requires no distinction between simple regressions of trend stationary series and of cointegrated variables with drifts.     

Testing for a Unit Root in Autoregressive Moving-average Models with Missing Data

Testing for a single autoregressive unit root in an autoregressive moving-average (ARMA) model is considered in the case when data contain missing values. The proposed test statistics are based on an ordinary least squares type estimator of the unit root parameter which is a simple approximation of the one-step Newton–Raphson estimator. The limiting distributions of the test statistics are the same as those of the regression statistics in AR(1) models tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Stat. Assoc . 74 (1979), 427–31) for the complete data situation. The tests accommodate models with a fitted intercept and a fitted time trend.     

SEMI‐PARAMETRIC ESTIMATION OF LINEAR COINTEGRATING MODELS WITH NONLINEAR CONTEMPORANEOUS ENDOGENEITY

This article considers linear cointegrating models with unknown nonlinear short‐run contemporaneous endogeneity. Two estimators are proposed to estimate the linear cointegrating parameter after the nonlinear endogenous component is estimated by local linear regression approach. Both the proposed estimators are shown to have the same mixed normal limiting distribution with zero mean and smaller asymptotic variance than the fully modified ordinary least squares and instrumental variables estimators. Monte Carlo simulations are used to evaluate the finite sample performance of our proposed estimators, and an empirical application is also included.     

Least absolute deviation estimation for general autoregressive moving average time‐series models

We study least absolute deviation (LAD) estimation for general autoregressive moving average time‐series models that may be noncausal, noninvertible or both. For ARMA models with Gaussian noise, causality and invertibility are assumed for the parameterization to be identifiable. The assumptions, however, are not required for models with non‐Gaussian noise, and hence are removed in our study. We derive a functional limit theorem for random processes based on an LAD objective function, and establish the consistency and asymptotic normality of the LAD estimator. The performance of the estimator is evaluated via simulation and compared with the asymptotic theory. Application to real data is also provided.     

基于岭估计的化工过程测量置信水平研究(英文)

Ordinary least squares （OLS） algorithm is widely applied in process measurement, because the sensor model used to estimate unknown parameters can be approximated through multivariate linear model. However, with few or noisy data or multi-collinearity, unbiased OLS leads to large variance. Biased estimators, especially ridge es-timator, have been introduced to improve OLS by trading bias for variance. Ridge estimator is feasible as an esti-mator with smaller variance. At the same confidence level, with additive noise as the normal random variable, the less variance one estimator has, the shorter the two-sided symmetric confidence interval is. However, this finding is limited to the unbiased estimator and few studies analyze and compare the confidence levels between ridge estima-tor and OLS. This paper derives the matrix of ridge parameters under necessary and sufficient conditions based on which ridge estimator is superior to OLS in terms of mean squares error matrix, rather than mean squares error. Then the confidence levels between ridge estimator and OLS are compared under the condition of OLS fixed sym-metric confidence interval, rather than the criteria for evaluating the validity of different unbiased estimators. We conclude that the confidence level of ridge estimator can not be directly compared with that of OLS based on the criteria available for unbiased estimators, which is verified by a simulation and a laboratory scale experiment on a single parameter measurement.