EFFICIENT ESTIMATION OF NONSTATIONARY TIME SERIES REGRESSION

Abstract. A multiple time series regression model with trending regressors has residuals that are believed to be not only serially dependent but nonstationary. Assuming the residuals can be decomposed as a stationary autoregressive process of known order multiplied by an unknown time-varying scale factor, we propose estimators of the regression coefficients and show them to be as efficient as estimators based on known scale factors. Our estimators have features in common with adaptive estimators proposed by Carroll (1982) and Hannan (1963) for different regression problems, involving respectively independent residuals with heteroskedasticity of unknown type, and stationary residuals with unknown serial dependence structure.     

SEMIPARAMETRIC TIME SERIES REGRESSION

Abstract. Let ( X _i, Y _i), i = 0, pL 1,… denote a bivariate stationary time series with X _i being R^d-valued and Y _i being real-valued. We consider the regression model Y _i=θ( X _i) + Z _i, where θ(·) is an unknown function and Z_i is an autoregressive process. Given a realization of length n , we examine the problem of estimating the nonparametric function θ(·) and the parametric component Z _i. Under appropriate regularity conditions, it is shown that both components can be optimally estimated.     

ALGORITHMS FOR ESTIMATION OF POSSIBLY NONSTATIONARY VECTOR TIME SERIES

Abstract. This paper presents efficient algorithms for evaluating the likelihood function and its gradient of possibly nonstationary vector autoregressive moving-average (VARMA) processes.     

ESTIMATION AND BLIND DECONVOLUTION OF AUTOREGRESSIVE SYSTEMS WITH NONSTATIONARY BINARY INPUTS

Abstract. The problem of parameter estimation and blind deconvolution of auto-regressive (AR) systems with independent nonstationary binary inputs is considered. The estimation procedure consists of applying a moving-average filter (equalizer) to the observed data and adjusting the parameters of the filter so as to minimize a criterion that measures the binariness of its output. The output sequence itself serves as an estimate of the unobservable binary input of the AR system. Without assuming stationarity of the inputs, it is shown that the proposed method produces a consistent estimator of the AR system not only in the sense of converging to the true parameter as the sample size increases, but also in the sense of attaining the true parameter of the AR system for a sufficiently large sample size. For noisy data, the estimation criterion is modified on the basis of an asymptotic analysis of the effect of the noise. It is shown that the modified criterion is also consistent (in the usual sense) and its variability depends upon the filtered noise. Some simulation results are presented to demonstrate the performance of the proposed method for parameter estimation as well as for blind deconvolution.     

TIME SERIES RESIDUALS WITH APPLICATION TO PROBABILITY DENSITY ESTIMATION

Abstract. A linear stationary and invertible process y _t models the second-order properties of T observations on a discrete time series, up to finitely many unknown parameters θ. Two estimators of the residuals or innovations ɛ _t of y _t are presented, based on a θ estimator which is root- T consistent with respect to a wide class of ɛ _t distributions, such as a Gaussian estimator. One sets unobserved y _t equal to their mean, the other treats y _t as a circulant and may be best computed via two passes of the fast Fourier transform. The convergence of both estimators to ɛ _t is investigated. We apply the estimated ɛ _t to estimate the probability density function of ɛ _t . Kernel density estimators are shown to converge uniformly in probability to the true density. A new sub-class of linear time series models is motivated.     

A METHODOLOGICAL NOTE ON THE DISAGGREGATION OF TIME SERIES TOTALS

Abstract. This paper sheds new light on a generalized least squares approach for disaggregating a series of time series totals to estimate an underlying unaggregated series. By reinterpreting the generalized least squares problem as a time series prediction problem we can produce considerable computational savings relative to standard least squares approaches. Our reinterpretation gives us insight into the nature of the matrices which need to be inverted to compute the disaggregates.     

REGRESSION OF SPECTRAL ESTIMATORS WITH FRACTIONALLY INTEGRATED TIME SERIES

Abstract. Assuming a normal distribution we supplement the proof of periodogram regression suggested by Geweke and Porter-Hudak ( J. Time Ser. Anal. 4 (1983) 221–38) in order to estimate and test the difference parameter of fractionally integrated autoregressive moving-average models. The procedure proposed by Kashyap and Eom ( J. Time Ser. Anal. 9 (1988) 35–41) arises as a special case and is found to be correct if the true parameter value is negative. Regression of the smoothed periodogram yields estimators for the difference parameter with much faster vanishing variance; no asymptotic distribution can be derived, however. In computer experiments we find that the smoothed periodogram regression may be superior to pure periodogram regression when we have to discriminate between autoregression and fractional integration     

ESTIMATION AND INTERPOLATION OF MISSING VALUES OF A STATIONARY TIME SERIES

Abstract. A complete solution of the important problem of estimating (interpolating) the missing values of a stationary time series is obtained by decomposing it into a prediction plus regression problem. This makes it possible to estimate the missing values by finding the multistep-ahead predictors and using the existing computer packages for time series analysis. Such a solution is vital for the E step of the EM algorithm, and it is shown how this algorithm can be used to develop a simultaneous procedure for estimating the parameters and missing values of a time series.     

PREDICTION ERROR OF MULTIVARIATE TIME SERIES WITH MIS-SPECIFIED MODELS

We consider the situation in which an incorrectly specified autoregressive moving-average model is used to predict future values of a stationary multivariate time series. The use of an incorrect model for prediction results in an increase in mean-square prediction error over that of the optimal predictor, and an expression for this increase is first given for fixed values of the parameters in the incorrect model. For the case in which the incorrect model is an autoregression, we also take into account parameter estimation error by first deriving the asymptotic distribution and limiting moment properties of the least-squares estimator of the parameters in the mis-specified model. An asymptotic approximation to the increase in mean-square prediction error is then obtained. Numerical examples are provided to demonstrate the accuracy of the asymptotic approximation in finite samples. Our results are consistent with those obtained in the univariate case, indicating that fitted autoregressions of high order can yield substantially sub-optimal forecasts.     

ESTIMATION FOR THE FIRST-ORDER DIAGONAL BILINEAR TIME SERIES MODEL

Abstract. The problem of estimation of the parameter b in the simple diagonal bilinear model { X _t }, X_t = e_t + be _{t -1} X_{t -1}, is considered, where { e_t } is Gaussian white noise with zero mean and possibly unknown variance ². The asymptotic normality of the moment estimator of b is established for the two cases when ² is known and ² is unknown. It is noted that the limit distribution of the least-squares cannot easily be derived analytically. A bootstrap comparison of the sampling distributions of the least-squares and moment estimates shows that both are asymptotically normal with the least-squares estimate being the more efficient.     

ESTIMATION OF MULTIVARIATE TIME SERIES

Abstract. The algorithm proposed here is a multivariate generalization of a procedure discussed by Pearlman (1980) for calculating the exact likelihood of a univariate ARMA model. Ansley and Kohn (1983) have shown how the Kalman filter can be used to calculate the exact likelihood function when not all the observations are known. In Shea (1983) it is shown that this algorithm is much quicker than that of Ansley and Kohn (1983) for all ARMA models except an ARMA (2, 1) and a couple of low-order AR processes and therefore when we have no missing observations this algorithm should be used instead. The Fortran subroutine G13DCF in the NAG (1987) Library fits a vector ARMA model using an adaptation of this algorithm. Experience in the use of this routine suggests that having reasonably good initial estimates of the ARMA parameter matrices, and in particular the residual error covariance matrix, can not only substantially reduce the computing time but more important improve the convergence properties of the minimization procedure. We therefore propose a method of calculating initial estimates of the ARMA parameters which involves using a generalization of the concept of inverse cross covariances from the univariate to the multivariate case. Finally theory is put into practice with the fitting of a bivariate model to a couple of real-life time series.     

TESTING THE ORDER OF DIFFERENCING IN TIME SERIES REGRESSION

Abstract. In this paper we develop a test procedure for detecting overdifferencing or a moving-average unit root in time series regression models with Gaussian autoregressive moving-average errors. In addition to an intercept term the regressors consist of stable or asymptotically stationary variables and non-stationary trending variables generated by an integrated process of order 1. The test of the paper is based on the theory of locally best invariant unbiased tests. Its limiting distribution is derived under the null hypothesis and found to be non-standard but free of unknown nuisance parameters. Asymptotic critical values, which depend on the number of integrated regressors, are obtained by simulation. A limited simulation study is carried out to illustrate the finite sample properties of the test.     

KERNEL REGRESSION SMOOTHING OF TIME SERIES

Abstract. A class of non-parametric regression smoothers for times series is defined by the kernel method. The kernel approach allows flexible modelling of a time series without reference to a specific parametric class. The technique is applicable to detection of non-linear dependences in time series and to prediction in smooth regression models with serially correlated observations.
In practice these estimators are to be tuned by a smoothing parameter. A data-driven selector for this smoothing parameter is presented that asymptotically minimizes a squared error measure. We prove asymptotic optimality of this selector. We illustrate the technique with a simulated example and by constructing a smooth prediction curve for the variation of gold prices. In both cases the non-parametric method proves to be useful in uncovering non-linear structure.     

ESTIMATION OF COEFFICIENTS OF AN AUTOREGRESSIVE PROCESS BY USING A HIGHER ORDER MOMENT

Abstract. When we use the estimators, obtained by solving Yule-Walker equations, of the coefficients of an autoregressive process, we cannot discriminate X _t and Y _t where all the solutions of the associated polynomial equation of X _t are less than 1 in the absolute value and, at least, one of the solutions of that of Y _t is greater than 1 in the absolute value. To discriminate between X _t and Y _t Rosenblatt proposed a method. We propose another method by using a higher order moment.     

VALIDITY OF EDGEWORTH EXPANSIONS FOR STATISTICS OF TIME SERIES

Abstract. In this paper, we discuss the validity of the multivariate Edgeworth expansion of distribution functions of statistics which need not be standardized sums of independent and identically distributed vectors. We apply this result to statistics of time series. In particular, we shall give the asymptotic expansion of the distribution of the maximum likelihood estimator of a parameter of a circular autoregresive moving average process.     

NON-PARAMETRIC ESTIMATION WITH STRONGLY DEPENDENT MULTIVARIATE TIME SERIES

Smooth non-parametric kernel density and regression estimators are studied when the data are strongly dependent. In particular, we derive central (and non-central) limit theorems for the kernel density estimator of a multivariate Gaussian process and an infinite-order moving average of an independent identically distributed process, as well as the estimator's consistency for other types of data, such as non-linear functions of a Gaussian process. We find that the kernel density estimator at two different points, under certain conditions, is not only perfectly correlated but may converge to the same random variable. Also, central (and non-central) limit theorems of the non-parametric kernel regression estimator are studied. One important and surprising characteristic found is that its asymptotic variance does not depend on the point at which the regression function is estimated and also that its asymptotic properties are the same whether or not regressors are strongly dependent. Finally, a Monte Carlo experiment is reported to assess the behaviour of the estimators in finite samples.     

SEARCHING FOR ADDITIVE OUTLIERS IN NONSTATIONARY TIME SERIES*

Abstract. Recently, Vogelsang (1999) proposed a method to detect outliers which explicitly imposes the null hypothesis of a unit root. It works in an iterative fashion to select multiple outlier in a given series. We show, via simulations, that, under the null hypothesis of no outliers, it has the right size in finite samples to detect a single outlier but, when applied in an iterative fashion to select multiple outliers, it exhibits severe size distortions towards finding an excessive number of outliers. We show that his iterative method is incorrect and derive the appropriate limiting distribution of the test at each step of the search. Whether corrected or not, we also show that the outliers need to be very large for the method to have any decent power. We propose an alternative method based on first‐differenced data that has considerably more power. We also show that our method to identify outliers leads to unit root tests with more accurate finite sample size and robustness to departures from a unit root. The issues are illustrated using two US/Finland real‐exchange rate series.     

ROBUST REGRESSION AND INTERPOLATION FOR TIME SERIES

Abstract. In this paper we shall consider the interpolation problem under the condition that the spectral density of a stationary process concerned is vaguely known (i.e., Huber's ε -contaminated model). Then we can get a minimax robust interpolator for the class of spectral densities S ={ g:g(x)=(1-ε)f(x)+εh(x)ε Ar D_o, 0<ε<1}, where f(x) is a known spectral density and D ₀ is a certain class of spectral densities. Also we shall consider the time series regression problem under the condition that the residual spectral density is vaguely known. Then we can get a minimax robust regression coefficient estimate for the class of the residual spectral densities S .     

A COMPUTATIONAL METHOD FOR ESTIMATING DENSITIES OF NON-GAUSSIAN NONSTATIONARY UNIVARIATE TIME SERIES

Abstract. Kitagawa's (1987a) numerical integration formulae used to approximate the filtering, prediction and smoothing densities of nonlinear non-Gaussian state-space models are modified. The method involves integration by parts which permits the integration of the conditional system density and, possibly, the observational density prior to and independently of the computation of the filtering, prediction and smoothing densities. In addition to a substantial reduction in computing time and an increase in accuracy, this approach eliminates the necessity of incorporating dynamic adjustments to the filtering, prediction and smoothing processes to accommodate difficult noise densities.
Three numerical examples are presented. One example replicates Kitagawa's non-Gaussian state-space model; the second is a linear Gaussian model; and the third is a nonlinear non-Gaussian model. Comparisons of speed and accuracy between alternative methods and between three computers (personal computer, minicomputer and supercomputer) are made.     

DISTRIBUTION OF RESIDUAL AUTOCORRELATIONS IN NONSTATIONARY AUTOREGRESSIVE PROCESSES

Abstract. The residual autocorrelations in nonstationary autoregressive processes with autoregressive characteristic roots on the unit circle are considered. Limiting distributions of the residual autocovariances and the residual autocorrelations are shown to be the same as the limiting distributions when parameters are estimated with all roots on the unit circle known. The portmanteau statistic is shown to have a x² limiting distribution. The Canadian lynx data set is analysed to illustrate our theory. The portmanteau test seems also useful when the characteristic roots are close to the unit circle.