AUTOMATIC SEMIPARAMETRIC ESTIMATION OF THE MEMORY PARAMETER OF A LONG-MEMORY TIME SERIES

Abstract. In Geweke and Porter-Hudak's estimator ω d ^(GPH) of the memory parameter of a long-memory process, a critical choice the user must make is the number of frequencies, M , to be used in the regression of the log periodogram on log frequency. This choice is critical in practice because by simply varying M for a given data set it is often possible to obtain a very wide range of values of the estimator. Although Geweke and Porter-Hudak have found that choosing M to be the square root of the sample size gave good results in simulation, they gave no theoretical justification for this choice. Here, we propose automatic criteria for selecting M , and another tuning constant used in related estimates of the memory parameter, based on frequency domain cross-validation. We provide some theoretical and heuristic justification for the proposed criteria. In a simulation study, we compare some of the resulting automatic methods of estimating the memory parameter with existing non-automatic ones.     

ESTIMATION FOR THE FIRST-ORDER DIAGONAL BILINEAR TIME SERIES MODEL

Abstract. The problem of estimation of the parameter b in the simple diagonal bilinear model { X _t }, X_t = e_t + be _{t -1} X_{t -1}, is considered, where { e_t } is Gaussian white noise with zero mean and possibly unknown variance ². The asymptotic normality of the moment estimator of b is established for the two cases when ² is known and ² is unknown. It is noted that the limit distribution of the least-squares cannot easily be derived analytically. A bootstrap comparison of the sampling distributions of the least-squares and moment estimates shows that both are asymptotically normal with the least-squares estimate being the more efficient.     

SPECTRAL ESTIMATION AND DECONVOLUTION FOR A LINEAR TIME SERIES MODEL

Abstract. A vector linear time series model is observed as the sum of a convolution of an unknown signal and an additive noise process. The main objective is the estimation or deconvolution of the signal when the spectra of the signal and noise processes are unknown. We prove the strong consistency of a class of nonparametric spectral estimators derived by maximizing a particular Gaussian likelihood function. We also study the mean square convergence of the finite-sample deconvolution estimators as a function of the sample length T , the filter length M and the spectral bandwidth B_T = L_T/T .     

ESTIMATION OF MULTIVARIATE TIME SERIES

Abstract. The algorithm proposed here is a multivariate generalization of a procedure discussed by Pearlman (1980) for calculating the exact likelihood of a univariate ARMA model. Ansley and Kohn (1983) have shown how the Kalman filter can be used to calculate the exact likelihood function when not all the observations are known. In Shea (1983) it is shown that this algorithm is much quicker than that of Ansley and Kohn (1983) for all ARMA models except an ARMA (2, 1) and a couple of low-order AR processes and therefore when we have no missing observations this algorithm should be used instead. The Fortran subroutine G13DCF in the NAG (1987) Library fits a vector ARMA model using an adaptation of this algorithm. Experience in the use of this routine suggests that having reasonably good initial estimates of the ARMA parameter matrices, and in particular the residual error covariance matrix, can not only substantially reduce the computing time but more important improve the convergence properties of the minimization procedure. We therefore propose a method of calculating initial estimates of the ARMA parameters which involves using a generalization of the concept of inverse cross covariances from the univariate to the multivariate case. Finally theory is put into practice with the fitting of a bivariate model to a couple of real-life time series.     

AN INTRODUCTION TO LONG-MEMORY TIME SERIES MODELS AND FRACTIONAL DIFFERENCING

Abstract. The idea of fractional differencing is introduced in terms of the infinite filter that corresponds to the expansion of (1- B ) ^d . When the filter is applied to white noise, a class of time series is generated with distinctive properties, particularly in the very low frequencies and provides potentially useful long-memory forecasting properties. Such models are shown to possibly arise from aggregation of independent components. Generation and estimation of these models are considered and applications on generated and real data presented.     

FREQUENCY-DOMAIN ESTIMATION OF BILINEAR TIME SERIES MODELS

Abstract. Two frequency-domain methods of estimation of the parameters of linear time series models–one based on maximum likelihood, called the 'Whittle criterion', and the other based on least squares, called the 'Taniguchi criterion'–are discussed in this paper. A heuristic justification for their use in models such as bilinear models is given. The estimation theory and associated asymptotic theory of these methods are numerically illustrated for the bilinear model BL( p ,0, p , 1). For that purpose, an approach based on the calculus of Kronecker product matrices is used to obtain the derivatives of the spectral density function of the state-space form of the model.     

EFFICIENT ESTIMATION OF NONSTATIONARY TIME SERIES REGRESSION

Abstract. A multiple time series regression model with trending regressors has residuals that are believed to be not only serially dependent but nonstationary. Assuming the residuals can be decomposed as a stationary autoregressive process of known order multiplied by an unknown time-varying scale factor, we propose estimators of the regression coefficients and show them to be as efficient as estimators based on known scale factors. Our estimators have features in common with adaptive estimators proposed by Carroll (1982) and Hannan (1963) for different regression problems, involving respectively independent residuals with heteroskedasticity of unknown type, and stationary residuals with unknown serial dependence structure.     

ASYMPTOTICS FOR THE LOW-FREQUENCY ORDINATES OF THE PERIODOGRAM OF A LONG-MEMORY TIME SERIES

Abstract. We consider the asymptotic distribution of the normalized periodogram ordinates I(ω_j)/f(ω_j) ( j = 1,2,…) of a general long-memory time series. Here, I (ω;) is the periodogram based on a sample size n , f (ω) is the spectral density and ω_j= 2π j/n. We assume that n →∝ with j held fixed, and so our focus is on low frequencies; these are the most important frequencies for the periodogram-based estimation of the memory parameter d. Contrary to popular belief, the normalized periodogram ordinates obtained from a Gaussian process are asymptotically neither independent identically distributed nor exponentially distributed. In fact, lim n E{I(ω_j)/f(ω_j)} depends on both j and d and is typically greater than unity, implying a positive asymptotic relative bias in I(ω_j) as an estimator of f(ω_j). Tapering is found to reduce this bias dramatically, except at frequency ω₁. The asymptotic distribution of I(ω_j)/f(ω_j) for a Gaussian process is, in general, that of an unequally weighted linear combination of two independent X²₁ random variables. The asymptotic mean of the log normalized periodogram depends on j and d and is not in general equal to the negative of Euler's constant, as is commonly assumed. Consequently, the regression estimator of d proposed by Geweke and Porter-Hudak will be asymptotically biased if the number of frequencies used in the regression is held fixed as n →∝.     

ESTIMATION OF PERIODICALLY VARYING MEANS AND STANDARD DEVIATIONS IN TIME SERIES DATA

Abstract. Models which account for seasonal changes in mean and standard deviation in time series which are serially correlated are discussed. Full Gaussian maximum likelihood estimation of the parameters specifying the mean function, of the parameters specifying the standard deviation function, and of the parameters specifying the stationary serial correlation structure is discussed and the asymptotic distribution derived without the assumption of Gaussian data. A commonly used method which estimates the three components separately, and uses Fourier series parameterisation, is also reviewed and the asymptotic distribution for these estimates is also derived. Both of these have asymptotic covariances which depend upon third and fourth cumulants of the underlying distribution. When these vanish (e.g., Gaussian case) then a simple revision of the inefficient estimates yields estimates which are asymptotically equivalent to the Gaussian estimates. An application to a series of monthly Atlantic Ocean Sea surface temperatures is given to illustrate this last method.     

NON-PARAMETRIC ESTIMATION WITH STRONGLY DEPENDENT MULTIVARIATE TIME SERIES

Smooth non-parametric kernel density and regression estimators are studied when the data are strongly dependent. In particular, we derive central (and non-central) limit theorems for the kernel density estimator of a multivariate Gaussian process and an infinite-order moving average of an independent identically distributed process, as well as the estimator's consistency for other types of data, such as non-linear functions of a Gaussian process. We find that the kernel density estimator at two different points, under certain conditions, is not only perfectly correlated but may converge to the same random variable. Also, central (and non-central) limit theorems of the non-parametric kernel regression estimator are studied. One important and surprising characteristic found is that its asymptotic variance does not depend on the point at which the regression function is estimated and also that its asymptotic properties are the same whether or not regressors are strongly dependent. Finally, a Monte Carlo experiment is reported to assess the behaviour of the estimators in finite samples.     

IMPROVED NON-FOURIER FREQUENCY ESTIMATION OF SAMPLED STATIONARY TIME SERIES

A general multi-component cyclic descent least-squares model used in conjunction with a power-of-two fast Fourier transform to perform the harmonic analysis presents numerous advantages over their individual use. Through the appropriate choice of a window taper and a linear filter applied to the time series data, severe leakage is effectively reduced by decreasing the interference between peaks throughout the band. The taper and filter characteristics are dependent on the given application. The resulting assembled frequency components are updated iteratively by means of the least-squares model

A brief portion of this study is devoted to the development of the Fourier transform as a general spectral estimator. Particular attention is given to its limitations. Methods for improving frequency identification in the transform are covered. A brief development of the cyclic descent least-squares model is given as a means of reducing errors inherent in a Fourier analysis. A numerical example is presented depicting the method.     

TIME SERIES RESIDUALS WITH APPLICATION TO PROBABILITY DENSITY ESTIMATION

Abstract. A linear stationary and invertible process y _t models the second-order properties of T observations on a discrete time series, up to finitely many unknown parameters θ. Two estimators of the residuals or innovations ɛ _t of y _t are presented, based on a θ estimator which is root- T consistent with respect to a wide class of ɛ _t distributions, such as a Gaussian estimator. One sets unobserved y _t equal to their mean, the other treats y _t as a circulant and may be best computed via two passes of the fast Fourier transform. The convergence of both estimators to ɛ _t is investigated. We apply the estimated ɛ _t to estimate the probability density function of ɛ _t . Kernel density estimators are shown to converge uniformly in probability to the true density. A new sub-class of linear time series models is motivated.     

ALGORITHMS FOR ESTIMATION OF POSSIBLY NONSTATIONARY VECTOR TIME SERIES

Abstract. This paper presents efficient algorithms for evaluating the likelihood function and its gradient of possibly nonstationary vector autoregressive moving-average (VARMA) processes.     

ESTIMATION FOR NON-LINEAR TIME SERIES MODELS USING ESTIMATING EQUATIONS

Godambe's (1985) theorem on optimal estimating equations for stochastic processes is applied to non-linear time series estimation problems. Examples are considered from the usual classes of non-linear time series models. A recursive estimation procedure based on optimal estimating equations is provided. It is also shown that pre-filtered estimates can be used to obtain the optimal estimate from a non-linear state-space model.     

LAG WINDOW ESTIMATION OF THE DEGREE OF DIFFERENCING IN FRACTIONALLY INTEGRATED TIME SERIES MODELS

Abstract. In this paper we consider the estimation of the degree of differencing d in the fractionally integrated autoregressive moving-average time series model ARFIMA ( p, d, q ). Using lag window spectral density estimators we develop a regression type estimator of d which is easy to calculate and does not require prior knowledge of p and q. Some large sample properties of the estimator are studied and the performance of the estimator for small samples is investigated using the simulation method for a range of commonly used lag windows. Some practical recommendations on the choice of lag windows and the choice of the window parameters are provided.     

EXPECTATION-MAXIMIZATION ALGORITHMS AND THE ESTIMATION OF TIME SERIES MODELS IN THE PRESENCE OF OUTLIERS

Abstract. The expectation-maximization algorithm is reviewed briefly. The algorithm is applied to time series situations where outliers may be present. An approximation of the algorithm is considered to reduce the computational complexity. Examples are given to illustrate the application of this algorithm.     

PROFILE SUMMARIES FOR ARIMA TIME SERIES MODEL PARAMETERS

Abstract. Autoregressive intergrated moving average (ARIMA) times series models are nonlinear in the parameters and so summarizing the inferential results for such models can be difficult. A common approach is to present parameter joint and marginal inference regions based on the linear approximation, and although such approximate regions are easy to calculate, it is not known generally whether they approximate the true regions adequately. In this paper we present exact approaches for summarizing the inferential results for time series model parameters, called profile t and profile trace plots, which are based on the work of Bates and Watts. Calculations for the profile plots are simple and can be used to determine exact regions, and so can be used to assess the accuracy of linear approximation regions. In addition to developing the profile plots for time series models, the main finding of the paper is that, for ARIMA model parameters, linear approximation regions are very satisfactory except when a parameter estimate is within about two standard errors of the stationarity or invertibility region boundary.     

A TIME SERIES MODEL WITH SUDDENLY CHANGING PARAMETERS

Abstract. A process X_t = θ_t + e , is investigated where e_t is a strict white noise and θ_t is a Markov chain with two real states. A realization of X_t fluctuates around two levels which correspond to the two states of θ_t . Formulae for the extrapolation of the process X_t and for the mean square error of the extrapolation are derived. The moments of X_t and its covariance function are calculated. The results are used to derive moment estimators for the parameters of the model.     

A CONDITIONAL LEAST SQUARES APPROACH TO BILINEAR TIME SERIES ESTIMATION

Abstract. In this paper a conditional least squares (CLS) procedure for estimating bilinear time series models is introduced. This method is applied to a special superdiagonal bilinear model which includes the classical linear autoregressive moving-average model as a particular case and it is proven that the limiting distribution of the CLS estimates is Gaussian and that the law of the iterated logarithm holds.     

THE ESTIMATION AND APPLICATION OF LONG MEMORY TIME SERIES MODELS

Abstract. The definitions of fractional Gaussian noise and integrated (or fractionally differenced) series are generalized, and it is shown that the two concepts are equivalent. A new estimator of the long memory parameter in these models is proposed, based on the simple linear regression of the log periodogram on a deterministic regressor. The estimator is the ordinary least squares estimator of the slope parameter in this regression, formed using only the lowest frequency ordinates of the log periodogram. Its asymptotic distribution is derived, from which it is evident that the conventional interpretation of these least squares statistics is justified in large samples. Using synthetic data the asymptotic theory proves to be reliable in samples of 50 observations or more. For three postwar monthly economic time series, the estimated integrated series model provides more reliable out-of-sample forecasts than do more conventional procedures.