Empirical likelihood in long‐memory time series models

This article studies the empirical likelihood method for long‐memory time series models. By virtue of the Whittle likelihood, one obtains a score function that can be viewed as an estimating equation of the parameters of a fractional integrated autoregressive moving average (ARFIMA) model. This score function is used to obtain an empirical likelihood ratio which is shown to be asymptotically chi‐square distributed. Confidence regions for the parameters are constructed based on the asymptotic distribution of the empirical likelihood ratio. Bartlett correction and finite sample properties of the empirical likelihood confidence regions are examined.     

Likelihood‐based Analysis of a Class of Generalized Long‐Memory Time Series Models

Abstract. This article introduces a family of ‘generalized long‐memory time series models’, in which observations have a specified conditional distribution, given a latent Gaussian fractionally integrated autoregressive moving‐average (ARFIMA) process. The observations may have discrete or continuous distributions (or a mixture of both). The family includes existing models such as ARFIMA models themselves, long‐memory stochastic volatility models, long‐memory censored Gaussian models and others. Although the family of models is flexible, the latent long‐memory process poses problems for analysis. Therefore, we introduce a Markov chain Monte Carlo sampling algorithm and develop a set of recursions which makes it feasible. This makes it possible, among other things, to carry out exact likelihood‐based analysis of a wide range of non‐Gaussian long‐memory models without resorting to the use of likelihood approximations. The procedure also yields predictive distributions that take into account model parameter uncertainty. The approach is demonstrated in two case studies.     

Bayesian analysis of autoregressive fractionally integrated moving-average processes

For the autoregressive fractionally integrated moving-average (ARFIMA) processes which characterize both long-memory and short-memory behavior in time series, we formulate Bayesian inference using Markov chain Monte Carlo methods. We derive a form for the joint posterior distribution of the parameters that is computationally feasible for repetitive evaluation within a modified Gibbs sampling algorithm that we employ. We illustrate our approach through two examples.     

Computationally efficient methods for two multivariate fractionally integrated models

Abstract.  We discuss two distinct multivariate time-series models that extend the univariate ARFIMA (autoregressive fractionally integrated moving average) model. We discuss the different implications of the two models and describe an extension to fractional cointegration. We describe algorithms for computing the covariances of each model, for computing the quadratic form and approximating the determinant for maximum likelihood estimation and for simulating from each model. We compare the speed and accuracy of each algorithm with existing methods individually. Then, we measure the performance of the maximum likelihood estimator and of existing methods in a Monte Carlo. These algorithms are much more computationally efficient than the existing algorithms and are equally accurate, making it feasible to model multivariate long memory time series and to simulate from these models. We use maximum likelihood to fit models to data on goods and services inflation in the United States.     

Comparing the bias and misspecification in ARFIMA models

We investigate the bias in both the short-term and long-term parameters for a range of autoregressive fractional integrated moving-average (ARFIMA) models using both semi-parametric and maximum likelihood (ML) estimation methods. The results suggest that, provided the correct model is estimated, the ML method outperforms the semi-parametric methods in terms of the bias and smaller mean square errors in both the long-term and short-term parameter estimates. These biases often cause model selection criteria to select an incorrect ARFIMA specification. Taking account of the potential misspecification the biases associated with the ML procedure tend to increase, although it continues to have a smaller worst-case bias than either of the semi-parametric procedures.     

BAYESIAN THRESHOLD AUTOREGRESSIVE MODELS FOR NONLINEAR TIME SERIES

Abstract. This paper provides a Bayesian approach to statistical inference in the threshold autoregressive model for time series. The exact posterior distribution of the delay and threshold parameters is derived, as is the multi-step-ahead predictive density. The proposed methods are applied to the Wolfe's sunspot and Canadian lynx data sets.     

A GENERALIZED FRACTIONALLY INTEGRATED AUTOREGRESSIVE MOVING-AVERAGE PROCESS

Abstract. This paper considers the long memory Gegenbauer autoregressive movingaverage (GARMA) process that generalizes the fractionally integrated ARMA (ARFIMA) process to allow for hyperbolic and sinusoidal decay in autocorrelations. We propose the conditional sum of squares method for estimation (which is asymptotically equivalent to the maximum likelihood estimation) and develop the asymptotic theory. Many results are in sharp contrast to those of the ARFIMA model. Simulations are conducted to assess the performance of the proposed estimators in small sample applications. Two applications to the sunspot data and the US inflation rates based on the wholesale price index are provided.     

Parameter Estimation Under Failure-Censored Constant-Stress Life Testing Model: A Bayesian Approach

Abstract

This article compares likelihood and Bayesian estimations for partially accelerated constant-stress life test model under type II censoring assuming Pareto distribution of the second kind. Both maximum likelihood and Bayesian estimators of the model parameters are derived. The posterior means and posterior variances are obtained under the squared error loss function using Lindley's approximation procedure. The advantages of this proposed procedure are shown. Monte Carlo simulations are conducted under different samples sizes and different parameter values to assess and compare the proposed methods of estimation. A noninformative prior on the model parameters is used to make the comparison more meaningful. It has been observed that Lindley's method usually provides posterior variances and mean squared errors smaller than those of the maximum likelihood estimators. That is, Lindley's method produces improved estimates, which is an advantage of this method.     

Likelihood functions for state space models with diffuse initial conditions

State space models with non‐stationary processes and/or fixed regression effects require a state vector with diffuse initial conditions. Different likelihood functions can be adopted for the estimation of parameters in time‐series models with diffuse initial conditions. In this article, we consider profile, diffuse and marginal likelihood functions. The marginal likelihood function is defined as the likelihood function of a transformation of the data vector. The transformation is not unique. The diffuse likelihood is a marginal likelihood for a data transformation that may depend on parameters. Therefore, the diffuse likelihood cannot be used generally for parameter estimation. The marginal likelihood function is based on an orthonormal data transformation that does not depend on parameters. Here we develop a marginal likelihood function for state space models that can be evaluated by the Kalman filter. The so‐called diffuse Kalman filter is designed for computing the diffuse likelihood function. We show that a minor modification of the diffuse Kalman filter is needed for the evaluation of our marginal likelihood function. Diffuse and marginal likelihood functions have better small sample properties compared with the profile likelihood function for the estimation of parameters in linear time series models. The results in our article confirm the earlier findings and show that the diffuse likelihood function is not appropriate for a range of state space model specifications.     

Estimation of stationary autoregressive models with the Bayesian LASSO

This article explores the problem of estimating stationary autoregressive models from observed data using the Bayesian least absolute shrinkage and selection operator (LASSO). By characterizing the model in terms of partial autocorrelations, rather than coefficients, it becomes straightforward to guarantee that the estimated models are stationary. The form of the negative log‐likelihood is exploited to derive simple expressions for the conditional likelihood functions, leading to efficient algorithms for computing the posterior mode by coordinate‐wise descent and exploring the posterior distribution by Gibbs sampling. Both empirical Bayes and Bayesian methods are proposed for the estimation of the LASSO hyper‐parameter from the data. Simulations demonstrate that the Bayesian LASSO performs well in terms of prediction when compared with a standard autoregressive order selection method.     

ESTIMATION OF MULTIVARIATE TIME SERIES

Abstract. The algorithm proposed here is a multivariate generalization of a procedure discussed by Pearlman (1980) for calculating the exact likelihood of a univariate ARMA model. Ansley and Kohn (1983) have shown how the Kalman filter can be used to calculate the exact likelihood function when not all the observations are known. In Shea (1983) it is shown that this algorithm is much quicker than that of Ansley and Kohn (1983) for all ARMA models except an ARMA (2, 1) and a couple of low-order AR processes and therefore when we have no missing observations this algorithm should be used instead. The Fortran subroutine G13DCF in the NAG (1987) Library fits a vector ARMA model using an adaptation of this algorithm. Experience in the use of this routine suggests that having reasonably good initial estimates of the ARMA parameter matrices, and in particular the residual error covariance matrix, can not only substantially reduce the computing time but more important improve the convergence properties of the minimization procedure. We therefore propose a method of calculating initial estimates of the ARMA parameters which involves using a generalization of the concept of inverse cross covariances from the univariate to the multivariate case. Finally theory is put into practice with the fitting of a bivariate model to a couple of real-life time series.     

Portmanteau tests for ARMA models with infinite variance

Abstract. Autoregressive and moving‐average (ARMA) models with stable Paretian errors are some of the most studied models for time series with infinite variance. Estimation methods for these models have been studied by many researchers but the problem of diagnostic checking of fitted models has not been addressed. In this article, we develop portmanteau tests for checking the randomness of a time series with infinite variance and for ARMA diagnostic checking when the innovations have infinite variance. It is assumed that least squares or an asymptotically equivalent estimation method, such as Gaussian maximum likelihood, is used. It is also assumed that the distribution of the innovations is identically and independently distributed (i.i.d.) stable Paretian. It is seen via simulation that the proposed portmanteau tests do not converge well to the corresponding limiting distributions for practical series length so a Monte Carlo test is suggested. Simulation experiments show that the proposed Monte Carlo test procedure works effectively. Two illustrative applications to actual data are provided to demonstrate that an incorrect conclusion may result if the usual portmanteau test based on the finite variance assumption is used.     

EXACT MAXIMUM LIKELIHOOD ESTIMATE AND LAGRANGE MULTIPLIER TEST STATISTIC FOR ARMA MODELS

Abstract. The paper provides a method for the computation of the derivatives of the exact log likelihood function of a Gaussian time series. Based on this result and using Fisher's scoring technique, an efficient method for computing the maximum likelihood estimates for an autoregressive moving average model has been obtained. Simulations suggest that the new procedure is as fast as the Box and Jenkins conditional least squares method. In a similar way, a procedure is derived to compute the Lagrange multiplier test statistics for testing the goodness of fit of the model.     

TESTING CHANGE-POINTS IN THE EXPLOSIVE GAUSSIAN AUTOREGRESSIVE PROCESSES

Abstract. A functional limit theorem with a particular function class and topology is derived for non-ergodic type time series. This limit theorem allows us to study the asymptotic law of the associated likelihood ratio test (LRT) statistic for testing the presence of a change in the covariance parameter in the explosive Gaussian autoregressive model. We show that the level of the LRT cannot be approximated without introducing appropriate normalization. The limit law of a particular weighted likelihood ratio test is examined through a simulation study and is compared with the well-known Kolmogorov distribution obtained in the stationary case; we conclude that for practical applications when the root is really close to unity one can use the same thresholds as in the stationary case. This procedure is applied to the study of three real time series known to be non-stationary.     

Generalized autoregressive moving average models with GARCH errors

One of the important and widely used classes of models for non-Gaussian time series is the generalized autoregressive model average models (GARMA), which specifies an ARMA structure for the conditional mean process of the underlying time series. However, in many applications one often encounters conditional heteroskedasticity. In this article, we propose a new class of models, referred to as GARMA-GARCH models, that jointly specify both the conditional mean and conditional variance processes of a general non-Gaussian time series. Under the general modeling framework, we propose three specific models, as examples, for proportional time series, non-negative time series, and skewed and heavy-tailed financial time series. Maximum likelihood estimator (MLE) and quasi Gaussian MLE are used to estimate the parameters. Simulation studies and three applications are used to demonstrate the properties of the models and the estimation procedures.     

FAST LINEAR ESTIMATION METHODS FOR VECTOR AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. Three linear methods for estimating parameter values of vector auto-regressive moving-average (VARMA) models which are in general at least an order of magnitude faster than maximum likelihood estimation are developed in this paper. Simulation results for different model structures with varying numbers of component series and observations suggest that the accuracy of these procedures is in most cases comparable with maximum likelihood estimation. Procedures for estimating parameter standard error are also discussed and used for identification of nonzero elements in the VARMA polynomial structures. These methods can also be used to establish the order of the VARMA structure. We note, however, that the primary purpose of these estimates is to generate initial estimates for the nonzero parameters in order to reduce subsequent computational time of more efficient estimation procedures such as exact maximum likelihood.     

GQL Versus Conditional GQL Inferences for Non‐Stationary Time Series of Counts with Overdispersion

Abstract. This article proposes an autoregressive model for time series of counts with non‐stationary means, variances and covariances as functions of certain time‐dependant covariates. For the estimation of the regression, overdispersion and correlation index parameters, a conditional generalized quasilikelihood (CGQL) approach is developed under the assumption that the count responses marginally satisfy the first two moments of a negative binomial distribution. Thus this CGQL approach avoids the use of the likelihood or so‐called partial likelihood of the data which are known to be extremely complicated in the present non‐stationary time series set‐up. It is shown through an extensive simulation study that the proposed CGQL approach performs very well in estimating the parameters of the model. This is also shown that the CGQL approach performs better than an existing GQL approach, especially for the estimation of the overdispersion parameter of the model.     

Bayesian analysis of switching ARCH models

We consider a time series model with autoregressive conditional heteroscedasticity that is subject to changes in regime. The regimes evolve according to a multistate latent Markov switching process with unknown transition probabilities, and it is the constant in the variance process of the innovations that is subject to regime shifts. The joint estimation of the latent process and all model parameters is performed within a Bayesian framework using the method of Markov chain Monte Carlo (MCMC) simulation. We perform model selection with respect to the number of states and the number of autoregressive parameters in the variance process using Bayes factors and model likelihoods. To this aim, the model likelihood is estimated by the method of bridge sampling. The usefulness of the sampler is demonstrated by applying it to the data set previously used by Hamilton and Susmel (1994 ) who investigated models with switching autoregressive conditional heteroscedasticity using maximum likelihood methods. The paper concludes with some issues related to maximum likelihood methods, to classical model selection, and to potential straightforward extensions of the model presented here.     

Parametric Spectral Discrimination

This article is concerned with determining whether two independent time series have been generated by underlying stochastic processes with the same spectral shape. There are many methods that do so using the periodogram. Alternative approaches test for the equality of a finite number of autocovariances or autocorrelations. Non‐parametric methods usually have low power when compared with parametric methods. The parametric approach we introduce fits autoregressions to the two time series and tests whether the model parameters are equal using a likelihood ratio test. The test performs well when the time series are from autoregressions. However, problems arise when this is not the case. A modification to the test is proposed, which fits fixed order autoregressions. Simulations show that the modified test performs well even when the two time series are not from autoregressive processes. The parametric approach is shown to outperform non‐parametric alternatives in a power study.     

Oracle M‐Estimation for Time Series Models

We propose a thresholding M‐estimator for multivariate time series. Our proposed estimator has the oracle property that its large‐sample properties are the same as of the classical M‐estimator obtained under the a priori information that the zero parameters were known. We study the consistency of the standard block bootstrap, the centred block bootstrap and the empirical likelihood block bootstrap distributions of the proposed M‐estimator. We develop automatic selection procedures for the thresholding parameter and for the block length of the bootstrap methods. We present the results of a simulation study of the proposed methods for a sparse vector autoregressive VAR(2) time series model. The analysis of two real‐world data sets illustrate applications of the methods in practice.