On Residual Variance Estimation in Autoregressive Models

In this paper we consider time series models belonging to the autoregressive (AR) family and deal with the estimation of the residual variance. This is important because estimates of the variance are involved in, for example, confidence sets for the parameters of the model, estimation of the spectrum, expressions for the estimated error of prediction and sample quantities used to make inferences about the order of the model. We consider the asymptotic biases for moment and least squares estimators of the residual variance, and compare them with known results when available and with those for maximum likelihood estimators under normality. Simulation results are presented for finite samples     

YULE-WALKER ESTIMATES FOR CONTINUOUS-TIME AUTOREGRESSIVE MODELS

Abstract. I consider continuous-time autoregressive processes of order p and develop estimators of the model parameters based on Yule-Walker type equations. For continuously recorded data, it is shown that these estimators are least squares estimators and have the same asymptotic distribution as maximum likelihood estimators.
In practice, though, data can only be observed discretely. For discrete data, I consider approximations to the continuous-time estimators. It is shown that some of these discrete-time estimators are asymptotically biased. Alternative estimators based on the autocovariance function are suggested. These are asymptotically unbiased and are a fast alternative to the maximum likelihood estimators described by Jones. They may also be used as starting values for maximum likelihood estimation.     

FISHER'S INFORMATION MATRIX FOR SEASONAL AUTOREGRESSIVE-MOVING AVERAGE MODELS

Abstract. Two procedures are described for obtaining Fisher's information matrix of a multiplicative seasonal autoregressive-moving average process. They can be useful in determining the asymptotic covariance matrix of Gaussian maximum likelihood estimators of the parameters. Components of the information matrix are expressed in the first procedure as integrals of rational functions. The second procedure makes use of the autocorrelation function of several autoregressive processes.     

Efficient use of higher-lag autocorrelations for estimating autoregressive processes

The Yule–Walker estimator is commonly used in time-series analysis, as a simple way to estimate the coefficients of an autoregressive process. Under strong assumptions on the noise process, this estimator possesses the same asymptotic properties as the Gaussian maximum likelihood estimator. However, when the noise is a weak one, other estimators based on higher-order empirical autocorrelations can provide substantial efficiency gains. This is illustrated by means of a first-order autoregressive process with a Markov-switching white noise. We show how to optimally choose a linear combination of a set of estimators based on empirical autocorrelations. The asymptotic variance of the optimal estimator is derived. Empirical experiments based on simulations show that the new estimator performs well on the illustrative model.     

Rank‐based estimation for autoregressive moving average time series models

Abstract. We establish asymptotic normality and consistency for rank‐based estimators of autoregressive‐moving average model parameters. The estimators are obtained by minimizing a rank‐based residual dispersion function similar to the one given by L.A. Jaeckel [Ann. Math. Stat. Vol. 43 (1972) 1449–1458]. These estimators can have the same asymptotic efficiency as maximum likelihood estimators and are robust. The quality of the asymptotic approximations for finite samples is studied via simulation.     

Effects of outliers on the identification and estimation of GARCH models

Abstract. This paper analyses how outliers affect the identification of conditional heteroscedasticity and the estimation of generalized autoregressive conditionally heteroscedastic (GARCH) models. First, we derive the asymptotic biases of the sample autocorrelations of squared observations generated by stationary processes and show that the properties of some conditional homoscedasticity tests can be distorted. Second, we obtain the asymptotic and finite sample biases of the ordinary least squares (OLS) estimator of ARCH(p) models. The finite sample results are extended to generalized least squares (GLS), maximum likelihood (ML) and quasi‐maximum likelihood (QML) estimators of ARCH(p) and GARCH(1,1) models. Finally, we show that the estimated asymptotic standard deviations are biased estimates of the sample standard deviations.     

Multivariate modelling of the autoregressive random variance process

The autoregressive random variance (ARV) model proposed by Taylor (Financial returns modelled by the product of two stochastic processes, a study of daily sugar prices 1961–79. In Time Series Analysis: Theory and Practice 1 (ed. O. D. Anderson). Amsterdam: North-Holland, 1982, pp. 203–26) is useful in modelling stochastic changes in the variance structure of a time series. In this paper we focus on a general multivariate ARV model. A traditional EM algorithm is derived as the estimation method. The proposed EM approach is simple to program, computationally efficient and numerically well behaved. The asymptotic variance--covariance matrix can be easily computed as a by-product using a well-kno wn asymptotic result for extremum estimators. A result that is of interest in itself is that the dimension of the augmented state space form used in computing the variance–covariance matrix can be shown to be greatly reduced, resulting in greater computational efficiency . The multivariate ARV model considered here is useful in studying the lead–lag (causality) relationship of the variance structure across different time series. As an example, the leading effect of Thailand on Malaysia in terms of vari ance changes in the stock indices is demonstrated.     

Bootstrapping a weighted linear estimator of the ARCH parameters

Abstract.  A standard assumption while deriving the asymptotic distribution of the quasi maximum likelihood estimator in ARCH models is that all ARCH parameters must be strictly positive. This assumption is also crucial in deriving the limit distribution of appropriate linear estimators (LE). We propose a weighted linear estimator (WLE) of the ARCH parameters in the classical ARCH model and show that its limit distribution is multivariate normal even when some of the ARCH coefficients are zero. The asymptotic dispersion matrix involves unknown quantities. We consider appropriate bootstrapped version of this WLE and prove that it is asymptotically valid in the sense that the bootstrapped distribution (given the data) is a consistent estimate (in probability) of the distribution of the WLE. Although we do not show theoretically that the bootstrap outperforms the normal approximation, our simulations demonstrate that it yields better approximations than the limiting normal.     

Further Results on Pseudo-Maximum Likelihood Estimation and Testing in the Constant Elasticity of Variance Continuous Time Model

Constant elasticity volatility processes have been shown to be useful, for example, to encompass a number of existing models that have closed-form likelihood functions. In this article, we extend the existing literature in two directions: first we find explicit closed form solutions of the pseudo maximum likelihood estimators (MLEs) by discretizing the diffusion function and we provide their asymptotic theory in the context of the constant elasticity of variance (CEV) model characterized by a general CEV parameter ρ ≥ 0. Second we obtain bias expansions for those pseudo MLEs also in terms of ρ ≥ 0. We provide a general framework since only the cases with ρ = 0 and ρ = 0.5 have been considered in the literature so far. When the time series is not positive almost surely, we need to impose the restriction that ρ is a non-negative integer.     

Bayesian analysis of switching ARCH models

We consider a time series model with autoregressive conditional heteroscedasticity that is subject to changes in regime. The regimes evolve according to a multistate latent Markov switching process with unknown transition probabilities, and it is the constant in the variance process of the innovations that is subject to regime shifts. The joint estimation of the latent process and all model parameters is performed within a Bayesian framework using the method of Markov chain Monte Carlo (MCMC) simulation. We perform model selection with respect to the number of states and the number of autoregressive parameters in the variance process using Bayes factors and model likelihoods. To this aim, the model likelihood is estimated by the method of bridge sampling. The usefulness of the sampler is demonstrated by applying it to the data set previously used by Hamilton and Susmel (1994 ) who investigated models with switching autoregressive conditional heteroscedasticity using maximum likelihood methods. The paper concludes with some issues related to maximum likelihood methods, to classical model selection, and to potential straightforward extensions of the model presented here.     

Least tail‐trimmed squares for infinite variance autoregressions

We develop a robust least squares estimator for autoregressions with possibly heavy tailed errors. Robustness to heavy tails is ensured by negligibly trimming the squared error according to extreme values of the error and regressors. Tail‐trimming ensures asymptotic normality and super‐‐convergence with a rate comparable to the highest achieved amongst M‐estimators for stationary data. Moreover, tail‐trimming ensures robustness to heavy tails in both small and large samples. By comparison, existing robust estimators are not as robust in small samples, have a slower rate of convergence when the variance is infinite, or are not asymptotically normal. We present a consistent estimator of the covariance matrix and treat classic inference without knowledge of the rate of convergence. A simulation study demonstrates the sharpness and approximate normality of the estimator, and we apply the estimator to financial returns data. Finally, tail‐trimming can be easily extended beyond least squares estimation for a linear stationary AR model. We discuss extensions to quasi‐maximum likelihood for GARCH, weighted least squares for a possibly non‐stationary random coefficient autoregression, and empirical likelihood for robust confidence region estimation, in each case for models with possibly heavy tailed errors.     

Estimation and testing for the parameters of ARCH(q) under ordered restriction

Abstract.  In this paper, we study a stationary ARCH( q ) model with parameters α ₀, α ₁, α ₂,…, α _q . It is known that the model requires all parameters α _i to be non-negative, but sometimes the usual algorithm based on Newton–Raphson's method leads us to obtain some negative solutions. So this study proposes a method of computing the maximum likelihood estimator (MLE) of parameters under the non-negative restriction. A similar method is also proposed for the case where the parameters are restricted by a simple order: α ₁≥ α ₂≥⋯≥ α _p . The strong consistency of the above two estimators is discussed. Furthermore, we consider the problem of testing homogeneity of parameters against the simple order restriction. We give the likelihood ratio (LR) test statistic for the testing problem and derive its asymptotic null distribution.     

Estimation of regression and dynamic dependence paremeters for non‐stationary multinomial time series

In a time‐series regression setup, multinomial responses along with time dependent observable covariates are usually modelled by certain suitable dynamic multinomial logistic probabilities. Frequently, the time‐dependent covariates are treated as a realization of an exogenous random process and one is interested in the estimation of both the regression and the dynamic dependence parameters conditional on this realization of the covariate process. There exists a partial likelihood estimation approach able to deal with the general dependence structures arising from the influence of both past covariates and past multinomial responses on the covariates at a given time by sequentially conditioning on the history of the joint process (response and covariates), but it provides standard errors for the estimators based on the observed information matrix, because such a matrix happens to be the Fisher information matrix obtained by conditioning on the whole history of the joint process. This limitation of the partial likelihood approach holds even if the covariate history is not influeced by lagged response outcomes. In this article, a general formulation of the auto‐covariance structure of a multinomial time series is presented and used to derive an explicit expression for the Fisher information matrix conditional on the covariate history, providing the possibility of computing the variance of the maximum likelihood estimators given a realization of the covariate process for the multinomial‐logistic model. The difference between the standard errors of the parameter estimators under these two conditioning schemes (covariates Vs. joint history) is illustrated through an intensive simulation study based on the premise of an exogenous covariate process.     

BOOTSTRAPPING STATIONARY AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. In this paper we develop an asymptotic theory for application of the bootstrap to stationary stochastic processes of autoregressive moving-average (ARMA) type, with known order ( p, q ). We give a proof of the asymptotic validity of the bootstrap proposal applied to M estimators for the unknown parameter vector of the process. For this purpose we derive an asymptotic expansion for M estimators in ARMA models and construct an estimate for the unknown distribution function of the residuals which in principle are not observable. A small simulation study is also included.     

Second order asymptotic efficlency in terms of the asymptotic variance of seqrential estimation procedures in the presence of nrisance parameters

In the presence of nuisance parameters, the Bhattacharyya type bound for the asymptotic variance of estimation procedures is obtained. It is shown that the modified maximum likelihood (ML) estimation procedures together with any stopping rule does not attain the bound. Further it is shown that the modified ML estimation procedure with the appropriate stopping rule is second order asymptotically efficient in some class of estimation procedures in the sense that it attains the lower bound for the asymptotic variance in the class.     

An approximate likelihood function for panel data with a mixed ARMA(p, q) remainder disturbance model

Abstract. An approximate likelihood function for panel data with an autoregressive moving‐average (ARMA)(p, q) model remainder disturbance is presented and Whittle's approximate maximum likelihood estimator (MLE) is used to yield an asymptotic estimator. Although an asymptotic approach, the power test is quite successful for estimating and testing. In this approach, we do not need to calculate the transformation matrix in exact form. Through the Riemann sum approach, we can construct a simple approximate concentrated likelihood function. In addition, the model is also extended to the restricted maximum likelihood (REML) function, in which the package of Gilmour, Thompson and Cullis [Biometrics (1995) Vol. 51, pp. 1440–1450] is applied without difficulty. In the case study, we implement the model on the characteristic line for the investment analysis of Taiwanese computer motherboard makers.     

Ranking and Selection Techniques with Overlapping Variance Estimators for Simulations

Abstract

Some ranking and selection (R&S) procedures for steady-state simulation require estimates of the asymptotic variance parameter of each system to guarantee a certain probability of correct selection. In this paper, we show that the performance of such R&S procedures depends highly on the quality of the variance estimates that are used. In fact, we study the performance of R&S procedures using three new variance estimators—overlapping area, overlapping Cramér–von Mises, and overlapping modified jackknifed Durbin–Watson estimators—that show better long-run performance than other estimators previously used in conjunction with R&S procedures for steady-state simulations.     

Conway–Maxwell–Poisson Autoregressive Moving Average Model for Equidispersed,Underdispersed, and Overdispersed Count Data

In this work, we propose a dynamic regression model based on the ConwayŮMaxwell–Poisson (CMP) distribution with time-varying conditional mean depending on covariates and lagged observations. This new class of ConwayŮMaxwell–Poisson autoregressive moving average (CMP-ARMA) models is suitable for the analysis of time series of counts. The CMP distribution is a two-parameter generalization of the Poisson distribution that allows the modeling of underdispersed, equidispersed, and overdispersed data. Our main contribution is to combine this dispersion flexibility with the inclusion of lagged terms to model the conditional mean response, inducing an autocorrelation structure, usually relevant in time series. We present the conditional maximum likelihood estimation, hypothesis testing inference, diagnostic analysis, and forecasting along with their asymptotic properties. In particular, we provide closed-form expressions for the conditional score vector and conditional Fisher information matrix. We conduct a Monte Carlo experiment to evaluate the performance of the estimators in finite sample sizes. Finally, we illustrate the usefulness of the proposed model by exploring two empirical applications.     

REDUCTION OF THE ASYMPTOTIC BIAS OF AUTOREGRESSIVE AND SPECTRAL ESTIMATORS BY TAPERING

Abstract. The asymptotic bias to terms of order T ^-1, where T is the observed series length, is studied for estimators of the coefficients and disturbance variance in an AR( p ) model. Reduction of the asymptotic bias by tapering is established and, if the tapering function is defined appropriately to depend on T , not only is the asymptotic bias reduced, but the asymptotic distribution of the estimators is not altered. In addition, the asymptotic biases of other time series parameter estimators constructed from the sample covariance function, such as several types of spectral estimators, can also be reduced by tapering.     

VALIDITY OF EDGEWORTH EXPANSIONS FOR STATISTICS OF TIME SERIES

Abstract. In this paper, we discuss the validity of the multivariate Edgeworth expansion of distribution functions of statistics which need not be standardized sums of independent and identically distributed vectors. We apply this result to statistics of time series. In particular, we shall give the asymptotic expansion of the distribution of the maximum likelihood estimator of a parameter of a circular autoregresive moving average process.