Likelihood analysis of a first-order autoregressive model with exponential innovations

Abstract. This paper derives the exact distribution of the maximum likelihood estimator of a first-order linear autoregression with an exponential disturbance term. We also show that, even if the process is stationary, the estimator is T -consistent, where T is the sample size. In the unit root case, the estimator is T ²-consistent, while, in the explosive case, the estimator is ρ ^T -consistent. Further, the likelihood ratio test statistic for a simple hypothesis on the autoregressive parameter is asymptotically uniform for all values of the parameter.     

MAXIMUM LIKELIHOOD ESTIMATION FOR AUTOREGRESSIVE PROCESSES DISTURBED BY A MOVING AVERAGE

Abstract. Maximum likelihood estimation for stationary autoregressive processes when the signal is subject to a moving-average sampling error is discussed. A modified maximum likelihood estimator is proposed. An algorithm for computing derivatives of the modified likelihood is suggested. Maximum likelihood estimators of the parameter vector are shown to be strongly consistent and to have a multivariate normal limiting distribution. A Monte Carlo simulation shows that the modified maximum likelihood estimator performs better than other available estimators. US current labour force data are analysed as an example.     

Consistent Estimation of Linear and Non-linear Autoregressive Models with Markov Regime

An autoregressive process with Markov regime is an autoregressive process for which the regression function at each time-point is given by a (non-observable) Markov chain. We examine maximum likelihood estimation for such models and show consistency of a conditional maximum likelihood estimator. Also identifiability issues are discussed     

Testing for a Unit Root in Autoregressive Moving-average Models with Missing Data

Testing for a single autoregressive unit root in an autoregressive moving-average (ARMA) model is considered in the case when data contain missing values. The proposed test statistics are based on an ordinary least squares type estimator of the unit root parameter which is a simple approximation of the one-step Newton–Raphson estimator. The limiting distributions of the test statistics are the same as those of the regression statistics in AR(1) models tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Stat. Assoc . 74 (1979), 427–31) for the complete data situation. The tests accommodate models with a fitted intercept and a fitted time trend.     

ALTERNATIVE ESTIMATORS AND UNIT ROOT TESTS FOR THE AUTOREGRESSIVE PROCESS

Abstract. We compare several estimators for the second-order autoregressive process and compare the associated tests for a unit root. Monte Carlo results are reported for the ordinary least squares estimator, the simple symmetric least squares estimator and the weighted symmetric least squares estimator. The weighted symmetric least squares estimator of the autoregressive parameters generally has smaller mean square error than that of the ordinary least squares estimator, particularly when one root is close to one in absolute value. For the second-order model with known zero intercept, the one-sided ordinary least squares test for a unit root is more powerful than the symmetric tests. For the model with an estimated intercept, the one-sided weighted symmetric least squares test is the most powerful test.     

TESTING FOR A UNIT ROOT IN AN AR(1) TIME SERIES USING IRREGULARLY OBSERVED DATA

Abstract. For an AR(1) model having a unit root with nonconsecutively observed or missing data we consider the ordinary least squares estimator, the one-step Newton-Raphson estimator and an ordinary least squares type estimator which is a simple approximation of the Newton-Raphson estimator. It is shown that the limiting distributions of these estimators of the unit root are the same as those of the regression estimators as tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Statist. Assoc. 74 (1979), 427–31) for the complete data situation. Simulation results show that our proposed unit root tests perform very well for small samples.     

RESULTS ON ESTIMATION AND TESTING FOR A UNIT ROOT IN THE NONSTATIONARY AUTOREGRESSIVE MOVING-AVERAGE MODEL

Abstract. We review the limiting distribution theory for Gaussian estimation of the univariate autoregressive moving-average (ARMA) model in the presence of a unit root in the autoregressive (AR) operator, and present the asymptotic distribution of the associated likelihood ratio (LR) test statistic for testing for a unit root in the ARMA model. The finite sample properties of the LR statistic as well as other unit root test procedures for the ARMA model are examined through a limited simulation study. We conclude that, for practical empirical work that relies on standard computations, the LR test procedure generally performs better than other standard procedures in the presence of a substantial moving-average component in the ARMA model.     

The restricted likelihood ratio test at the boundary in autoregressive series

Abstract.  The restricted likelihood ratio test, RLRT, for the autoregressive coefficient in autoregressive models has recently been shown to be second-order pivotal when the autoregressive coefficient is in the interior of the parameter space and so is very well approximated by the     distribution. In this article, the non-standard asymptotic distribution of the RLRT for the unit root boundary value is obtained and is found to be almost identical to that of the     in the right tail. Together, these two results imply that the     distribution approximates the RLRT distribution very well even for near unit root series and transitions smoothly to the unit root distribution.     

ESTIMATION OF THE MULTIVARIATE AUTOREGRESSIVE MOVING AVERAGE HAVING PARAMETER RESTRICTIONS AND AN APPLICATION TO ROTATIONAL SAMPLING

Abstract. The vector autoregressive moving average model with nonlinear parametric restrictions is considered. A simple and easy-to-compute Newton-Raphson estimator is proposed that approximates the restricted maximum likelihood estimator which takes full advantage of the information contained in the restrictions. In the case when there are no parametric restrictions, our Newton-Raphson estimator is equivalent to the estimator proposed by Reinsel et al. (Maximum likelihood estimators in the multivariate autoregressive moving-average model from a generalized least squares view point. J. Time Ser. Anal. 13 (1992), 133–45). The Newton-Raphson estimation procedure also extends to the vector ARMAX model. Application of our Newton-Raphson estimation method in rotational sampling problems is discussed. Simulation results are presented for two different restricted models to illustrate the estimation procedure and compare its performance with that of two alternative procedures that ignore the parametric restrictions.     

Maximum Likelihood Estimation for a First‐Order Bifurcating Autoregressive Process with Exponential Errors

Abstract. Exact and asymptotic distributions of the maximum likelihood estimator of the autoregressive parameter in a first‐order bifurcating autoregressive process with exponential innovations are derived. The limit distributions for the stationary, critical and explosive cases are unified via a single pivot using a random normalization. The pivot is shown to be asymptotically exponential for all values of the autoregressive parameter.     

INFERENCE FOR A SPECIAL BILINEAR TIME‐SERIES MODEL

It is well known that estimating bilinear models is quite challenging. Many different ideas have been proposed to solve this problem. However, there is not a simple way to do inference even for its simple cases. This article proposes a generalized autoregressive conditional heteroskedasticity‐type maximum likelihood estimator for estimating the unknown parameters for a special bilinear model. It is shown that the proposed estimator is consistent and asymptotically normal under only finite fourth moment of errors.     

Computationally efficient methods for two multivariate fractionally integrated models

Abstract.  We discuss two distinct multivariate time-series models that extend the univariate ARFIMA (autoregressive fractionally integrated moving average) model. We discuss the different implications of the two models and describe an extension to fractional cointegration. We describe algorithms for computing the covariances of each model, for computing the quadratic form and approximating the determinant for maximum likelihood estimation and for simulating from each model. We compare the speed and accuracy of each algorithm with existing methods individually. Then, we measure the performance of the maximum likelihood estimator and of existing methods in a Monte Carlo. These algorithms are much more computationally efficient than the existing algorithms and are equally accurate, making it feasible to model multivariate long memory time series and to simulate from these models. We use maximum likelihood to fit models to data on goods and services inflation in the United States.     

Local Likelihood for non‐parametric ARCH(1) models

Abstract. We propose a non‐parametric local likelihood estimator for the log‐transformed autoregressive conditional heteroscedastic (ARCH) (1) model. Our non‐parametric estimator is constructed within the likelihood framework for non‐Gaussian observations: it is different from standard kernel regression smoothing, where the innovations are assumed to be normally distributed. We derive consistency and asymptotic normality for our estimators and show, by a simulation experiment and some real‐data examples, that the local likelihood estimator has better predictive potential than classical local regression. A possible extension of the estimation procedure to more general multiplicative ARCH(p) models with p > 1 predictor variables is also described.     

An approximate likelihood function for panel data with a mixed ARMA(p, q) remainder disturbance model

Abstract. An approximate likelihood function for panel data with an autoregressive moving‐average (ARMA)(p, q) model remainder disturbance is presented and Whittle's approximate maximum likelihood estimator (MLE) is used to yield an asymptotic estimator. Although an asymptotic approach, the power test is quite successful for estimating and testing. In this approach, we do not need to calculate the transformation matrix in exact form. Through the Riemann sum approach, we can construct a simple approximate concentrated likelihood function. In addition, the model is also extended to the restricted maximum likelihood (REML) function, in which the package of Gilmour, Thompson and Cullis [Biometrics (1995) Vol. 51, pp. 1440–1450] is applied without difficulty. In the case study, we implement the model on the characteristic line for the investment analysis of Taiwanese computer motherboard makers.     

Sequential estimation for the parameters of a stationary auto regressive model

This paper considers the problem of sequential point estimation and fixed accuracy confidence set procedures of autoregressive parameters in a ρ-th order stationary autoregressive model. The sequential estimator proposed here is based on the least squares estimator and is shown to be risk efficient as the cost of estimation error tends to infinity. Furthermore, the proposed procedure for fixed-width confidence set is shown to be both asymptotically consistent and asymptotically efficient as the width approaches zero.     

Efficient use of higher-lag autocorrelations for estimating autoregressive processes

The Yule–Walker estimator is commonly used in time-series analysis, as a simple way to estimate the coefficients of an autoregressive process. Under strong assumptions on the noise process, this estimator possesses the same asymptotic properties as the Gaussian maximum likelihood estimator. However, when the noise is a weak one, other estimators based on higher-order empirical autocorrelations can provide substantial efficiency gains. This is illustrated by means of a first-order autoregressive process with a Markov-switching white noise. We show how to optimally choose a linear combination of a set of estimators based on empirical autocorrelations. The asymptotic variance of the optimal estimator is derived. Empirical experiments based on simulations show that the new estimator performs well on the illustrative model.     

Quasi‐Maximum Likelihood Estimation for a Class of Continuous‐time Long‐memory Processes

Tsai and Chan (2003) has recently introduced the Continuous‐time Auto‐Regressive Fractionally Integrated Moving‐Average (CARFIMA) models useful for studying long‐memory data. We consider the estimation of the CARFIMA models with discrete‐time data by maximizing the Whittle likelihood. We show that the quasi‐maximum likelihood estimator is asymptotically normal and efficient. Finite‐sample properties of the quasi‐maximum likelihood estimator and those of the exact maximum likelihood estimator are compared by simulations. Simulations suggest that for finite samples, the quasi‐maximum likelihood estimator of the Hurst parameter is less biased but more variable than the exact maximum likelihood estimator. We illustrate the method with a real application.     

Consistent Estimation for Non-Gaussian Non-Causal Autoregessive Processes

Traditional estimation based on least squares or Gaussian likelihood cannot distinguish between causal and non-causal representation of a stationary autoregressive (AR) process. Breidt et al . (Maximum likelihood estimation for non-causal autoregressive processes. J. Multivariate Anal. 36 (1991), 175–98) proved the existence of a consistent likelihood estimation of possibly non-causal AR processes; however, in this case an existence result is not very useful since the likelihood function generally exhibits multiple maxima. Moreover the method assumes full knowledge of the distribution of the innovation process. This paper shows a constructive proof that a modified L ₁ estimate is consistent if the innovation process has a stable law distribution with index α∈ (1, 2). It is also shown that neither non-Gaussianity nor infinite variance is sufficient to ensure consistency.     

DIFFERENTIAL GEOMETRY OF ARMA MODELS

Abstract. A general approach for the development of a statistical inference on autoregressive moving-average (ARMA) models is presented based on geometric arguments. ARMA models are characterized as members of the curved exponential family. Geometric properties of ARMA models are computed and used to suggest parameter transformations that satisfy predetermined properties. In particular, the effect on the asymptotic bias of the maximum likelihood estimator of model parameters is illustrated. Hypothesis testing of parameters is discussed through the application of a modified form of the likelihood ratio test statistic.