Bayesian selection of threshold autoregressive models

Abstract.  An approach to Bayesian model selection in self-exciting threshold autoregressive (SETAR) models is developed within a reversible jump Markov chain Monte Carlo (RJMCMC) framework. Our approach is examined via a simulation study and analysis of the Zurich monthly sunspots series. We find that the method converges rapidly to the optimal model, whilst efficiently exploring suboptimal models to quantify model uncertainty. A key finding is that the parsimony of the model selected is influenced by the specification of prior information, which can be examined and subjected to criticism. This is a strength of the Bayesian approach, allowing physical understanding to constrain the model selection algorithm.     

Mixtures of Nonlinear Poisson Autoregressions

We study nonlinear infinite order Markov switching integer‐valued ARCH models for count time series data. Markov switching models take into account complex dynamics and can deal with several stylistic facts of count data including proper modelling of nonlinearities, overdispersion and outliers. We study structural properties of those models. Under mild conditions, we prove consistency and asymptotic normality of the maximum likelihood estimator for the case of finite order autoregression. In addition, we give conditions which imply that the marginal likelihood ratio test, for testing the number of regimes, converges to a Gaussian process. This result enables us to prove that the BIC provides a consistent estimator for selecting the true number of regimes. A real data example illustrates the methodology and compares this approach with alternative methods.     

Reversible Jump Markov Chain Monte Carlo Strategies for Bayesian Model Selection in Autoregressive Processes

Abstract.  This paper addresses the problem of Bayesian inference in autoregressive (AR) processes in the case where the correct model order is unknown. Original hierarchical prior models that allow the stationarity of the model to be enforced are proposed. Obtaining the quantities of interest, such as parameter estimates, predictions of future values of the time series, posterior model-order probabilities, etc., requires integration with respect to the full posterior distribution, an operation which is analytically intractable. Reversible jump Markov chain Monte Carlo (MCMC) algorithms are developed to perform the required integration implicitly by simulating from the posterior distribution. The methods developed are evaluated in simulation studies on a number of synthetic and real data sets.     

On Bayesian analysis of nonlinear continuous‐time autoregression models

Abstract. This article introduces a method for performing fully Bayesian inference for nonlinear conditional autoregressive continuous‐time models, based on a finite skeleton of observations. Our approach uses Markov chain Monte Carlo and involves imputing data from times at which observations are not made. It uses a reparameterization technique for the missing data, and because of the non‐Markovian nature of the models, it is necessary to adopt an overlapping blocks scheme for sequentially updating segments of missing data. We illustrate the methodology using both simulated data and a data set from the S & P 500 index.     

Parameter Estimation in the Error‐in‐Variables Models Using the Gibbs Sampler

Least squares and maximum likelihood techniques have long been used in parameter estimation problems. However, those techniques provide only point estimates with unknown or approximate uncertainty information. Bayesian inference coupled with the Gibbs Sampler is an approach to parameter estimation that exploits modern computing technology. The estimation results are complete with exact uncertainty information. The Error‐in‐Variables model (EVM) approach is investigated in this study. In it, both dependent and independent variables contain measurement errors, and the true values and uncertainties of all measurements are estimated. This EVM set‐up leads to unusually large dimensionality in the estimation problem, which makes parameter estimation very difficult with classical techniques. In this paper, an innovative way of performing parameter estimation is introduced to chemical engineers. The paper shows that the method is simple and efficient; as well, complete and accurate uncertainty information about parameter estimates is readily available. Two real‐world EVM examples are demonstrated: a large‐scale linear model and an epidemiological model. The former is simple enough for most readers to understand the new concepts without difficulty. The latter has very interesting features in that a Poisson distribution is assumed, and a parameter with known distribution is retained while other unknown parameters are estimated. The Gibbs Sampler results are compared with those of the least squares.     

Bayesian analysis of multivariate Gaussian hidden Markov models with an unknown number of regimes

Multivariate Gaussian hidden Markov models with an unknown number of regimes are introduced here in the Bayesian setting and new efficient reversible jump Markov chain Monte Carlo algorithms for estimating both the dimension and the unknown parameters of the model are presented. Hidden Markov models are an extension of mixture models that can be applied to time series so as to classify the observations in a small number of groups, to understand when change points occur in the dynamics of the series and to model data heterogeneity through the switching among subseries with different means and covariance matrices. These aims can be achieved by assuming that the observed phenomenon is driven by a latent, or hidden, Markov chain. The methodology is illustrated through two different examples of multivariate time series.     

Time-varying autoregressions with model order uncertainty

We explore some aspects of the analysis of latent component structure in non-stationary time series based on time-varying autoregressive (TVAR) models that incorporate uncertainty on model order. Our modelling approach assumes that the AR coefficients evolve in time according to a random walk and that the model order may also change in time following a discrete random walk. In addition, we use a conjugate prior structure on the autoregressive coefficients and a discrete uniform prior on model order. Simulation from the posterior distribution of the model parameters can be obtained via standard forward filtering backward simulation algorithms. Aspects of implementation and inference on decompositions, latent structure and model order are discussed for a synthetic series and for an electroencephalogram (EEG) trace previously analysed using fixed order TVAR models.     

A Bayesian regime‐switching time‐series model

This article provides a new Bayesian approach for AR(2) time‐series models with multiple regime‐switching points. Our formulation of the regime‐switching model involves a binary discrete variable that indicates the regime change. This variable is specified to be detected by data in each regime. The model is estimated using Stochastic approximation Monte Carlo method proposed by Liang et al. [JASA (2007)]. This methodology is quite useful since it allows for fitting of more complex regime‐switching models without transition constraint. The proposed model is illustrated using simulated and real data such as GNP and US interest rate data.     

Estimation in the Mixture Transition Distribution Model

This paper introduces a new iterative algorithm for the estimation of the mixture transition distribution (MTD) model, which does not require the use of any specific external optimization procedure and can therefore be programmed in any computing language. Comparisons with previously published results show that this new algorithm performs at least as well as or better than other methods. The choice of initial values is also discussed.
The MTD model was designed for the modeling of high-order Markov chains and has already proved to be a useful tool for the analysis of different types of time series such as wind speeds and social relationships. In this paper, we also propose to use it for the modeling of one-dimensional spatial data. An application using a DNA sequence shows that this approach can lead to better results than the classical Potts model.     

Cross-validation Criteria for Setar Model Selection

Three cross-validation criteria, denoted by respectively C , C _c, and C _u, are proposed for selecting the orders of a self-exciting threshold autoregressive (SETAR) model when both the delay and the threshold value are unknown. The derivation of C is within a natural cross-validation framework. The criterion C _c is similar in spirit as AIC_c, a bias-corrected version of AIC for SETAR model selection introduced by Wong and Li (1998). The criterion C _u is a variant of C _c having a similar poperty as AIC_u, a model selection proposed by McQuarrie et al . (1997) for linear models. In a Monte Carlo study, the performance of each of the criteria C , C _c, C _u, AIC, AIC_c, AIC_u, and BIC is investigated in detail for various models and various sample sizes. It will be shown that C _u consistently outperforms all other criteria when the sample size is moderate to large.     

A Structural‐Factor Approach to Modeling High‐Dimensional Time Series and Space‐Time Data

This article considers a structural‐factor approach to modeling high‐dimensional time series and space‐time data by decomposing individual series into trend, seasonal, and irregular components. For ease in analyzing many time series, we employ a time polynomial for the trend, a linear combination of trigonometric series for the seasonal component, and a new factor model for the irregular components. The new factor model simplifies the modeling process and achieves parsimony in parameterization. We propose a Bayesian information criterion to consistently select the order of the polynomial trend and the number of trigonometric functions, and use a test statistic to determine the number of common factors. The convergence rates for the estimators of the trend and seasonal components and the limiting distribution of the test statistic are established under the setting that the number of time series tends to infinity with the sample size, but at a slower rate. We study the finite‐sample performance of the proposed analysis via simulation, and analyze two real examples. The first example considers modeling weekly PM_2.5 data of 15 monitoring stations in the southern region of Taiwan and the second example consists of monthly value‐weighted returns of 12 industrial portfolios.