Likelihood‐based Analysis of a Class of Generalized Long‐Memory Time Series Models

Abstract. This article introduces a family of ‘generalized long‐memory time series models’, in which observations have a specified conditional distribution, given a latent Gaussian fractionally integrated autoregressive moving‐average (ARFIMA) process. The observations may have discrete or continuous distributions (or a mixture of both). The family includes existing models such as ARFIMA models themselves, long‐memory stochastic volatility models, long‐memory censored Gaussian models and others. Although the family of models is flexible, the latent long‐memory process poses problems for analysis. Therefore, we introduce a Markov chain Monte Carlo sampling algorithm and develop a set of recursions which makes it feasible. This makes it possible, among other things, to carry out exact likelihood‐based analysis of a wide range of non‐Gaussian long‐memory models without resorting to the use of likelihood approximations. The procedure also yields predictive distributions that take into account model parameter uncertainty. The approach is demonstrated in two case studies.     

A Nonparametric Model for Stationary Time Series

Stationary processes are a natural choice as statistical models for time series data, owing to their good estimating properties. In practice, however, alternative models are often proposed that sacrifice stationarity in favour of the greater modelling flexibility required by many real‐life applications. We present a family of time‐homogeneous Markov processes with nonparametric stationary densities, which retain the desirable statistical properties for inference, while achieving substantial modelling flexibility, matching those achievable with certain non‐stationary models. A latent extension of the model enables exact inference through a trans‐dimensional Markov chain Monte Carlo method. Numerical illustrations are presented.     

Multiway continuous hidden Markov model‐based approach for fault detection and diagnosis

A fault detection and classification scheme that uses probabilistic inference based on multiway continuous hidden Markov models (MCHMM) which is capable of capturing complex system dynamics and uncertainty is proposed. A set of observations from normal and faulty runs of the system was collected and used to generate the training dataset. The training data is assumed to follow a finite Gaussian mixture model. The number of mixture components and associated parameters for the optimal Gaussian mixture fit of the observed data was computed subsequently by clustering using the Figueiredo–Jain algorithm for unsupervised learning. The segmental k‐means algorithm was used to compute the HMM parameters. The applicability of the proposed scheme is investigated for the case of an inverted pendulum system and a fluidized catalytic cracker. The monitoring results for the above cases with the proposed scheme was found to be superior to the multiway discrete hidden Markov model (MDHMM) based scheme in terms of the accuracy of fault detection, especially in case of noisy observations. © 2014 American Institute of Chemical Engineers AIChE J, 60: 2035–2047, 2014     

Latent mixed Markov modelling of smoking transitions using Monte Carlo bootstrapping

It has been established that measures and reports of smoking behaviours are subject to substantial measurement errors. Thus, the manifest Markov model which does not consider measurement error in observed responses may not be adequate to mathematically model changes in adolescent smoking behaviour over time. For this purpose we fit several Mixed Markov Latent Class (MMLC) models using data sets from two longitudinal panel studies--the third Waterloo Smoking Prevention study and the UWO smoking study, which have varying numbers of measurements on adolescent smoking behaviour. However, the conventional statistics used for testing goodness of fit of these models do not follow the theoretical chi-square distribution when there is data sparsity. The two data sets analysed had varying degrees of sparsity. This problem can be solved by estimating the proper distribution of fit measures using Monte Carlo bootstrap simulation. In this study, we showed that incorporating response uncertainty in smoking behaviour significantly improved the fit of a single Markov chain model. However, the single chain latent Markov model did not adequately fit the two data sets indicating that the smoking process was heterogeneous with regard to latent Markov chains. It was found that a higher percentage of students (except for never smokers) changed their smoking behaviours over time at the manifest level compared to the latent or true level. The smoking process generally accelerated with time. The students had a tendency to underreport their smoking behaviours while response uncertainty was estimated to be considerably less for the Waterloo smoking study which adopted the 'bogus pipeline' method for reducing measurement error while the UWO study did not. For the two-chain latent mixed Markov models, incorporating a 'stayer' chain to an unrestricted Markov chain led to a significant improvement in model fit for the UWO study only. For both data sets, the assumption for the existence of an independence chain did not lead to significant improvement in model fit. The unrestricted two-chain latent mixed Markov model led to a significant improvement of model fit compared to a simple latent Markov model, but this model was overparameterized when the latent transition probabilities and/or response probabilities were assumed nonstationary. For the other models, the manifest/latent transition probabilities and response probabilities (for the four-wave Waterloo study only) were tested to be nonstationary for both data sets.     

On Bayesian analysis of nonlinear continuous‐time autoregression models

Abstract. This article introduces a method for performing fully Bayesian inference for nonlinear conditional autoregressive continuous‐time models, based on a finite skeleton of observations. Our approach uses Markov chain Monte Carlo and involves imputing data from times at which observations are not made. It uses a reparameterization technique for the missing data, and because of the non‐Markovian nature of the models, it is necessary to adopt an overlapping blocks scheme for sequentially updating segments of missing data. We illustrate the methodology using both simulated data and a data set from the S & P 500 index.     

Soft sensor modelling of acrolein conversion based on hidden Markov model of principle component analysis and fireworks algorithm

Based on the existing propylene oxidation process, it is important to measure acrolein conversion for the production of acrylic acid. The gas chromatographic analyzer is generally used to analyze the acrolein conversion as an off‐line method. In this paper, a soft sensor modelling method of acrolein conversion based on the hidden Markov model with principle component analysis (PCA) and the fireworks algorithm (FWA) is proposed. Firstly, PCA is used to decrease the input variables of hidden Markov model. Then, FWA is applied to optimize the initial parameters of the hidden Markov model. Finally, the hidden Markov model based on PCA and the FWA is employed to predict the acrolein conversion. The proposed method is compared with the support vector machine (SVM), the artificial neural network (ANN), and the hidden Markov method (HMM) to show its superior performance.     

Convolution‐closed models for count time series with applications

There has recently been an upsurge of interest in time series models for count data. Many papers focus on the model with first‐order (Markov) dependence and Poisson innovations. Our paper considers practical models that can capture higher‐order dependence based on the work of Joe (1996). In this framework we are able to model both equidispersed and overdispersed marginal distributions of data. The latter is approached using generalized Poisson innovations. Central to the models is the use of the property of closure under convolution of certain families of random variables. The models can be thought of as stationary Markov chains of finite order. Parameter estimation is undertaken by maximum likelihood, inference procedures are considered and means of assessing model adequacy employed. Applications to two new data sets are provided.     

Evaluating Specification Tests for Markov‐Switching Time‐Series Models

Abstract. We evaluate the performance of several specification tests for Markov regime‐switching time‐series models. We consider the Lagrange multiplier (LM) and dynamic specification tests of Hamilton (1996) and Ljung–Box tests based on both the generalized residual and a standard‐normal residual constructed using the Rosenblatt transformation. The size and power of the tests are studied using Monte Carlo experiments. We find that the LM tests have the best size and power properties. The Ljung–Box tests exhibit slight size distortions, though tests based on the Rosenblatt transformation perform better than the generalized residual‐based tests. The tests exhibit impressive power to detect both autocorrelation and autoregressive conditional heteroscedasticity (ARCH). The tests are illustrated with a Markov‐switching generalized ARCH (GARCH) model fitted to the US dollar–British pound exchange rate, with the finding that both autocorrelation and GARCH effects are needed to adequately fit the data.     

A Robbins–Monro Algorithm for Non‐Parametric Estimation of NAR Process with Markov Switching: Consistency

We approach the problem of non‐parametric estimation for autoregressive Markov switching processes. In this context, the Nadaraya–Watson‐type regression functions estimator is interpreted as a solution of a local weighted least‐square problem, which does not admit a closed‐form solution in the case of hidden Markov switching. We introduce a non‐parametric recursive algorithm to approximate the estimator. Our algorithm restores the missing data by means of a Monte Carlo step and estimates the regression function via a Robbins–Monro step. We prove that non‐parametric autoregressive models with Markov switching are identifiable when the hidden Markov process has a finite state space. Consistency of the estimator is proved using the strong α‐mixing property of the model. Finally, we present some simulations illustrating the performances of our non‐parametric estimation procedure.     

Scalable inference for space‐time Gaussian Cox processes

The log‐Gaussian Cox process is a flexible and popular stochastic process for modeling point patterns exhibiting spatial and space‐time dependence. Model fitting requires approximation of stochastic integrals which is implemented through discretization over the domain of interest. With fine scale discretization, inference based on Markov chain Monte Carlo is computationally burdensome because of the cost of matrix decompositions and storage, such as the Cholesky, for high dimensional covariance matrices associated with latent Gaussian variables. This article addresses these computational bottlenecks by combining two recent developments: (i) a data augmentation strategy that has been proposed for space‐time Gaussian Cox processes that is based on exact Bayesian inference and does not require fine grid approximations for infinite dimensional integrals, and (ii) a recently developed family of sparsity‐inducing Gaussian processes, called nearest‐neighbor Gaussian processes, to avoid expensive matrix computations. Our inference is delivered within the fully model‐based Bayesian paradigm and does not sacrifice the richness of traditional log‐Gaussian Cox processes. We apply our method to crime event data in San Francisco and investigate the recovery of the intensity surface.     

An Extended Portmanteau Test for VARMA Models With Mixing Nonlinear Constraints

Abstract. The portmanteau test is a widely used diagnostic tool for univariate and multivariate time‐series models. Its asymptotic distribution is known for the unconstrained vector autoregressive moving‐average (VARMA) case and for VAR models with constraints on the autoregressive coefficients. In this article, we give conditions under which the test can be applied to constrained VARMA models. Unfortunately, it cannot generally be applied to models with constraints that simultaneously affect the ARMA polynomial coefficients and the covariance matrix of the innovations (mixing constraints). This happens in latent‐variable models such as dynamic factor models (DFM). In addition, when there are constraints on the covariance matrix it seems convenient to check the goodness of fit using the zero‐lag residual covariances. We propose an extended portmanteau test that not only checks the autocorrelations of the residuals but also whether their covariance matrix is consistent with the constraints. We prove that the statistic is asymptotically distributed as a chi‐square for ARMA models under the assumption that the innovations have Gaussian‐like fourth‐order moments. We also show that the test is appropriate for the DFM, Peña–Box model and factor‐structural vector autoregression (FSVAR).     

Fault Detection and Diagnosis in a Food Pasteurization Process with Hidden Markov Models

Hidden Markov Models (HMM) are used to detect abnormal operation of dynamic processes and diagnose sensor and actuator faults. The method is illustrated by monitoring the operation of a pasteurization plant and diagnosing causes of abnormal operation. Process data collected under the influence of faults of different magnitude and duration in sensors and actuators are used to illustrate the use of HMM in the detection and diagnosis of process faults. Case studies with experimental data from a high‐temperature‐short‐time pasteurization system showed that HMM can diagnose the faults with certain characteristics such as fault duration and magnitude.     

A novel approach to scheduling multipurpose batch plants using unit‐slots

Several models for scheduling multipurpose batch plants exist in the literature. The models using unit‐specific event points have shown better solution efficiency on various literature examples. This article presents a novel approach to scheduling multipurpose batch plants, which uses unit‐slots instead of process‐slots to manage shared resources such as material storage. We develop two slightly different models that are even more compact and simpler than that of Sundaramoorthy and Karimi, Chem Eng Sci. 2005;60:2679–2702. Although we focus on material as a shared resource, our multi‐grid approach rationalizes, generalizes, and improves the current multi‐grid approaches for scheduling with shared resources. Our models allow nonsimultaneous transfers of materials into and out of a batch. We show by an example that this flexibility can give better schedules than those from existing models in some cases. Furthermore, our approach uses fewer slots (event‐points) on some examples than even those required by the most recent unit‐specific event‐based model. Numerical evaluation using literature examples shows significant gains in solution efficiency from the use of unit‐slots except where the number of unit‐slots required for the optimal solution equals that of process slots. We also highlight the importance of constraint sequencing in GAMS implementation for evaluating mixed‐integer linear programming based scheduling models fairly. © 2009 American Institute of Chemical Engineers AIChE J, 2010     

MCMC for Integer‐Valued ARMA processes

Abstract. The classical statistical inference for integer‐valued time‐series has primarily been restricted to the integer‐valued autoregressive (INAR) process. Markov chain Monte Carlo (MCMC) methods have been shown to be a useful tool in many branches of statistics and is particularly well suited to integer‐valued time‐series where statistical inference is greatly assisted by data augmentation. Thus in this article, we outline an efficient MCMC algorithm for a wide class of integer‐valued autoregressive moving‐average (INARMA) processes. Furthermore, we consider noise corrupted integer‐valued processes and also models with change points. Finally, in order to assess the MCMC algorithms inferential and predictive capabilities we use a range of simulated and real data sets.     

Improved Prediction Limits For AR(p) and ARCH(p) Processes

Abstract. A new simulation‐based prediction limit that improves on any given estimative d‐step‐ahead prediction limit for a Markov process is described. This improved prediction limit can be found with almost no algebraic manipulations. Nonetheless, it has the same asymptotic coverage properties as the Barndorff‐Nielsen and Cox [Inference and Asymptotics (1994) Chapman and Hall, London] and Vidoni [Journal of Time Series Analysis Vol. 25, pp. 137–154.] (2004) improved prediction limits. The new simulation‐based prediction limit is ideally suited to those Markov process models for which the algebraic manipulations required for the latter improved prediction limits are very complicated. We illustrate the new method by applying it in the context of one‐step‐ahead prediction for a zero‐mean Gaussian AR(2) process and an ARCH(2) process.     

Joint Determination of the State Dimension and Autoregressive Order for Models with Markov Regime Switching

Abstract. This paper is concerned with the problem of joint determination of the state dimension and autoregressive order of models with Markov‐switching parameters. A model selection procedure is proposed which is based on optimization of complexity‐penalized likelihood criteria. The efficacy of the procedure is evaluated by means of Monte Carlo experiments.     

Integrating data‐based modeling and nonlinear control tools for batch process control

A data‐based multimodel approach is developed in this work for modeling batch systems in which multiple local linear models are identified using latent variable regression and combined using an appropriate weighting function that arises from fuzzy c‐means clustering. The resulting model is used to generate empirical reverse‐time reachability regions (RTRRs) (defined as the set of states from where the data‐based model can be driven inside a desired end‐point neighborhood of the system), which are subsequently incorporated in a predictive control design. Simulation results of a fed‐batch reactor system under proportional‐integral (PI) control and the proposed RTRR‐based design demonstrate the superior performance of the RTRR‐based design in both a fault‐free and faulty environment. The data‐based modeling methodology is then applied on a nylon‐6,6 batch polymerization process to design a trajectory tracking predictive controller. Closed‐loop simulation results illustrate the superior tracking performance of the proposed predictive controller over PI control. © 2011 American Institute of Chemical Engineers AIChE J, 2012     

A Bayesian nonlinearity test for threshold moving average models

We propose a Bayesian test for nonlinearity of threshold moving average (TMA) models. First, we obtain the marginal posterior densities of all parameters, including the threshold and delay, of the TMA model using Gibbs sampler with the Metropolis–Hastings algorithm. And then, we adopt reversible‐jump Markov chain Monte Carlo methods to calculate the posterior probabilities for MA and TMA models. Posterior evidence in favour of the TMA model indicates threshold nonlinearity. Simulation experiments and a real example show that our method works very well in distinguishing MA and TMA models.     

A PARAMETER‐DRIVEN LOGIT REGRESSION MODEL FOR BINARY TIME SERIES

We consider a parameter‐driven regression model for binary time series, where serial dependence is introduced by an autocorrelated latent process incorporated into the logit link function. Unlike in the case of parameter‐driven Poisson log‐linear or negative binomial logit regression model studied in the literature for time series of counts, generalized linear model (GLM) estimation of the regression coefficient vector, which suppresses the latent process and maximizes the corresponding pseudo‐likelihood, cannot produce a consistent estimator. As a remedial measure, in this article, we propose a modified GLM estimation procedure and show that the resulting estimator is consistent and asymptotically normal. Moreover, we develop two procedures for estimating the asymptotic covariance matrix of the estimator and establish their consistency property. Simulation studies are conducted to evaluate the finite‐sample performance of the proposed procedures. An empirical example is also presented.     

DETERMINING THE NUMBER OF REGIMES IN MARKOV SWITCHING VAR AND VMA MODELS

We give stable finite‐order vector autoregressive moving average (p ^* ,q ^* ) representations for M‐state Markov switching second‐order stationary time series whose autocovariances satisfy a certain matrix relation. The upper bounds for p ^* and q ^* are elementary functions of the dimension K of the process, the number M of regimes, the autoregressive and moving‐average orders of the initial model. If there is no cancellation, the bounds become equalities, and this solves the identification problem. Our classes of time series include every M‐state Markov switching multi‐variate moving‐average models and autoregressive models in which the regime variable is uncorrelated with the observable. Our results include, as particular cases, those obtained by Krolzig (1997) and improve the bounds given by Zhang and Stine (2001) and Francq and Zakoïan (2001) for our classes of dynamic models. A Monte Carlo experiment and an application on foreign exchange rates complete the article. Copyright © 2013 John Wiley & Sons, Ltd.