Consistent Estimation of Linear and Non-linear Autoregressive Models with Markov Regime

An autoregressive process with Markov regime is an autoregressive process for which the regression function at each time-point is given by a (non-observable) Markov chain. We examine maximum likelihood estimation for such models and show consistency of a conditional maximum likelihood estimator. Also identifiability issues are discussed     

Time‐Varying Transition Probabilities for Markov Regime Switching Models

We propose a new Markov switching model with time‐varying transitions probabilities. The novelty of our model is that the transition probabilities evolve over time by means of an observation driven model. The innovation of the time‐varying probability is generated by the score of the predictive likelihood function. We show how the model dynamics can be readily interpreted. We investigate the performance of the model in a Monte Carlo study and show that the model is successful in estimating a range of different dynamic patterns for unobserved regime switching probabilities. We also illustrate the new methodology in an empirical setting by studying the dynamic mean and variance behaviour of US industrial production growth.     

Fault Detection and Diagnosis in a Food Pasteurization Process with Hidden Markov Models

Hidden Markov Models (HMM) are used to detect abnormal operation of dynamic processes and diagnose sensor and actuator faults. The method is illustrated by monitoring the operation of a pasteurization plant and diagnosing causes of abnormal operation. Process data collected under the influence of faults of different magnitude and duration in sensors and actuators are used to illustrate the use of HMM in the detection and diagnosis of process faults. Case studies with experimental data from a high‐temperature‐short‐time pasteurization system showed that HMM can diagnose the faults with certain characteristics such as fault duration and magnitude.     

Broadband Semiparametric Estimation of the Memory Parameter of a Long-Memory Time Series Using Fractional Exponential Models

We consider a fractional exponential, or FEXP estimator of the memory parameter of a stationary Gaussian long-memory time series. The estimator is constructed by fitting a FEXP model of slowly increasing dimension to the log periodogram at all Fourier frequencies by ordinary least squares, and retaining the corresponding estimated memory parameter. We do not assume that the data were necessarily generated by a FEXP model, or by any other finite-parameter model. We do, however, impose a global differentiability assumption on the spectral density except at the origin. Because of this, and its use of all Fourier frequencies, we refer to the FEXP estimator as a broadband semiparametric estimator. We demonstrate the consistency of the FEXP estimator, and obtain expressions for its asymptotic bias and variance. If the true spectral density is sufficiently smooth, the FEXP estimator can strongly outperform existing semiparametric estimators, such as the Geweke–Porter-Hudak (GPH) and Gaussian semiparametric estimators (GSE), attaining an asymptotic mean squared error proportional to (log n )/ n , where n is the sample size. In a simulation study, we demonstrate the merits of using a finite-sample correction to the asymptotic variance, and we also explore the possibility of automatically selecting the dimension of the exponential model using Mallows' C_L criterion.     

Using the Penalized Likelihood Method for Model Selection with Nuisance Parameters Present only under the Alternative: An Application to Switching Regression Models

Abstract. We study the problem of model selection with nuisance parameters present only under the alternative. The common approach for testing in this case is to determine the true model through the use of some functionals over the nuisance parameters space. Since in such cases the distribution of these statistics is not known, critical values had to be approximated usually through computationally intensive simulations. Furthermore, the computed critical values are data and model dependent and hence cannot be tabulated. We address this problem by using the penalized likelihood method to choose the correct model. We start by viewing the likelihood ratio as a function of the unidentified parameters. By using the empirical process theory and the uniform law of the iterated logarithm (LIL) together with sufficient conditions on the penalty term, we derive the consistency properties of this method. Our approach generates a simple and consistent procedure for model selection. This methodology is presented in the context of switching regression models. We also provide some Monte Carlo simulations to analyze the finite sample performance of our procedure.     

Analytic Convergence Rates and Parameterization Issues for the Gibbs Sampler Applied to State Space Models

In this paper we obtain a closed form expression for the convergence rate of the Gibbs sampler applied to the unobserved states of a first-order autoregression plus noise model. The rate is expressed in terms of the parameters of the model, which are regarded as fixed. For the case where the unconditional mean of the states is a parameter of interest we provide evidence that a 'centred' parameterization of a state space model is preferable for the performance of the Gibbs sampler. These two results provide guidance when the Gaussianity or linearity of the state space form is lost. We illustrate this by examining the performance of a Markov chain Monte Carlo sampler for the stochastic volatility model.     

Likelihood‐based Analysis of a Class of Generalized Long‐Memory Time Series Models

Abstract. This article introduces a family of ‘generalized long‐memory time series models’, in which observations have a specified conditional distribution, given a latent Gaussian fractionally integrated autoregressive moving‐average (ARFIMA) process. The observations may have discrete or continuous distributions (or a mixture of both). The family includes existing models such as ARFIMA models themselves, long‐memory stochastic volatility models, long‐memory censored Gaussian models and others. Although the family of models is flexible, the latent long‐memory process poses problems for analysis. Therefore, we introduce a Markov chain Monte Carlo sampling algorithm and develop a set of recursions which makes it feasible. This makes it possible, among other things, to carry out exact likelihood‐based analysis of a wide range of non‐Gaussian long‐memory models without resorting to the use of likelihood approximations. The procedure also yields predictive distributions that take into account model parameter uncertainty. The approach is demonstrated in two case studies.     

Estimation of the Dominating Frequency for Stationary and Nonstationary Fractional Autoregressive Models

This paper was motivated by the investigation of certain physiological series for premature infants. The question was whether the series exhibit periodic fluctuations with a certain dominating period. The observed series are nonstationary and/or have long-range dependence. The assumed model is a Gaussian process X _t whose m th difference Y_t = (1 − B ) ^m X_t is stationary with a spectral density f that may have a pole (or a zero) at the origin. the problem addressed in this paper is the estimation of the frequency ω_max where f achieves the largest local maximum in the open interval (0, π). The process X_t is assumed to belong to a class of parametric models, characterized by a parameter vector θ, defined in Beran (1995). An estimator of ω_max is proposed and its asymptotic distribution is derived, with θ being estimated by maximum likelihood. In particular, m and a fractional differencing parameter that models long memory are estimated from the data. Model choice is also incorporated. Thus, within the proposed framework, a data driven procedure is obtained that can be applied in situations where the primary interest is in estimating a dominating frequency. A simulation study illustrates the finite sample properties of the method. In particular, for short series, estimation of ω_max is difficult, if the local maximum occurs close to the origin. The results are illustrated by two of the data examples that motivated this research.     

Multiway continuous hidden Markov model‐based approach for fault detection and diagnosis

A fault detection and classification scheme that uses probabilistic inference based on multiway continuous hidden Markov models (MCHMM) which is capable of capturing complex system dynamics and uncertainty is proposed. A set of observations from normal and faulty runs of the system was collected and used to generate the training dataset. The training data is assumed to follow a finite Gaussian mixture model. The number of mixture components and associated parameters for the optimal Gaussian mixture fit of the observed data was computed subsequently by clustering using the Figueiredo–Jain algorithm for unsupervised learning. The segmental k‐means algorithm was used to compute the HMM parameters. The applicability of the proposed scheme is investigated for the case of an inverted pendulum system and a fluidized catalytic cracker. The monitoring results for the above cases with the proposed scheme was found to be superior to the multiway discrete hidden Markov model (MDHMM) based scheme in terms of the accuracy of fault detection, especially in case of noisy observations. © 2014 American Institute of Chemical Engineers AIChE J, 60: 2035–2047, 2014     

A Robbins–Monro Algorithm for Non‐Parametric Estimation of NAR Process with Markov Switching: Consistency

We approach the problem of non‐parametric estimation for autoregressive Markov switching processes. In this context, the Nadaraya–Watson‐type regression functions estimator is interpreted as a solution of a local weighted least‐square problem, which does not admit a closed‐form solution in the case of hidden Markov switching. We introduce a non‐parametric recursive algorithm to approximate the estimator. Our algorithm restores the missing data by means of a Monte Carlo step and estimates the regression function via a Robbins–Monro step. We prove that non‐parametric autoregressive models with Markov switching are identifiable when the hidden Markov process has a finite state space. Consistency of the estimator is proved using the strong α‐mixing property of the model. Finally, we present some simulations illustrating the performances of our non‐parametric estimation procedure.     

Finite Sample Theory of QMLE in ARCH Models with Dynamics in the Mean Equation

Abstract. We provide simulation and theoretical results concerning the finite‐sample theory of quasi‐maximum‐likelihood estimators in autoregressive conditional heteroskedastic (ARCH) models when we include dynamics in the mean equation. In the setting of the AR(q)–ARCH(p), we find that in some cases bias correction is necessary even for sample sizes of 100, especially when the ARCH order increases. We warn about the existence of important biases and potentially low power of the t‐tests in these cases. We also propose ways to deal with them. We also find simulation evidence that when conditional heteroskedasticity increases, the mean‐squared error of the maximum‐likelihood estimator of the AR(1) parameter in the mean equation of an AR(1)‐ARCH(1) model is reduced. Finally, we generalize the Lumsdaine [J. Bus. Econ. Stat. 13 (1995) pp. 1–10] invariance properties for the biases in these situations.     

Modeling of biocatalytic reactions: A workflow for model calibration,selection, and validation using Bayesian statistics

We present a workflow for kinetic modeling of biocatalytic reactions which combines methods from Bayesian learning and uncertainty quantification for model calibration, model selection, evaluation, and model reduction in a consistent statistical framework. Our workflow is particularly tailored to sparse data settings in which a considerable variability of the parameters remains after the models have been adapted to available data, a ubiquitous problem in many real-world applications. Our workflow is exemplified on an enzyme-catalyzed two-substrate reaction mechanism describing the symmetric carboligation of 3,5-dimethoxy-benzaldehyde to (R)-3,3′,5,5′-tetramethoxybenzoin catalyzed by benzaldehyde lyase from Pseudomonas fluorescens. Results indicate a substrate-dependent inactivation of enzyme, which is in accordance with other recent studies.     

Model Development and Parameter Estimation for the Oxidation of Pyrrhotite‐Containing Rock Surfaces

A novel experimental method was developed for the evaluation of the biotic oxidation rate of waste rock containing sulphide minerals. Sized waste rock samples were inoculated with an Acidithiobacillus ferrooxidans suspension and sparged with discontinuing humidified air to yield the oxygen consumption rate. A kinetic/mass transport model was proposed based on oxidation of pyrrhotite. Bayesian statistical analysis using the Metropolis‐Hastings algorithm showed that at ambient oxygen concentration the mean (i.e., estimated) values of the initial oxygen flux, the reaction order, the oxygen diffusion coefficient through the oxidation product layer, and the surface reaction rate constant were 1.3×10^?7 mol/m².s, 0.55, 2.91×10^?13 m²/s and 3.23×10^?7 mol^0.45/m^0.35.s, respectively.     

A graph theory assisted Monte Carlo algorithm for simulating the topology structure of LDPE and connection with molecular dynamics simulation

Precise simulation of the topology structure of low-density polyethylene (LDPE) chain is significant for product properties research. However, the chain structure symbolization methods in literature all rely on specially designed methods or temporary tricks, which are hard to understand and may induce some deviations in branch distribution prediction. To overcome this problem, a graph theory assisted Monte Carlo algorithm is developed for chain topology structure simulation of LDPE. The new symbolization method provides better accuracy and is much easy for code implementation and integration in other software. In detail, the average long chain branching predicted by this method is always smaller than that predicted by traditional methods, and much closer to the experimental results. Furthermore, the predicted topology structure information of LDPE chains can be provided as input data for molecular dynamics simulation to study the crystal process, and the predicted crystallinity and density also show good agreement with experiments.     

Some Results Concerning the Asymptotic Distribution of Sample Fourier Transforms and Periodograms for a Discrete-time Stationary Process with a Continuous Spectrum

We consider a stationary process ( X_t , t = 0, ±1, ...) with a continuous spectrum. Denote by D_n (λ) a tapered Fourier transform of ( X ₀, X ₁, ..., X _{n −1}) at (angular) frequency λ. We obtain the asymptotic distribution of D_n (λ) and the joint asymptotic distribution of { D_n (λ _j ), 1 ≤ j ≤ k } with continuity of the spectral density f (.) at the relevant frequencies as the only assumption concerning the second-order structure of ( X_t ); all other assumptions required are easily stated. The results are extended to processes for which f (.) is continuous except at λ = 0, with lim_λ←0 f (λ)λ^{2 d} = K , a constant, where 0 < d < ½, as is typical of certain types of processes with long-range dependence. Results for the sample periodogram, proportional to | D_n (λ)|², follow immediately.     

Practical benefits for the separation process of drying that can be realized from novel multiscale modeling procedures that utilize the scientific information and results obtained from molecular dynamics modeling and simulation studies

Novel multiscale modeling procedures are constructed and presented that use the scientific information and results determined from microscopic molecular dynamics (MD) modeling and simulation studies to calculate local effective values for the parameters that characterize the heat and mass transfer mechanisms of dynamic macroscopic continuum models (Euler physics of continua) that are used in practice to describe and predict the dynamic behavior of large scale in time and space (e.g., industrial scale), separation (e.g., drying; adsorption), and chemical and biochemical reaction engineering (e.g., chemical catalysis; biocatalysis; immobilized cell bioreactor systems) processes involving porous media whose pore structure is formed either by a solid rigid matter or by a solid soft matter. Furthermore, the results determined from MD modeling and simulation studies with regard to the energies of interaction between the molecules of the different species of the porous media during the time evolution (time varying) of the drying process can be used to design a time optimally controlled heat input system that could appropriately and accurately supply at any time during drying the amount of heat necessary to provide a desired drying rate with respect to both free and bound water and to satisfy the constraints that safeguard the quality properties of the product.