On Bayesian analysis of nonlinear continuous‐time autoregression models

Abstract. This article introduces a method for performing fully Bayesian inference for nonlinear conditional autoregressive continuous‐time models, based on a finite skeleton of observations. Our approach uses Markov chain Monte Carlo and involves imputing data from times at which observations are not made. It uses a reparameterization technique for the missing data, and because of the non‐Markovian nature of the models, it is necessary to adopt an overlapping blocks scheme for sequentially updating segments of missing data. We illustrate the methodology using both simulated data and a data set from the S & P 500 index.     

Parameter estimation in models with hidden variables : An application to a biotech process

Biological processes are often characterised by significant nonlinearities, noisy measurements and hidden process variables. The dynamic behaviour of such processes can be represented by stochastic differential equations obtained from physical laws. We propose a Bayesian algorithm for parameter estimation in stochastic nonlinear biological processes with unmeasured (or hidden) variables. The proposed algorithm, involves drawing random samples iteratively from a posterior density functions of the parameters and the hidden variables. A Bayesian sampling techniques is used to approximate these posterior density functions. Both Metropolis–Hastings algorithm and Gibbs sampling are used for sample generation. The algorithm is extended to handle multiple data sets and missing observations. The algorithm is applied to an experimental data set collected from an algal bioreactor system. © 2011 Canadian Society for Chemical Engineering     

Selecting optimal weighting factors in iPDA for parameter estimation in continuous-time dynamic models

Iteratively refined principal differential analysis (iPDA) is a spline-based method for estimating parameters in ordinary differential equation (ODE) models. In this article we extend iPDA for use in differential equation models with stochastic disturbances and we demonstrate the probabilistic basis for the iPDA objective function using a maximum likelihood argument. This development naturally leads to a method for selecting the optimal weighting factor in the iPDA objective function. We demonstrate the effectiveness of iPDA using a simple two-output continuous-stirred-tank-reactor example, and we use Monte Carlo simulations to show that iPDA parameter estimates are superior to those obtained using traditional nonlinear least squares techniques, which do not account for stochastic disturbances.     

Broadband semi‐parametric estimation of long‐memory time series by fractional exponential models

This article proposes broadband semi‐parametric estimation of a long‐memory parameter by fractional exponential (FEXP) models. We construct the truncated Whittle likelihood based on FEXP models in a semi‐parametric setting to estimate the parameter and show that the proposed estimator is more efficient than the FEXP estimator by Moulines and Soulier (1999) in linear processes. A Monte Carlo simulation suggests that the proposed estimation is more preferable than the existing broadband semi‐parametric estimation.     

Convolution‐closed models for count time series with applications

There has recently been an upsurge of interest in time series models for count data. Many papers focus on the model with first‐order (Markov) dependence and Poisson innovations. Our paper considers practical models that can capture higher‐order dependence based on the work of Joe (1996). In this framework we are able to model both equidispersed and overdispersed marginal distributions of data. The latter is approached using generalized Poisson innovations. Central to the models is the use of the property of closure under convolution of certain families of random variables. The models can be thought of as stationary Markov chains of finite order. Parameter estimation is undertaken by maximum likelihood, inference procedures are considered and means of assessing model adequacy employed. Applications to two new data sets are provided.     

A maximum-likelihood method for estimating parameters,stochastic disturbance intensities and measurement noise variances in nonlinear dynamic models with process disturbances

An improved approximate maximum likelihood algorithm is developed for estimating measurement noise variances along with model parameters and disturbance intensities in nonlinear stochastic differential equation (SDE) models. This algorithm uses a Laplace approximation and B-spline basis functions for approximating the likelihood function of the parameters given the measurements. The resulting Laplace approximation maximum likelihood estimation (LAMLE) algorithm is tested using a nonlinear continuous stirred tank reactor (CSTR) model. Estimation results for four model parameters, two process disturbance intensities and two measurement noise variances are obtained using LAMLE and are compared with results from two other maximum-likelihood-based methods, the continuous-time stochastic method (CTSM) of Kristensen and Madsen (2003) and the Fully Laplace Approximation Estimation Method (FLAEM) (Karimi and McAuley, 2014). Parameter estimations using 100 simulated data sets reveal that the LAMLE estimation results tend to be more precise and less biased than corresponding estimates obtained using CTSM and FLAEM.     

An adaptive parallel tempering method for the dynamic data‐driven parameter estimation of nonlinear models

An adaptive parallel tempering algorithm is developed in a user‐friendly fashion that efficiently and robustly generates near‐optimum solutions. Using adaptive, implicit, time‐integration methods, the method allows fitting model parameters to dynamic data. The proposed approach is relatively insensitive to the initial guess and requires minimal fine‐tuning: most of the algorithm parameters can be determined adaptively based on the analysis of few model simulations, while default values are proposed for the few remaining ones, the exact values of which do not sensitively affect the solution. The method is extensively validated through its application to a number of algebraic and dynamic global optimization problems from Chemical Engineering literature. We then apply it to a multi‐parameter, highly nonlinear, model of the rheology of a thixotropic system where we show how the present approach can be used to robustly determine model parameters by fitting to dynamic, large amplitude, oscillatory stress vs. shear rate, data. © 2016 American Institute of Chemical Engineers AIChE J, 63: 1937–1958, 2017     

Estimation of regression and dynamic dependence paremeters for non‐stationary multinomial time series

In a time‐series regression setup, multinomial responses along with time dependent observable covariates are usually modelled by certain suitable dynamic multinomial logistic probabilities. Frequently, the time‐dependent covariates are treated as a realization of an exogenous random process and one is interested in the estimation of both the regression and the dynamic dependence parameters conditional on this realization of the covariate process. There exists a partial likelihood estimation approach able to deal with the general dependence structures arising from the influence of both past covariates and past multinomial responses on the covariates at a given time by sequentially conditioning on the history of the joint process (response and covariates), but it provides standard errors for the estimators based on the observed information matrix, because such a matrix happens to be the Fisher information matrix obtained by conditioning on the whole history of the joint process. This limitation of the partial likelihood approach holds even if the covariate history is not influeced by lagged response outcomes. In this article, a general formulation of the auto‐covariance structure of a multinomial time series is presented and used to derive an explicit expression for the Fisher information matrix conditional on the covariate history, providing the possibility of computing the variance of the maximum likelihood estimators given a realization of the covariate process for the multinomial‐logistic model. The difference between the standard errors of the parameter estimators under these two conditioning schemes (covariates Vs. joint history) is illustrated through an intensive simulation study based on the premise of an exogenous covariate process.     

High‐frequency sampling and kernel estimation for continuous‐time moving average processes

Interest in continuous‐time processes has increased rapidly in recent years, largely because of high‐frequency data available in many applications. We develop a method for estimating the kernel function g of a second‐order stationary Lévy‐driven continuous‐time moving average (CMA) process Y based on observations of the discrete‐time process Y^Δ obtained by sampling Y at Δ, 2Δ, …, nΔ for small Δ. We approximate g by g^Δ based on the Wold representation and prove its pointwise convergence to g as Δ → 0 for continuous‐time autoregressive moving average (CARMA) processes. Two non‐parametric estimators of g^Δ, on the basis of the innovations algorithm and the Durbin–Levinson algorithm, are proposed to estimate g. For a Gaussian CARMA process, we give conditions on the sample size n and the grid spacing Δ(n) under which the innovations estimator is consistent and asymptotically normal as n → ∞. The estimators can be calculated from sampled observations of any CMA process, and simulations suggest that they perform well even outside the class of CARMA processes. We illustrate their performance for simulated data and apply them to the Brookhaven turbulent wind speed data. Finally, we extend results of Brockwell et al. (2012) for sampled CARMA processes to a much wider class of CMA processes.     

Inference for single and multiple change‐points in time series

The article reviews methods of inference for single and multiple change‐points in time series, when data are of retrospective (off‐line) type. The inferential methods reviewed for a single change‐point in time series include likelihood, Bayes, Bayes‐type and some relevant non‐parametric methods. Inference for multiple change‐points requires methods that can handle large data sets and can be implemented efficiently for estimating the number of change‐points as well as their locations. Our review in this important area focuses on some of the recent advances in this direction. Greater emphasis is placed on multivariate data while reviewing inferential methods for a single change‐point in time series. Throughout the article, more attention is paid to estimation of unknown change‐point(s) in time series, and this is especially true in the case of multiple change‐points. Some specific data sets for which change‐point modelling has been carried out in the literature are provided as illustrative examples under both single and multiple change‐point scenarios.     

Testing Parameter Change in General Integer‐Valued Time Series

We consider the structural change in a class of discrete valued time series, which the conditional distribution belongs to the one‐parameter exponential family. We propose a change point test based on the maximum likelihood estimator of the model's parameter. Under the null hypothesis (of no change), the test statistic converges to a well‐known distribution, allowing the calculation of the critical value of the test. The test statistic diverges to infinity under the alternative, meaning that the test has asymptotic power one. Some simulation results and real data applications are reported to show the effectiveness of the proposed procedure.     

Structure and estimation of a class of nonstationary yet nonexplosive GARCH models

This article considers GARCH(1,1) models in which the time‐varying coefficients are functions of the realizations of an exogenous stochastic process. Time series generated by this model are in general nonstationary. Necessary and sufficient conditions are given for the existence of nonexplosive solutions, and for the existence of moments of these solutions. The asymptotic properties of the quasi‐maximum likelihood estimator are derived under mild assumptions.     

A light‐tailed conditionally heteroscedastic model with applications to river flows

Abstract. A conditionally heteroscedastic model, different from the more commonly used autoregressive moving average–generalized autoregressive conditionally heteroscedastic (ARMA‐GARCH) processes, is established and analysed here. The time‐dependent variance of innovations passing through an ARMA filter is conditioned on the lagged values of the generated process, rather than on the lagged innovations, and is defined to be asymptotically proportional to those past values. Designed this way, the model incorporates certain feedback from the modelled process, the innovation is no longer of GARCH type, and all moments of the modelled process are finite provided the same is true for the generating noise. The article gives the condition of stationarity, and proves consistency and asymptotic normality of the Gaussian quasi‐maximum likelihood estimator of the variance parameters, even though the estimated parameters of the linear filter contain an error. An analysis of six diurnal water discharge series observed along Rivers Danube and Tisza in Hungary demonstrates the usefulness of such a model. The effect of lagged river discharge turns out to be highly significant on the variance of innovations, and nonparametric estimation approves its approximate linearity. Simulations from the new model preserve well the probability distribution, the high quantiles, the tail behaviour and the high‐level clustering of the original series, further justifying model choice.     

A full‐condition monitoring method for nonstationary dynamic chemical processes with cointegration and slow feature analysis

Chemical processes are in general subject to time variant conditions because of load changes, product grade transitions, or other causes, resulting in typical nonstationary dynamic characteristic. It is of a considerable challenge for process monitoring to consider all possible operation conditions simultaneously including multifarious steady states and dynamic switchings. In this work, a novel full‐condition monitoring strategy is proposed based on both cointegration analysis (CA) and slow feature analysis (SFA) with the following considerations: (1) Despite that the operation conditions may vary over time, they may follow certain equilibrium relations that extend beyond the current time, and (2) there may exist certain dynamic relations that stay invariant under normal process operation despite process may operate at different operating conditions. To monitor both equilibrium and dynamic relations, in the proposed method, nonstationary variables are separated from stationary variables first. Then by CA and SFA, the long‐term equilibrium relation is distinguished from the specific relation held by the current conditions from both static and dynamic aspects. Various monitoring statistics are designed with clear physical interpretation. It can distinguish between the changes of operation conditions and real faults by checking deviations from equilibrium relation and deviations from the specific relation. Case study on a chemical industrial scale multiphase flow experimental rig shows the validity of the proposed full‐condition monitoring method. © 2017 American Institute of Chemical Engineers AIChE J, 64: 1662–1681, 2018     

Trajectory modeling of gas–liquid flow in microchannels with stochastic differential equation and optical measurement

The numbering‐up of microchannel reactors definitely faces great challenge in uniformly distributing fluid flow in every channel, especially for multiphase systems. A model of stochastic differential equations (SDEs) is proposed based on the experimental data recorded by a long‐term optical measurement to well quantify the stochastic trajectories of gas bubbles and liquid slugs in parallel microchannels interconnected with two dichotomic distributors. The expectation and variance of each subflow rate are derived explicitly from the SDEs associated with the Fokker–Planck equation and solved numerically. A bifurcation in the trajectory is found using the original model, then a modification on interactions of feedback and crosstalk is introduced, the evolutions of subflow rates calculated by the modified model match well with experimental results. The established methodology is helpful for characterizing the flow uniformity and numbering‐up the microchannel reactors of multiphase system. © 2015 American Institute of Chemical Engineers AIChE J, 61: 4028–4034, 2015     

Asymptotic self‐similarity and wavelet estimation for long‐range dependent fractional autoregressive integrated moving average time series with stable innovations

Abstract. Methods for parameter estimation in the presence of long‐range dependence and heavy tails are scarce. Fractional autoregressive integrated moving average (FARIMA) time series for positive values of the fractional differencing exponent d can be used to model long‐range dependence in the case of heavy‐tailed distributions. In this paper, we focus on the estimation of the Hurst parameter H = d + 1/α for long‐range dependent FARIMA time series with symmetric α‐stable (1 < α < 2) innovations. We establish the consistency and the asymptotic normality of two types of wavelet estimators of the parameter H. We do so by exploiting the fact that the integrated series is asymptotically self‐similar with parameter H. When the parameter α is known, we also obtain consistent and asymptotically normal estimators for the fractional differencing exponent d = H ? 1/α. Our results hold for a larger class of causal linear processes with stable symmetric innovations. As the wavelet‐based estimation method used here is semi‐parametric, it allows for a more robust treatment of long‐range dependent data than parametric methods.