FILTERING AND SMOOTHING IN STATE SPACE MODELS WITH PARTIALLY DIFFUSE INITIAL CONDITIONS

Abstract. Ansley and Kohn ( Annals of Statistics , 1985) generalized the Kalman filter to handle state space models with partially diffuse initial conditions and used this filter to compute the marginal likelihood of the observations efficiently. In this paper we simplify the algorithm and make it numerically more accurate and operationally more efficient. Based on this filtering algorithm we obtain a corresponding smoothing algorithm for the state vector.     

Maximum Likelihood Estimation of VARMA Models Using a State‐Space EM Algorithm

Abstract. We introduce a state‐space representation for vector autoregressive moving‐average models that enables maximum likelihood estimation using the EM algorithm. We obtain closed‐form expressions for both the E‐ and M‐steps; the former requires the Kalman filter and a fixed‐interval smoother, and the latter requires least squares‐type regression. We show via simulations that our algorithm converges reliably to the maximum, whereas gradient‐based methods often fail because of the highly nonlinear nature of the likelihood function. Moreover, our algorithm converges in a smaller number of function evaluations than commonly used direct‐search routines. Overall, our approach achieves its largest performance gains when applied to models of high dimension. We illustrate our technique by estimating a high‐dimensional vector moving‐average model for an efficiency test of California's wholesale electricity market.     

NOTE ON THE KALMAN FILTER WITH ESTIMATED PARAMETERS

Abstract. State estimation and prediction problems are considered for a stochastic process represented by a state space form which involves unknown parameters. We first study the stability of the Kalman filter corresponding to the state space form without assuming the stationarity of the process. Second, we consider the state estimation and prediction when the process is stationary, and show some asymptotic properties of the state estimates and predicted values obtained by the Kalman filter with estimated parameters which converge to the true parameters or to the equivalent classes of the true parameters with probability one.     

INITIALIZING THE KALMAN FILTER FOR NONSTATIONARY TIME SERIES MODELS

Abstract. The problem of initializing the Kalman filter for nonstationary time series models is considered. Ansley and Kohn have developed a 'modified Kalman filter' for use with nonstationary models to produce estimates from what they call a 'transformation approach'. We show that the same results can be obtained with a suitable initialization of the ordinary Kalman filter. Assuming there are d starting values for the nonstationary series, we initialize the Kalman filter using data through time d with the transformation approach estimate of the state vector and its associated error covariance matrix at time d . We give details of the initialization for ARIMA models, ARIMA component models and dynamic linear models. We present an example to illustrate how the results may differ from results obtained under more naive initializations that have been suggested.     

Parameter and state estimation in nonlinear stochastic continuous‐time dynamic models with unknown disturbance intensity

Approximate Maximum Likelihood Estimation (AMLE) is an algorithm for estimating the states and parameters of models described by stochastic differential equations (SDEs). In previous work (Varziri et al., Ind. Eng. Chem. Res., 47 (2), 380‐393, (2008); Varziri et al., Comp. Chem. Eng., in press), AMLE was developed for SDE systems in which process‐disturbance intensities and measurement‐noise variances were assumed to be known. In the current article, a new formulation of the AMLE objective function is proposed for the case in which measurement‐noise variance is available but the process‐disturbance intensity is not known a priori. The revised formulation provides estimates of the model parameters and disturbance intensities, as demonstrated using a nonlinear CSTR simulation study. Parameter confidence intervals are computed using theoretical linearization‐based expressions. The proposed method compares favourably with a Kalman‐filter‐based maximum likelihood method. The resulting parameter estimates and information about model mismatch will be useful to chemical engineers who use fundamental models for process monitoring and control.     

不敏变换鲁棒Kalman滤波器在发酵过程中的应用(英文)

State estimation is the precondition and foundation of a bioprocess monitoring and optimal control. However, there are many difficulties in dealing with a non-linear system, such as the instability of process, un-modeled dynamics, parameter sensitivity, etc. This paper discusses the principles and characteristics of three different approaches, extended Kalman filters, strong tracking filters and unscented transformation based Kalman filters. By introducing the unscented transformation method and a sub-optimal fading factor to correct the prediction error covariance, an improved Kalman filter, unscented transformation based robust Kalman filter, is proposed. The performance of the algorithm is compared with the strong tracking filter and unscented transformation based Kalman filter and illustrated in a typical case study for glutathione fermentation process. The results show that the proposed algorithm presents better accuracy and stability on the state estimation in numerical calculations.     

STATE SPACE MODELS WITH DIFFUSE INITIAL CONDITIONS

Abstract. A state space model with diffuse initial conditions is considered. A simple and direct proof of the algorithm for computing the likelihood function and minimum mean square estimators of the state is given     

Empirical likelihood in long‐memory time series models

This article studies the empirical likelihood method for long‐memory time series models. By virtue of the Whittle likelihood, one obtains a score function that can be viewed as an estimating equation of the parameters of a fractional integrated autoregressive moving average (ARFIMA) model. This score function is used to obtain an empirical likelihood ratio which is shown to be asymptotically chi‐square distributed. Confidence regions for the parameters are constructed based on the asymptotic distribution of the empirical likelihood ratio. Bartlett correction and finite sample properties of the empirical likelihood confidence regions are examined.     

First‐order integer valued AR processes with zero inflated poisson innovations

The first‐order nonnegative integer valued autoregressive process has been applied to model the counts of events in consecutive points of time. It is known that, if the innovations are assumed to follow a Poisson distribution then the marginal model is also Poisson. This model may however not be suitable for overdispersed count data. One frequent manifestation of overdispersion is that the incidence of zero counts is greater than expected from a Poisson model. In this paper, we introduce a new stationary first‐order integer valued autoregressive process with zero inflated Poisson innovations. We derive some structural properties such as the mean, variance, marginal and joint distribution functions of the process. We consider estimation of the unknown parameters by conditional or approximate full maximum likelihood. We use simulation to study the limiting marginal distribution of the process and the performance of our fitting algorithms. Finally, we demonstrate the usefulness of the proposed model by analyzing some real time series on animal health laboratory submissions.     

A NONSTATIONARY TIME SERIES MODEL AND ITS FITTING BY A RECURSIVE FILTER

Abstract. The use of the state space representation for the analysis of nonstationary time series is proposed. For the fitting of the models, the use of a modified AIC based on the likelihood of the innovation process is proposed. A square root filter/smoother algorithm for the evaluation of the likelihood and state estimation is discussed.     

STRUCTURAL, DYNAMIC MODELLING IN UNOBSERVABLE SPACES OF COVARIANCE-STATIONARY STOCHASTIC PROCESSES

In this paper a structural, stationary version of the well-known state-space model is used to model covariance-stationary stochastic processes. The identifiability of the model parameters is discussed and a rank condition for local parameter identifiability is given. Ljung's results on prediction-error estimation are used to establish strong consistency and asymptotic efficiency of the non-linear ML-estimates obtained from dependent observations. It turns out that the model can be identified by using simultaneously the steady-state Kalman filter for the unobservable state vector and the prediction-error estimation method for the model parameters.     

Nonstationary Cointegration in the Fractionally Cointegrated VAR Model

We consider the fractional cointegrated vector autoregressive (CVAR) model of Johansen and Nielsen (2012a) and make two distinct contributions. First, in their consistency proof, Johansen and Nielsen (2012a) imposed moment conditions on the errors that depend on the parameter space, such that when the parameter space is larger, stronger moment conditions are required. We show that these moment conditions can be relaxed, and for consistency we require just eight moments regardless of the parameter space. Second, Johansen and Nielsen (2012a) assumed that the cointegrating vectors are stationary, and we extend the analysis to include the possibility that the cointegrating vectors are non‐stationary. Both contributions require new analysis and results for the asymptotic properties of the likelihood function of the fractional CVAR model, which we provide. Finally, our analysis follows recent research and applies a parameter space large enough that the usual (non‐fractional) CVAR model constitutes an interior point and hence can be tested against the fractional model using a Chi‐squared‐test.     

Efficient estimation and particle filter for max‐stable processes

Extreme values are often correlated over time, for example, in a financial time series, and these values carry various risks. Max‐stable processes such as maxima of moving maxima (M3) processes have been recently considered in the literature to describe time‐dependent dynamics, which have been difficult to estimate. This article first proposes a feasible and efficient Bayesian estimation method for nonlinear and non‐Gaussian state space models based on these processes and describes a Markov chain Monte Carlo algorithm where the sampling efficiency is improved by the normal mixture sampler. Furthermore, a unique particle filter that adapts to extreme observations is proposed and shown to be highly accurate in comparison with other well‐known filters. Our proposed algorithms were applied to daily minima of high‐frequency stock return data, and a model comparison was conducted using marginal likelihoods to investigate the time‐dependent dynamics in extreme stock returns for financial risk management.     

ESTIMATION OF MULTIVARIATE TIME SERIES

Abstract. The algorithm proposed here is a multivariate generalization of a procedure discussed by Pearlman (1980) for calculating the exact likelihood of a univariate ARMA model. Ansley and Kohn (1983) have shown how the Kalman filter can be used to calculate the exact likelihood function when not all the observations are known. In Shea (1983) it is shown that this algorithm is much quicker than that of Ansley and Kohn (1983) for all ARMA models except an ARMA (2, 1) and a couple of low-order AR processes and therefore when we have no missing observations this algorithm should be used instead. The Fortran subroutine G13DCF in the NAG (1987) Library fits a vector ARMA model using an adaptation of this algorithm. Experience in the use of this routine suggests that having reasonably good initial estimates of the ARMA parameter matrices, and in particular the residual error covariance matrix, can not only substantially reduce the computing time but more important improve the convergence properties of the minimization procedure. We therefore propose a method of calculating initial estimates of the ARMA parameters which involves using a generalization of the concept of inverse cross covariances from the univariate to the multivariate case. Finally theory is put into practice with the fitting of a bivariate model to a couple of real-life time series.     

利用状态空间模型和不敏卡尔曼滤波的分批补料发酵过程在线估计(英文)

On-line estimation of unmeasurable biological variables is important in fermentation processes, directly influencing the optimal control performance of the fermentation system as well as the quality and yield of the targeted product. In this study, a novel strategy for state estimation of fed-batch fermentation process is proposed. By combining a simple and reliable mechanistic dynamic model with the sample-based regressive measurement model, a state space model is developed. An improved algorithm, swarm energy conservation particle swarm optimization (SECPSO), is presented for the parameter identification in the mechanistic model, and the support vector machines (SVM) method is adopted to establish the nonlinear measurement model. The unscented Kalman filter (UKF) is designed for the state space model to reduce the disturbances of the noises in the fermentation process. The proposed on-line estimation method is demonstrated by the simulation experiments of a penicillin fed-batch fermentation process.     

An algorithm for the exact likelihood of a stationary vector autoregressive-moving average model

The so-called innovations form of the likelihood function implied by a stationary vector autoregressive-moving average model is considered without directly using a state–space representation. Specifically, it is shown in detail how to compute the exact likelihood by an adaptation to the multivariate case of the innovations algorithm of Ansley (1979 ) for univariate models. Comparisons with other existing methods are also provided, showing that the algorithm described here is computationally more efficient than the fastest methods currently available in many cases of practical interest.     

A maximum-likelihood method for estimating parameters,stochastic disturbance intensities and measurement noise variances in nonlinear dynamic models with process disturbances

An improved approximate maximum likelihood algorithm is developed for estimating measurement noise variances along with model parameters and disturbance intensities in nonlinear stochastic differential equation (SDE) models. This algorithm uses a Laplace approximation and B-spline basis functions for approximating the likelihood function of the parameters given the measurements. The resulting Laplace approximation maximum likelihood estimation (LAMLE) algorithm is tested using a nonlinear continuous stirred tank reactor (CSTR) model. Estimation results for four model parameters, two process disturbance intensities and two measurement noise variances are obtained using LAMLE and are compared with results from two other maximum-likelihood-based methods, the continuous-time stochastic method (CTSM) of Kristensen and Madsen (2003) and the Fully Laplace Approximation Estimation Method (FLAEM) (Karimi and McAuley, 2014). Parameter estimations using 100 simulated data sets reveal that the LAMLE estimation results tend to be more precise and less biased than corresponding estimates obtained using CTSM and FLAEM.     

A direct sampling particle filter from approximate conditional density function supported on constrained state space

Constraints on the state vector must be taken into account in the state estimation problem. Recently, acceptance/rejection and projection methods are proposed in the particle filter framework for constraining the particles. A weighted least squares formulation is used for constraining samples in unscented and ensemble Kalman filters. In this paper, direct sampling from an approximate conditional probability density function (pdf) is proposed. It is obtained by approximating the a priori pdf as a Gaussian. The support of the conditional density is a subset of the intersection of two supports, the 3-sigma bounds of the priori Gaussian and the constrained state space. A direct sampling algorithm is proposed for handling linear and nonlinear equality and inequality constraints. The algorithm uses the constrained mode for nonlinear constraints.     

A FREQUENCY DOMAIN APPROACH FOR THE ESTIMATION OF PARAMETERS OF SPATIO‐TEMPORAL STATIONARY RANDOM PROCESSES

A frequency domain methodology is proposed for estimating parameters of covariance functions of stationary spatio‐temporal processes. Finite Fourier transforms of the processes are defined at each location. Based on the joint distribution of these complex valued random variables, an approximate likelihood function is constructed. The sampling properties of the estimators are investigated. It is observed that the expectation of these transforms can be considered to be a frequency domain analogue of the classical variogram. We call this measure frequency variogram. The method is applied to simulated data and also to Pacific wind speed data considered earlier by Cressie and Huang (1999). The proposed method does not depend on the distributional assumptions about the process.