FAST LINEAR ESTIMATION METHODS FOR VECTOR AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. Three linear methods for estimating parameter values of vector auto-regressive moving-average (VARMA) models which are in general at least an order of magnitude faster than maximum likelihood estimation are developed in this paper. Simulation results for different model structures with varying numbers of component series and observations suggest that the accuracy of these procedures is in most cases comparable with maximum likelihood estimation. Procedures for estimating parameter standard error are also discussed and used for identification of nonzero elements in the VARMA polynomial structures. These methods can also be used to establish the order of the VARMA structure. We note, however, that the primary purpose of these estimates is to generate initial estimates for the nonzero parameters in order to reduce subsequent computational time of more efficient estimation procedures such as exact maximum likelihood.     

Maximum‐likelihood maximum‐entropy constrained probability density function estimation for prediction of rare events

This work addresses the problem of estimating complete probability density functions (PDFs) from historical process data that are incomplete (lack information on rare events), in the framework of Bayesian networks. In particular, this article presents a method of estimating the probabilities of events for which historical process data have no record. The rare‐event prediction problem becomes more difficult and interesting, when an accurate first‐principles model of the process is not available. To address this problem, a novel method of estimating complete multivariate PDFs is proposed. This method uses the maximum entropy and maximum likelihood principles. It is tested on mathematical and process examples, and the application and satisfactory performance of the method in risk assessment and fault detection are shown. Also, the proposed method is compared with a few copula methods and a nonparametric kernel method, in terms of performance, flexibility, interpretability, and rate of convergence. © 2014 American Institute of Chemical Engineers AIChE J, 60: 1013–1026, 2014     

An integrated numerical and experimental framework for modeling of CTB and GD1b ganglioside binding kinetics

A mathematical model is developed and validated for a multistep binding process between cholera toxin subunit B (CTB) and GD1b receptors that precedes cholera infection. To study the dynamics of the complex CTB‐GD1b binding mechanisms, cooperative binding effect and GD1b receptor aggregation in the host cell membrane are considered. More reliable parameters for the CTB‐GD1b binding kinetics are estimated by quantitatively calibrating the proposed multistep binding model against the experimental measurements obtained from the novel nanocube‐based biosensor. Specifically, a numerical scheme that includes the sensitivity analysis, parameter estimation and dynamic optimization is implemented for the model calibration. Through this scheme, identifiable model parameters are determined. After those selected parameters are estimated, the calibrated model and the experimental measurements were in reasonable agreement for different CTB and GD1b concentrations, which shows a promising approach for identification of the kinetics of CTB binding to the host cell membrane. © 2018 American Institute of Chemical Engineers AIChE J, 64: 3882–3893, 2018     

PARAMETER ESTIMATION IN EXPONENTIAL MODELS

Abstract. In this paper the problems of parameter estimation and order determination of an exponential (EX) model are studied in the time domain. In order to estimate the parameters, the parameter equations of an EX model are given in terms of the autocorrelation function, which is similar to the Yule-Walker equations of an autoregressive moving-average model. Estimates of parameters are obtained with the aid of the parameter equations and theorems are proved relating the convergence rate and asymptotic distribution of the estimates. We present two kinds of methods for estimating the order and prove that the estimates of the order are consistent.     

Parameter identification for state estimation—application to bioprocess software sensors

Together with some on-line measurements, a reliable process model is the key ingredient of a successful state observer design. In common practice, the model parameters are inferred from experimental data so as to minimize a model prediction error, e.g. so as to minimize an output least-squares criterion. In this procedure, no care is actually exercised to ensure that the unmeasured model states are sensitive to the measured states. In turn, if sensitivity is too low, the resulting state observer will probably generate poor estimates of the unmeasured states. To alleviate these problems, a new parameter identification procedure is proposed in this study, which is based on a cost function combining a conventional prediction error criterion with a state estimation sensitivity measure. Minimization of this combined cost function produces a model dedicated to state estimation purposes. A thorough analysis of the procedure is presented in the context of bioreactor modeling, including parameter identification, model validation and design of extended Kalman filters and full horizon observers.     

Structural parameter identifiability analysis for dynamic reaction networks

A fundamental problem in model identification is to investigate whether unknown parameters in a given model structure potentially can be uniquely recovered from experimental data. This issue of global or structural identifiability is essential during nonlinear first principles model development where for a given set of measured variables it is desirable to investigate which parameters may be estimated prior to spending computational effort on the actual estimation. This contribution addresses the structural parameter identifiability problem for the typical case of reaction network models. The proposed analysis is performed in two phases. The first phase determines the structurally identifiable reaction rates based on reaction network stoichiometry. The second phase assesses the structural parameter identifiability of the specific kinetic rate expressions using a generating series expansion method based on Lie derivatives. The proposed systematic two phase methodology is illustrated on a mass action based model for an enzymatically catalyzed reaction pathway network where only a limited set of variables is measured. The methodology clearly pinpoints the structurally identifiable parameters in dependence of the given measurements and input perturbations.     

基于特征线法的一阶双曲型分布参数模型辨识及其在烟气脱硫过程中的应用

引言在现代控制工程领域中,许多工业对象实际上是非线性分布参数系统。由于这类对象的复杂性,原始模型常常进行集中线性化处理后分析和设计控制系统,然而系统本质的分布特性以及非线性引起的模型失配将造成控制的失败。这种情况促使在先进控制中越来越多地直接采用非线性分布参数机理     

Constrained receding‐horizon experiment design and parameter estimation in the presence of poor initial conditions

An optimal experiment design assumes the existence of an initial or nominal process model. The efficiency of this procedure depends on how the initial model is chosen. This creates a practical dilemma as estimating the model is precisely what the experiment tries to achieve. A novel approach to experiment design for identification of nonlinear systems is developed, with the purpose of reducing the influence of poor initial values. The experiment design and the parameter estimation are conducted iteratively under a receding‐horizon framework. By taking steady‐state prior knowledge into account, constraints on the parameters can be derived. Such constraints help reduce influence of poor initial models. The proposed algorithm is illustrated through examples to demonstrate its efficiency. © 2010 American Institute of Chemical Engineers AIChE J, 2011     

Estimation of output measurement variances for error-in-variables model parameter estimation

Error-in-variables model (EVM) methods require information about variances of input and output measured variables when estimating the parameters in mathematical models for chemical processes. In EVM, using replicate experiments for estimating output measurement variances is complicated because true values of inputs may be different when multiple attempts are made to repeat an experiment. To address this issue, we categorize attempted replicate experiments as: (i) true replicates (TRs) when uncertain inputs are the same in replicated runs and (ii) pseudo replicates (PRs) when measured inputs are the same, but unknown true values of inputs are different. We propose methodologies to obtain output measurement variance estimates and associated parameter estimates for both situations. We also propose bootstrap methods for obtaining joint-confidence information for the resulting parameter estimates. A copolymerization case study is used to illustrate the proposed techniques. We show that different assumptions noticeably affect the uncertainties in the resulting reactivity-ratio estimates.     

IDENTIFIABILITY AND DISTINGUISHABILITY TESTING FOR LINEAR MODELS IN HETEROGENEOUS CATALYSIS

If several values of the parameters of a model are associated with the same behavior, then the model is not identifiable and there is no hope of estimating a unique best value for the parameter vector from experimental data. Similarly, if several models with different structures correspond to the same behavior, then these models are not distinguishable and there is no hope of selecting a structure that best corresponds to the experimental data. Two methods for testing linear models for identifiability and distinguish ability are recalled and applied to types of catenary compartmental models encountered for instance when studying the isobutane-isobutene-hydrogen system by transient isotopic tracing.     

Estimation of parameters in monte carlo modelling

This paper presents a general method for estimating model parameters from experimental data when the model relating the parameters and input variables to the output responses is a Monte Carlo simulation. From a statistical point of view a Bayesian approach is used in which the distribution of the parameters is handled in discretized form as elements of an array in computer storage. The stochastic nature of the Monte Carlo model allows only an estimate of the distribution to be calculated from which the true distribution must then be estimated. For this purpose an exponentiated polynomial function has been found to be useful. The method provides point estimates as well as joint probability regions. Marginal distributions and distributions of functions of the parameters can also be handled. The motivation for exploring this alternative parameter estimation technique comes from the recognition that for some systems, particularly when the underlying process is stochastic in nature, Monte Carlo simulation often is the most suitable way of modelling. As such, the Monte Carlo approach increases the range of problems which can be handled by mathematical modelling. The technique is applied to the modelling of binary copolymerization. Two models, the Mayo-Lewis and the Penultimate Group Effects models, are considered and a method for discriminating between these models in the light of sequence distribution data is proposed.     

Parameter and state estimation in nonlinear stochastic continuous‐time dynamic models with unknown disturbance intensity

Approximate Maximum Likelihood Estimation (AMLE) is an algorithm for estimating the states and parameters of models described by stochastic differential equations (SDEs). In previous work (Varziri et al., Ind. Eng. Chem. Res., 47 (2), 380‐393, (2008); Varziri et al., Comp. Chem. Eng., in press), AMLE was developed for SDE systems in which process‐disturbance intensities and measurement‐noise variances were assumed to be known. In the current article, a new formulation of the AMLE objective function is proposed for the case in which measurement‐noise variance is available but the process‐disturbance intensity is not known a priori. The revised formulation provides estimates of the model parameters and disturbance intensities, as demonstrated using a nonlinear CSTR simulation study. Parameter confidence intervals are computed using theoretical linearization‐based expressions. The proposed method compares favourably with a Kalman‐filter‐based maximum likelihood method. The resulting parameter estimates and information about model mismatch will be useful to chemical engineers who use fundamental models for process monitoring and control.     

基于变分贝叶斯算法的线性变参数系统辨识

线性变参数系统（LPV）将多阶段、非线性的过程建模转化为线性多模型的辨识问题,是解决非线性过程建模的一个有效手段。由于实际工业过程存在各种干扰因素,导致被建模系统呈现随机性及模型参数的不确定性。针对这一问题,考虑采用变分贝叶斯（VB）算法对LPV模型进行辨识。该算法首先给定参数相应的先验分布,通过最大化目标函数的下界,从而估计得到参数的后验分布。不仅可实现对参数的点估计,同时量化了估计值的不确定性。针对典型二阶过程和连续搅拌反应釜（CSTR）,运用提出的算法进行仿真实验,表明了该贝叶斯估计方法的优越性。     

A hybrid genetic algorithm for efficient parameter estimation of large kinetic models

The development of predictive models is a time consuming, knowledge intensive, iterative process where an approximate model is proposed to explain experimental data, the model parameters that best fit the data are determined and the model is subsequently refined to improve its predictive capabilities. Ascertaining the validity of the proposed model is based upon how thoroughly the parameter search has been conducted in the allowable range. The determination of the optimal model parameters is complicated by the complexity/non-linearity of the model, potentially large number of equations and parameters, poor quality of the data, and lack of tight bounds for the parameter ranges. In this paper, we will critically evaluate a hybrid search procedure that employs a genetic algorithm for identifying promising regions of the solution space followed by the use of an optimizer to search locally in the identified regions. It has been found that this procedure is capable of identifying solutions that are essentially equivalent to the global optimum reported by a state-of-the-art global optimizer but much faster. A 13 parameter model that results in 60 differential-algebraic equations for propane aromatization on a zeolite catalyst is proposed as a more challenging test case to validate this algorithm. This hybrid technique has been able to locate multiple solutions that are nearly as good with respect to the “sum of squares” error criterion, but imply significantly different physical situations.     

Recursive State-space Model Identification of Non-uniformly Sampled Systems Using Singular Value Decomposition

In this paper a recursive state-space model identification method is proposed for non-uniformly sampled systems in industrial applications. Two cases for measuring all states and only output(s) of such a system are considered for identification. In the case of state measurement, an identification algorithm based on the singular value decomposition (SVD) is developed to estimate the model parameter matrices by using the least-squares fitting. In the case of output measurement only, another identification algorithm is given by combining the SVD approach with a hierarchical identification strategy. An example is used to demonstrate the effectiveness of the proposed identification method.     

Regularized maximum likelihood estimation of sparse stochastic monomolecular biochemical reaction networks

A sparse parameter matrix estimation method is proposed for identifying a stochastic monomolecular biochemical reaction network system. Identification of a reaction network can be achieved by estimating a sparse parameter matrix containing the reaction network structure and kinetics information. Stochastic dynamics of a biochemical reaction network system is usually modeled by a chemical master equation (CME) describing the time evolution of probability distributions for all possible states. This paper considers closed monomolecular reaction systems for which an exact analytical solution of the corresponding chemical master equation can be derived. The estimation method presented in this paper incorporates the closed-form solution into a regularized maximum likelihood estimation (MLE) for which model complexity is penalized. A simulation result is provided to verify performance improvement of regularized MLE over least-square estimation (LSE), which is based on a deterministic mass-average model, in the case of a small population size.     

Recursive least squares based estimation schemes for self-tuning control

Recursive Least Squares (RLS) is the most popular parametric identification method used for on-line process model estimation and self-tuning control. The basic least squares scheme is outlined in this paper and its lack of ability to track changing process parameters is illustrated and explained. Several variants of the basic algorithm which have appeared elsewhere in the literature are discussed. Some of these algorithms contain different modifications to the basic scheme which are intended to prevent this loss of alertness to changing process parameters. Other variations of the least squares algorithm are presented which attempt to deal with parameter estimation in the presence of disturbances and unmodelled process dynamics.     

Iterative identification of output error model for industrial processes with time delay subject to colored noise

To dealwith colored noise and unexpected load disturbance in identification of industrial processes with time delay, a bias-eliminated iterative least-squares (ILS) identification method is proposed in this paper to estimate the output error model parameters and time delay simultaneously. An extended observation vector is constructed to establish an ILS identification algorithm. Moreover, a variable forgetting factor is introduced to enhance the convergence rate of parameter estimation. For consistent estimation, an instrumental variable method is given to deal with the colored noise. The convergence and upper bound error of parameter estimation are analyzed. Two illustrative examples are used to show the effectiveness and merits of the proposed method.     

An Artificial Neural Network approximation based decomposition approach for parameter estimation of system of ordinary differential equations

In this work a new approach for parameter estimation which is based upon decomposing the problem into two subproblems is proposed, the first subproblem generates an Artificial Neural Network (ANN) model from the given data and then the second subproblem uses the ANN model to obtain an estimate of the parameters. The analytical derivates from the ANN model obtained from the first subproblem are used for obtaining the differential terms in the formulation of the second subproblem. This greatly simplifies the parameter estimation problem. The key advantage of the proposed approach is that solution of a large optimization problem requiring high computational resources is avoided and instead two smaller problems are solved. This approach is particularly useful for large and noisy data sets and nonlinear models where ANN models are known to perform quite well and therefore plays an important role in the solution of the overall parameter estimation problem.