Parameter estimation in nonlinear chemical and biological processes with unmeasured variables from small data sets

We propose an algorithm for parameter estimation in nonlinear chemical and biological stochastic processes with unmeasured variables and small data sets. The algorithm relies on an iterative approach wherein random samples of parameters and unmeasured variables are generated, from their respective posterior density functions, through Markov chain Monte Carlo simulations. The random samples are then used in approximating the posterior density functions of the parameters. The effectiveness of the algorithm is demonstrated through two biological examples—a feed-forward loop genetic regulatory network and a JAK–STAT signal transduction pathway.     

基于热传导反问题的各向异性材料热物性预测方法

采用不同方法对基于热传导反问题的固体热导率预测进行了研究。分别采用采用不同方法对基于热传导反问题的固体热导率预测进行了研究。分别采用Bayesian统计方法、Levenberg-Marquardt和遗传算法对二维各向异性材料的热物性进行了预测;并进行了分析比较。研究结果表明;Bayesian方法中热传导反问题的解是其后验概率密度的数学期望;而后验概率密度函数（PPDF）通过测定的温度进行计算获得;用Markov chain Monte Carlo算法计算后验状态空间以得到未知热导率的统计估计;采用Metropolis-Hasting算法进行数据采样构造Markov chain;并截取收敛后的样本进行分析。遗传算法是一种相对较新的用于最优化估计的方法;也可以用于求解反问题。     

A Bayesian framework for the calibration of cohesive zone models

Adhesively bonded joints are used in several industrial sectors. Cohezive Zone Modes can be used to predict the adhesive mechanical behaviour. This work presents an approach to calibrate Cohesive Zone Models (CZM) by means of Statistical Inverse Analysis. The Bayesian framework for Inverse Problems is used to infer about the CZM model parameters. The solution corresponds to the exploration of the posterior probability density function of the model parameters. The exploration of the posterior density is performed by means of Markov Chain Monte Carlo (MCMC) methods mixing Population-Based MCMC with Adaptive Metropolis (AD) strategies. The assessment of the approach is performed using measured data from a single-lap shear experimental set-up. Measured data from 5 test-specimens is used for calibration and measured data from five other test-specimens is used for model validation. It is proposed a stochastic effective model for the CZM parameters. The predictions of maximum force and maximum displacement that are provided by the effective model are in accordance with measured data that is used for validation.     

DPpackage: Bayesian Non- and Semi-parametric Modelling in R

Data analysis sometimes requires the relaxation of parametric assumptions in order to gain modeling flexibility and robustness against mis-specification of the probability model. In the Bayesian context, this is accomplished by placing a prior distribution on a function space, such as the space of all probability distributions or the space of all regression functions. Unfortunately, posterior distributions ranging over function spaces are highly complex and hence sampling methods play a key role. This paper provides an introduction to a simple, yet comprehensive, set of programs for the implementation of some Bayesian non- and semi-parametric models in R, DPpackage. Currently DPpackage includes models for marginal and conditional density estimation, ROC curve analysis, interval-censored data, binary regression data, item response data, longitudinal and clustered data using generalized linear mixed models, and regression data using generalized additive models. The package also contains functions to compute pseudo-Bayes factors for model comparison, and for eliciting the precision parameter of the Dirichlet process prior. To maximize computational efficiency, the actual sampling for each model is carried out using compiled FORTRAN.     

Estimation of parameters in monte carlo modelling

This paper presents a general method for estimating model parameters from experimental data when the model relating the parameters and input variables to the output responses is a Monte Carlo simulation. From a statistical point of view a Bayesian approach is used in which the distribution of the parameters is handled in discretized form as elements of an array in computer storage. The stochastic nature of the Monte Carlo model allows only an estimate of the distribution to be calculated from which the true distribution must then be estimated. For this purpose an exponentiated polynomial function has been found to be useful. The method provides point estimates as well as joint probability regions. Marginal distributions and distributions of functions of the parameters can also be handled. The motivation for exploring this alternative parameter estimation technique comes from the recognition that for some systems, particularly when the underlying process is stochastic in nature, Monte Carlo simulation often is the most suitable way of modelling. As such, the Monte Carlo approach increases the range of problems which can be handled by mathematical modelling. The technique is applied to the modelling of binary copolymerization. Two models, the Mayo-Lewis and the Penultimate Group Effects models, are considered and a method for discriminating between these models in the light of sequence distribution data is proposed.     

Quasi‐Bayesian Estimation of Time‐Varying Volatility in DSGE Models

We propose a novel quasi‐Bayesian Metropolis‐within‐Gibbs algorithm that can be used to estimate drifts in the shock volatilities of a linearized dynamic stochastic general equilibrium (DSGE) model. The resulting volatility estimates differ from the existing approaches in two ways. First, the time variation enters non‐parametrically, so that our approach ensures consistent estimation in a wide class of processes, thereby eliminating the need to specify the volatility law of motion and alleviating the risk of invalid inference due to mis‐specification. Second, the conditional quasi‐posterior of the drifting volatilities is available in closed form, which makes inference straightforward and simplifies existing algorithms. We apply our estimation procedure to a standard DSGE model and find that the estimated volatility paths are smoother compared to alternative stochastic volatility estimates. Moreover, we demonstrate that our procedure can deliver statistically significant improvements to the density forecasts of the DSGE model compared to alternative methods.     

Estimation of stationary autoregressive models with the Bayesian LASSO

This article explores the problem of estimating stationary autoregressive models from observed data using the Bayesian least absolute shrinkage and selection operator (LASSO). By characterizing the model in terms of partial autocorrelations, rather than coefficients, it becomes straightforward to guarantee that the estimated models are stationary. The form of the negative log‐likelihood is exploited to derive simple expressions for the conditional likelihood functions, leading to efficient algorithms for computing the posterior mode by coordinate‐wise descent and exploring the posterior distribution by Gibbs sampling. Both empirical Bayes and Bayesian methods are proposed for the estimation of the LASSO hyper‐parameter from the data. Simulations demonstrate that the Bayesian LASSO performs well in terms of prediction when compared with a standard autoregressive order selection method.     

Maximum likelihood estimation of noise covariance matrices for state estimation of autonomous hybrid systems

A critical aspect of developing Bayesian state estimators for hybrid systems, that involve a combination of continuous and discrete state variables, is to have a reasonably accurate characterization of the stochastic disturbances affecting their dynamics. Recently, Bavdekar et al. (2011) have proposed a maximum likelihood (ML) based framework for estimation of the noise covariance matrices from operating input–output data when an EKF is used for state estimation. In this work, the ML framework is extended to estimation of the noise covariance matrices associated with autonomous hybrid systems, and, to a wider class of recursive Bayesian filters. Under the assumption that the innovations generated by an estimator form a white noise sequence, the proposed ML framework computes the noise covariance matrices such that they maximize the log-likelihood function of the estimator innovations. The efficacy of the proposed scheme is demonstrated through the simulation and experimental studies on the benchmark three-tank system.     

RECURSIVE ESTIMATION IN SWITCHING AUTOREGRESSIONS WITH A MARKOV REGIME

Abstract. A hidden Markov regime is a Markov process that governs the time or space dependent distributions of an observed stochastic process. We propose a recursive algorithm for parameter estimation in a switching autoregressive process governed by a hidden Markov chain. A common approach to the recursive estimation problem is to base the estimation on suboptimal modifications of Kalman filtering techniques. The main idea in this paper is to use the maximum likelihood method and from this develop a recursive EM algorithm.     

Reversible Jump Markov Chain Monte Carlo Strategies for Bayesian Model Selection in Autoregressive Processes

Abstract.  This paper addresses the problem of Bayesian inference in autoregressive (AR) processes in the case where the correct model order is unknown. Original hierarchical prior models that allow the stationarity of the model to be enforced are proposed. Obtaining the quantities of interest, such as parameter estimates, predictions of future values of the time series, posterior model-order probabilities, etc., requires integration with respect to the full posterior distribution, an operation which is analytically intractable. Reversible jump Markov chain Monte Carlo (MCMC) algorithms are developed to perform the required integration implicitly by simulating from the posterior distribution. The methods developed are evaluated in simulation studies on a number of synthetic and real data sets.     

Semiparametric Bayesian Inference of Long-Memory Stochastic Volatility Models

Abstract.  In this paper, a semiparametric, Bayesian estimator of the long-memory stochastic volatility model's fractional order of integration is presented. This new estimator relies on a highly efficient, Markov chain Monte Carlo (MCMC) sampler of the model's posterior distribution. The MCMC algorithm is set forth in the time-scale domain of the stochastic volatility model's wavelet representation. The key to and centerpiece of this new algorithm is the quick and efficient multi-state sampler of the latent volatility's wavelet coefficients. A multi-state sampler of the latent wavelet coefficients is only possible because of the near-independent multivariate distribution of the long-memory process's wavelet coefficients. Using simulated and empirical stock return data, we find that our algorithm produces uncorrelated draws of the posterior distribution and point estimates that rival existing long-memory stochastic volatility estimators.     

Parameter estimation in batch process using EM algorithm with particle filter

This paper investigates a parameter estimation problem for batch processes through the maximum likelihood method. In batch processes, the initial state usually relates to the states of previous batches. The proposed algorithm takes batch-to-batch correlations into account by employing an initial state transition equation to model the dynamics along the batch dimension. By treating the unmeasured states and the parameters as hidden variables, the maximum likelihood estimation is accomplished through the expectation–maximization (EM) algorithm, where the smoothing for the terminal state and the filtering for the initial state are specially considered. Due to the nonlinearity and non-Gaussianity in the state space model, particle filtering methods are employed for the implementation of filtering and smoothing. Through alternating between the expectation step and the maximization step, the unknown parameters along with states are estimated. Simulation examples demonstrate the proposed estimation approach.     

A novel dynamic bayesian network‐based networked process monitoring approach for fault detection,propagation identification,and root cause diagnosis

A novel networked process monitoring, fault propagation identification, and root cause diagnosis approach is developed in this study. First, process network structure is determined from prior process knowledge and analysis. The network model parameters including the conditional probability density functions of different nodes are then estimated from process operating data to characterize the causal relationships among the monitored variables. Subsequently, the Bayesian inference‐based abnormality likelihood index is proposed to detect abnormal events in chemical processes. After the process fault is detected, the novel dynamic Bayesian probability and contribution indices are further developed from the transitional probabilities of monitored variables to identify the major faulty effect variables with significant upsets. With the dynamic Bayesian contribution index, the statistical inference rules are, thus, designed to search for the fault propagation pathways from the downstream backwards to the upstream process. In this way, the ending nodes in the identified propagation pathways can be captured as the root cause variables of process faults. Meanwhile, the identified fault propagation sequence provides an in‐depth understanding as to the interactive effects of faults throughout the processes. The proposed approach is demonstrated using the illustrative continuous stirred tank reactor system and the Tennessee Eastman chemical process with the fault propagation identification results compared against those of the transfer entropy‐based monitoring method. The results show that the novel networked process monitoring and diagnosis approach can accurately detect abnormal events, identify the fault propagation pathways, and diagnose the root cause variables. © 2013 American Institute of Chemical Engineers AIChE J, 59: 2348–2365, 2013     

The Marginal Density of a TMA(1) Process

This note reconsiders the marginal density of a threshold moving average process and proposes a simple yet effective numerical algorithm to implement that by solving an associated integral equation. This algorithm can also be applied to calculate stationary probability density or distribution functions of a few other types of nonlinear stationary stochastic processes numerically.     

Machine learning modelling and evaluation of jet fires from natural gas processing,storage, and transport

Jet fires and their repercussions play a significant role in catastrophic incidents that typically have a cascading impact in process industries. Several hydrocarbon experiments from 19 papers were incorporated into the current endeavour to develop simulations of jet flames using machine learning (ML) models. Dimensionless characteristics have been used as output and input variables, including mass flow rates, fuel density, jet flame length, and heat release fluxes. When training three layers of the multi-layer feedforward neural network (MLFFNN) method, a Bayesian regularization backpropagation approach was adopted and evaluated with the radial based functions (RBF) algorithm. Through an optimization procedure, the first and second hidden layers of the MLFFNN have been optimized to include 10 and five neurons, respectively. The RBF algorithm with 40 neurons in a single layer has been computed using the same method. The best mean square error (MSE) validation results for RBF and MLFFNN were 0.006 and 0.0002, respectively, for 40 and 100 epochs. The MLFFNN and RBF models' respective regression statistical analysis outputs were 0.9949 and 0.9645. The ML method has been identified as a potentially useful technique for precisely predicting the geometrical and radiative characteristics of jet flames.     

A direct sampling particle filter from approximate conditional density function supported on constrained state space

Constraints on the state vector must be taken into account in the state estimation problem. Recently, acceptance/rejection and projection methods are proposed in the particle filter framework for constraining the particles. A weighted least squares formulation is used for constraining samples in unscented and ensemble Kalman filters. In this paper, direct sampling from an approximate conditional probability density function (pdf) is proposed. It is obtained by approximating the a priori pdf as a Gaussian. The support of the conditional density is a subset of the intersection of two supports, the 3-sigma bounds of the priori Gaussian and the constrained state space. A direct sampling algorithm is proposed for handling linear and nonlinear equality and inequality constraints. The algorithm uses the constrained mode for nonlinear constraints.     

Bayesian analysis of autoregressive fractionally integrated moving-average processes

For the autoregressive fractionally integrated moving-average (ARFIMA) processes which characterize both long-memory and short-memory behavior in time series, we formulate Bayesian inference using Markov chain Monte Carlo methods. We derive a form for the joint posterior distribution of the parameters that is computationally feasible for repetitive evaluation within a modified Gibbs sampling algorithm that we employ. We illustrate our approach through two examples.     

缺失数据下基于EM算法的非线性过程建模

线性时变参数系统(LPV)将多阶段、非线性的过程建模转化为线性多模型的辨识问题,近年来得到了极大关注。考虑缺失数据下LPV系统的离线建模问题,首先引入一个二进制变量表征输出样本缺失状态,选取过程关键变量作为调度变量,确定主要工况点;然后围绕不同工况点建立局部子模型,将输出缺失部分和采样数据的模型归属当作隐藏变量,利用EM算法进行参数估计,再采用高斯权重函数融合各子模型。最后分别针对典型二阶过程和连续搅拌反应釜(CSTR),运用提出的多模型和算法进行仿真实验,表明有效性。     

D-vine copulas混合模型及其在故障检测中的应用

过程监控技术是保证现代流程工业安全平稳运行及产品质量的有效手段。传统的过程监控方法大多采用维度约简方法提取数据特征,且要求过程数据必须服从高斯分布、线性等限制条件,对复杂工况条件下发生的故障难以取得较好的检测效果。因此,提出了混合D-vine copulas故障诊断模型,在不降维的情况下直接刻画数据中存在的复杂相关关系,构建过程变量的统计模型实现对存在非线性与非高斯性过程的精确描述。通过EM算法和伪极大似然估计优化混合模型参数,然后结合高密度区域（HDR）与密度分位数法等理论,构建广义贝叶斯概率（GBIP）指标实现对过程的实时监测。数值例子及在TE过程上的仿真结果说明了该混合模型的有效性及在故障检测中的良好性能。     

Bayesian Inference for Time Series with Stable Innovations

This paper describes Bayesian inference for a linear time series model with stable innovations. An advantage of the Bayesian approach is that it enables the simultaneous estimation of the parameters characterizing the stable law and the parameters of the linear autoregressive moving-average model. Our approach uses a Metropolis–Hastings algorithm to generate samples from the joint posterior distribution of all the parameters and subsequent inference is based on these samples. We illustrate our approach using data simulated from three linear processes with stable innovations and a real data set