Bayesian inference in probabilistic risk assessment—The current state of the art

Markov chain Monte Carlo (MCMC) approaches to sampling directly from the joint posterior distribution of aleatory model parameters have led to tremendous advances in Bayesian inference capability in a wide variety of fields, including probabilistic risk analysis. The advent of freely available software coupled with inexpensive computing power has catalyzed this advance. This paper examines where the risk assessment community is with respect to implementing modern computational-based Bayesian approaches to inference. Through a series of examples in different topical areas, it introduces salient concepts and illustrates the practical application of Bayesian inference via MCMC sampling to a variety of important problems.     

基于多链差分进化的贝叶斯有限元模型修正方法

针对传统贝叶斯算法在高维参数下采样效率低且收敛难的问题,建立了基于多链差分进化算法的贝叶斯有限元模型修正方法。在标准马尔可夫链蒙特卡罗（MCMC）方法的基础上,引入差分进化算法,通过多条马氏链间的随机差分运算来自适应选择条件分布的大小和方向以快速逼近目标分布;引入子空间采样算法,通过自适应选择优良的参数维度进行采样以提高采样效率;引入异常链检测算法,通过在采样的非平稳期对马氏链进行异常检测与剔除以提高在平稳期的采样效率。简支梁理论模型和实验室4层框架结构的模型修正结果表明：该方法修正精度较高,且具有良好的抗噪性,在高阶频率以及振型下的修正效果均优于DRAM算法,为解决不确定性模型修正中的计算精度提供了一种新手段。     

Sensitivity estimations for Bayesian inference models solved by MCMC methods

The advent of Markov Chain Monte Carlo (MCMC) methods to simulate posterior distributions has virtually revolutionized the practice of Bayesian statistics. Unfortunately, sensitivity analysis in MCMC methods is a difficult task. In this paper, a computationally low-cost method to estimate local parametric sensitivities in Bayesian models is proposed. The sensitivity measure considered here is the gradient vector of a posterior quantity with respect to the parameter. The gradient vector components are estimated by using a result based on the integral/derivative interchange. The MCMC simulations used to estimate the posterior quantity can be re-used to estimate the sensitivity measures and their errors, avoiding the need for further sampling. The proposed method is easy to apply in practice as it is shown with an illustrative example.     

A Monte Carlo method for calculating Bayesian uncertainties in internal dosimetry

This paper presents a novel Monte Carlo method (WeLMoS, Weighted Likelihood Monte-Carlo sampling method) that has been developed to perform Bayesian analyses of monitoring data. The WeLMoS method randomly samples parameters from continuous prior probability distributions and then weights each vector by its likelihood (i.e. its goodness of fit to the measurement data). Furthermore, in order to quality assure the method, and assess its strengths and weaknesses, a second method (MCMC, Markov chain Monte Carlo) has also been developed. The MCMC method uses the Metropolis algorithm to sample directly from the posterior distribution of parameters. The methods are evaluated and compared using an artificially generated case involving an exposure to a plutonium nitrate aerosol. In addition to calculating the uncertainty on internal dose, the methods can also calculate the probability distribution of model parameter values given the observed data. In other words, the techniques provide a powerful tool to obtain the estimates of parameter values that best fit the data and the associated uncertainty on these estimates. Current applications of the methodology, including the determination of lung solubility parameters, from volunteer and cohort data, are also discussed.     

Uncertainty assessment for reconstructions based on deformable geometry

Deformable geometric models can be used in the context of Bayesian analysis to solve ill-posed tomographic reconstruction problems. The uncertainties associated with a Bayesian analysis may be assessed by generating a set of random samples from the posterior, which may be accomplished using a Markov Chain Monte Carlo (MCMC) technique. We demonstrate the combination of these techniques for a reconstruction of a two-dimensional object from two orthogonal noisy projections. The reconstructed object is modeled in terms of a deformable geometrically defined boundary with a uniform interior density yielding a nonlinear reconstruction problem. We show how an MCMC sequence can be used to estimate uncertainties in the location of the edge of the reconstructed object. © 1997 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 8, 506–512, 1997     

Performance study of marginal posterior density estimation via Kullback-Leibler divergence

In this article, we introduce Kullback-Leibler (K-L) divergence as a performance measure of marginal posterior density estimation. We show that the K-L divergence can be used to compare two density estimators as well as to assess convergence of a marginal density estimator. We also examine performance of the importance-weighted marginal density estimation (IWMDE) proposed by Chen (1994) under the K-L divergence and we further extend the IWMDE to some more complex Bayesian models where the kernel method, which is widely used for estimating marginal densities using Markov chain Monte Carlo (MCMC) sampling outputs is not applicable. Finally, we use a constrained linear multiple regression model as an example to illustrate our methodology.     

三水源新安江模型参数不确定性分析PAM算法

针对水文模型参数不确定性分析常用方法 收敛速度缓慢,容易陷入参数空间局部最优区域等 问题,提出了PAM (parallel adaptive metropolis) 算法;对三水源新安江模型参数不确定性进行分析 研究。实例研究表明显著提高了计算速度和求解质 量,参数后验分布结果为区间预报提供了条件。     

结构物理参数识别的贝叶斯估计马尔可夫蒙特卡罗方法

从结构动力特征方程出发,以结构主模态参数为观测量,推得结构物理参数线性回归模型。对该模型应用贝叶斯估计理论得到物理参数后验联合分布,再结合马尔可夫蒙特卡罗抽样方法给出各个物理参数的边缘概率分布和最优估计值,而提出了基于结构主模态参数的结构物理参数识别贝叶斯估计马尔可夫蒙特卡罗方法。对五层剪切型结构的数值研究表明,此方法能够利用少数主模态参数给出结构质量和刚度参数的概率分布和最优识别值,而且在主模态参数较准确时识别误差很小。     

Performance study of marginal posterior density estimation via Kullback-Leibler divergence

In this article, we introduce Kullback-Leibler (K-L) divergence as a performance measure of marginal posterior density estimation. We show that the K-L divergence can be used to compare two density estimators as well as to assess convergence of a marginal density estimator. We also examine performance of the importance-weighted marginal density estimation (IWMDE) proposed by Chen (1994) under the K-L divergence and we further extend the IWMDE to some more complex Bayesian models where the kernel method, which is widely used for estimating marginal densities using Markov chain Monte Carlo (MCMC) sampling outputs is not applicable. Finally, we use a constrained linear multiple regression model as an example to illustrate our methodology.     

一种基于改进MCMC算法的模型修正方法

标准马尔可夫链蒙特卡罗(MCMC)算法不易收敛、拒绝率高,使其应用受到限制。在贝叶斯方法中引入最大熵值法来估计参数的后验概率密度函数最大值,进而将布谷鸟算法中新鸟巢更新的思想融入Metropolis-Hasting(MH)抽样算法得到改进的MH抽样算法,同时使用支持向量机(SVM)建立待修正参数与有限元模型输出之间的代理模型,以提高模型修正的计算效率。分别使用三自由度线性系统和平面桁架模型来验证本文方法的有效性,结果表明:修正后样本的马尔可夫链混合性能好,停滞概率低,修正后参数相对误差均小于2%。     

Markov chain Monte Carlo posterior density approximation for a groove-dimensioning purpose

The purpose of this paper is to present a new approach for measurand uncertainty characterization. The Markov chain Monte Carlo (MCMC) is applied to measurand probability density function (pdf) estimation, which is considered as an inverse problem. The measurement characterization is driven by the pdf estimation in a nonlinear Gaussian framework with unknown variance and with limited observed data. These techniques are applied to a realistic measurand problem of groove dimensioning using remote field eddy current (RFEC) inspection. The application of resampling methods such as bootstrap and the perfect sampling for convergence diagnostics purposes gives large improvements in the accuracy of the MCMC estimates.     

Bayesian analysis for estimating statistical parameter distributions of elasto-viscoplastic material models

High temperature design methods rely on constitutive models for inelastic deformation and failure typically calibrated against the mean of experimental data without considering the associated scatter. Variability may arise from the experimental data acquisition process, from heat-to-heat material property variations, or both and need to be accurately captured to predict parameter bounds leading to efficient component design. Applying the Bayesian Markov Chain Monte Carlo (MCMC) method to produce statistical models capturing the underlying uncertainty in the experimental data is an area of ongoing research interest. This work varies aspects of the Bayesian MCMC method and explores their effect on the posterior parameter distributions for a uniaxial elasto-viscoplastic damage model using synthetically generated reference data. From our analysis with the uniaxial inelastic model we determine that an informed prior distribution including different types of test conditions results in more accurate posterior parameter distributions. The parameter posterior distributions, however, do not improve when increasing the number of similar experimental data. Additionally, changing the amount of scatter in the data affects the quality of the posterior distributions, especially for the less sensitive model parameters. Moreover, we perform a sensitivity study of the model parameters against the likelihood function prior to the Bayesian analysis. The results of the sensitivity analysis help to determine the reliability of the posterior distributions and reduce the dimensionality of the problem by fixing the insensitive parameters. The comprehensive study described in this work demonstrates how to efficiently apply the Bayesian MCMC methodology to capture parameter uncertainties in high temperature inelastic material models. Quantifying these uncertainties in inelastic models will improve high temperature engineering design practices and lead to safer, more effective component designs.     

Mining protein kinases regulation using graphical models

Abnormal kinase activity is a frequent cause of diseases, which makes kinases a promising pharmacological target. Thus, it is critical to identify the characteristics of protein kinases regulation by studying the activation and inhibition of kinase subunits in response to varied stimuli. Bayesian network (BN) is a formalism for probabilistic reasoning that has been widely used for learning dependency models. However, for high-dimensional discrete random vectors the set of plausible models becomes large and a full comparison of all the posterior probabilities related to the competing models becomes infeasible. A solution to this problem is based on the Markov Chain Monte Carlo (MCMC) method. This paper proposes a BN-based framework to discover the dependency correlations of kinase regulation. Our approach is to apply the MCMC method to generate a sequence of samples from a probability distribution, by which to approximate the distribution. The frequent connections (edges) are identified from the obtained sampling graphical models. Our results point to a number of novel candidate regulation patterns that are interesting in biology and include inferred associations that were unknown.     

An Efficient Sampling Technique for Bayesian Inference With Computationally Demanding Models

In the environmental sciences, a large knowledge base is typically available on an investigated system or at least on similar systems. This makes the application of Bayesian inference techniques in environmental modeling very promising. However, environmental systems are often described by complex, computationally demanding simulation models. This strongly limits the application of Bayesian inference techniques, because numerical implementation of these techniques requires a very large number of simulation runs. The development of efficient sampling techniques that attempt to approximate the posterior distribution with a relatively small parameter sample can extend the range of applicability of Bayesian inference techniques to such models. In this article a sampling technique is presented that tries to achieve this goal. The proposed technique combines numerical techniques typically applied in Bayesian inference, including posterior maximization, local normal approximation, and importance sampling, with copula techniques for the construction of a multivariate distribution with given marginals and correlation structure and with low-discrepancy sampling. This combination improves the approximation of the posterior distribution by the sampling distribution and improves the accuracy of results for small sample sizes. The usefulness of the proposed technique is demonstrated for a simple model that contains the major elements of models used in the environmental sciences. The results indicate that the proposed technique outperforms conventional techniques (random sampling from simpler distributions or Markov chain Monte Carlo techniques) in cases in which the analysis can be limited to a relatively small number of parameters.     

Monte Carlo method for transient eddy-current calculations

This paper deals with the calculation of transient eddy currents by a Markov chain Monte Carlo (MCMC) method. After we illustrate the principle in a one-dimensional calculation, we treat a two-dimensional problem. Then, we show simulation results and we discuss advantages and disadvantages of this method. An important advantage is that MCMC methods can be more efficient than finite-element methods in transient problems because they give an estimate of the solution at a point in space without calculation of the whole field distribution.     

A Decomposition Strategy for the Variational Inference of Complex Systems

Markov chain Monte Carlo approaches have been widely used for Bayesian inference. The drawback of these methods is that they can be computationally prohibitive especially when complex models are analyzed. In such cases, variational methods may provide an efficient and attractive alternative. However, the variational methods reported to date are applicable to relatively simple models and most are based on a factorized approximation to the posterior distribution. Here, we propose a variational approach that is capable of handling models that consist of a system of differential-algebraic equations and whose posterior approximation can be represented by a multivariate distribution. Under the proposed approach, the solution of the variational inference problem is decomposed into three steps: a maximum a posteriori optimization, which is facilitated by using an orthogonal collocation approach, a preprocessing step, which is based on the estimation of the eigenvectors of the posterior covariance matrix, and an expected propagation optimization problem. To tackle multivariate integration, we employ quadratures derived from the Smolyak rule (sparse grids). Examples are reported to elucidate the advantages and limitations of the proposed methodology. The results are compared to the solutions obtained from a Markov chain Monte Carlo approach. It is demonstrated that significant computational savings can be gained using the proposed approach. This article has supplementary material online.     

Subset simulation for structural reliability sensitivity analysis

Based on two procedures for efficiently generating conditional samples, i.e. Markov chain Monte Carlo (MCMC) simulation and importance sampling (IS), two reliability sensitivity (RS) algorithms are presented. On the basis of reliability analysis of Subset simulation (Subsim), the RS of the failure probability with respect to the distribution parameter of the basic variable is transformed as a set of RS of conditional failure probabilities with respect to the distribution parameter of the basic variable. By use of the conditional samples generated by MCMC simulation and IS, procedures are established to estimate the RS of the conditional failure probabilities. The formulae of the RS estimator, its variance and its coefficient of variation are derived in detail. The results of the illustrations show high efficiency and high precision of the presented algorithms, and it is suitable for highly nonlinear limit state equation and structural system with single and multiple failure modes.     

Sequential Designs Based on Bayesian Uncertainty Quantification in Sparse Representation Surrogate Modeling

A numerical method, called overcomplete basis surrogate method (OBSM), was recently proposed, which employs overcomplete basis functions to achieve sparse representations. While the method can handle nonstationary response without the need of inverting large covariance matrices, it lacks the capability to quantify uncertainty in predictions. We address this issue by proposing a Bayesian approach that first imposes a normal prior on the large space of linear coefficients, then applies the Markov chain Monte Carlo (MCMC) algorithm to generate posterior samples for predictions. From these samples, Bayesian credible intervals can then be obtained to assess prediction uncertainty. A key application for the proposed method is the efficient construction of sequential designs. Several sequential design procedures with different infill criteria are proposed based on the generated posterior samples. Numerical studies show that the proposed schemes are capable of solving problems of positive point identification, optimization, and surrogate fitting.     

A fast Bayesian approach using adaptive densifying approximation technique accelerated MCMC

A novel algorithm is presented in this study to improve the efficiency and accuracy of Bayesian approach for fast sampling of posterior distributions of the unknown structure parameters. This algorithm can save a computational cost by resolving the efficiency problem in Bayesian identifications. In this algorithm, an approximation model based on the radial basis function is first used to replace the actual joint posterior distribution of the unknown parameters. An adaptive densifying technique is then introduced to increase the accuracy of the approximation model by reconstructing them with densified samples. Finally, the marginal posterior distributions for each parameter with fine accuracy can be efficiently achieved using the Markov Chain Monte Carlo method based on the present densified approximation model. Two numerical examples are investigated to demonstrate that the present algorithm can achieve significant computational gains without sacrificing the accuracy.     

Improper and proper posteriors with improper priors in a Poisson-gamma hierarchical model

In Bayesian analysis, Markov chain Monte Carlo techniques have become so easy to use that it is possible to erroneously generate observations from a posterior distribution that is improper. In this paper we discussed the Poisson-gamma hierarchical model. A flexible prior distribution is discussed, one which allows the user to choose improper priors. Necessary and sufficient conditions are given for the posterior distribution to be proper and for the posterior moments to exist. An example using data on brain lesions for multiple sclerosis patients is presented to demonstrate the difficulty in diagnosing whether the posterior is proper.