An Efficient Sampling Technique for Bayesian Inference With Computationally Demanding Models

In the environmental sciences, a large knowledge base is typically available on an investigated system or at least on similar systems. This makes the application of Bayesian inference techniques in environmental modeling very promising. However, environmental systems are often described by complex, computationally demanding simulation models. This strongly limits the application of Bayesian inference techniques, because numerical implementation of these techniques requires a very large number of simulation runs. The development of efficient sampling techniques that attempt to approximate the posterior distribution with a relatively small parameter sample can extend the range of applicability of Bayesian inference techniques to such models. In this article a sampling technique is presented that tries to achieve this goal. The proposed technique combines numerical techniques typically applied in Bayesian inference, including posterior maximization, local normal approximation, and importance sampling, with copula techniques for the construction of a multivariate distribution with given marginals and correlation structure and with low-discrepancy sampling. This combination improves the approximation of the posterior distribution by the sampling distribution and improves the accuracy of results for small sample sizes. The usefulness of the proposed technique is demonstrated for a simple model that contains the major elements of models used in the environmental sciences. The results indicate that the proposed technique outperforms conventional techniques (random sampling from simpler distributions or Markov chain Monte Carlo techniques) in cases in which the analysis can be limited to a relatively small number of parameters.     

An efficient analytical Bayesian method for reliability and system response updating based on Laplace and inverse first-order reliability computations

This paper presents an efficient analytical Bayesian method for reliability and system response updating without using simulations. The method includes additional information such as measurement data via Bayesian modeling to reduce estimation uncertainties. Laplace approximation method is used to evaluate Bayesian posterior distributions analytically. An efficient algorithm based on inverse first-order reliability method is developed to evaluate system responses given a reliability index or confidence interval. Since the proposed method involves no simulations such as Monte Carlo or Markov chain Monte Carlo simulations, the overall computational efficiency improves significantly, particularly for problems with complicated performance functions. A practical fatigue crack propagation problem with experimental data, and a structural scale example are presented for methodology demonstration. The accuracy and computational efficiency of the proposed method are compared with traditional simulation-based methods.     

A Bayesian Predictive Analysis of Step‐Stress Accelerated Tests in Gamma Degradation‐Based Processes

Degradation modeling might be an alternative to the conventional life test in reliability assessment for high quality products. This paper develops a Bayesian approach to the step‐stress accelerated degradation test. Reliability inference of the population is made based on the posterior distribution of the underlying parameters with the aid of Markov chain Monte Carlo method. Further sequential reliability inference on individual product under normal condition is also proposed. Simulation study and an illustrative example are presented to show the appropriateness of the proposed method. Copyright © 2017 John Wiley & Sons, Ltd.     

Bayesian analysis of reduced rank regression

Summary Multivariate regression is a well known method to relate multivariate responses to multivariate predictors. When the responses follow a factor analysis model and the latent factors can be linked with the predictors, a reduced rank regression model is obtained which is defined as a multivariate regression model with rank restrictions on the coefficient matrix. Many linear multivariate methods can be interpreted as special cases of such a reduced rank regression model. A Bayesian analysis with informative conjugate priors is presented, emphasizing the importance of an adequate parametrization to obtain interpretable results. Gibbs sampling a Markov chain Monte Carlo method, is proposed to calculate the posterior and predictive distribution. The methods are motivated and illustrated by an example where quantitative relationships between biological activity and chemical structure are searched for (QSAR).     

A stochastic approach to quantifying the blur with uncertainty estimation for high-energy X-ray imaging systems

One of the primary causes of blur in a high-energy X-ray imaging system is the shape and extent of the radiation source, or ‘spot’. It is important to be able to quantify the size of the spot as it provides a lower bound on the recoverable resolution for a radiograph, and penumbral imaging methods – which involve the analysis of blur caused by a structured aperture – can be used to obtain the spot’s spatial profile. We present a Bayesian approach for estimating the spot shape that, unlike variational methods, is robust to the initial choice of parameters. The posterior is obtained from a normal likelihood, which was constructed from a weighted least squares approximation to a Poisson noise model, and prior assumptions that enforce both smoothness and non-negativity constraints. A Markov chain Monte Carlo algorithm is used to obtain samples from the target posterior, and the reconstruction and uncertainty estimates are the computed mean and variance of the samples, respectively. Synthetic data-sets are used to demonstrate accurate reconstruction, while real data taken with high-energy X-ray imaging systems are used to demonstrate applicability and feasibility.     

A general Bayes weibull inference model for accelerated life testing

This article presents the development of a general Bayes inference model for accelerated life testing. The failure times at a constant stress level are assumed to belong to a Weibull distribution, but the specification of strict adherence to a parametric time-transformation function is not required. Rather, prior information is used to indirectly define a multivariate prior distribution for the scale parameters at the various stress levels and the common shape parameter. Using the approach, Bayes point estimates as well as probability statements for use-stress (and accelerated) life parameters may be inferred from a host of testing scenarios. The inference procedure accommodates both the interval data sampling strategy and type I censored sampling strategy for the collection of ALT test data. The inference procedure uses the well-known MCMC (Markov Chain Monte Carlo) methods to derive posterior approximations. The approach is illustrated with an example.     

A Bayesian approach to sequential monitoring of nonlinear profiles using wavelets

We consider change‐point detection and estimation in sequences of functional observations. This setting often arises when the quality of a process is characterized by such observations, called profiles, and monitoring profiles for changes in structure can be used to ensure the stability of the process over time. While interest in phase II profile monitoring has grown, few methods approach the problem from a Bayesian perspective. We propose a wavelet‐based Bayesian methodology that bases inference on the posterior distribution of the change point without placing restrictive assumptions on the form of profiles. By obtaining an analytic form of this posterior distribution, we allow the proposed method to run online without using Markov chain Monte Carlo (MCMC) approximation. Wavelets, an effective tool for estimating nonlinear signals from noise‐contaminated observations, enable us to flexibly distinguish between sustained changes in profiles and the inherent variability of the process. We analyze observed profiles in the wavelet domain and consider two possible prior distributions for coefficients corresponding to the unknown change in the sequence. These priors, previously applied in the nonparametric regression setting, yield tuning‐free choices of hyperparameters. We present additional considerations for controlling computational complexity over time and their effects on performance. The proposed method significantly outperforms a relevant frequentist competitor on simulated data.     

A Bayesian generalized Eyring-Weibull accelerated life testing model

In this paper, a novel approach to a Bayesian accelerated life testing model is presented. The Weibull distribution is used as the life distribution and the generalized Eyring model as the time transformation function. This is a model that allows for the use of more than one stressor, whereas other commonly used acceleration models, such as the Arrhenius and power law models, incorporate one stressor. The use of the generalized Eyring-Weibull model developed in this paper is demonstrated in a case study, where Markov chain Monte Carlo methods are utilized to generate samples for posterior inference.     

Markov switching negative binomial models: An application to vehicle accident frequencies

In this paper, two-state Markov switching models are proposed to study accident frequencies. These models assume that there are two unobserved states of roadway safety, and that roadway entities (roadway segments) can switch between these states over time. The states are distinct, in the sense that in the different states accident frequencies are generated by separate counting processes (by separate Poisson or negative binomial processes). To demonstrate the applicability of the approach presented herein, two-state Markov switching negative binomial models are estimated using five-year accident frequencies on Indiana interstate highway segments. Bayesian inference methods and Markov Chain Monte Carlo (MCMC) simulations are used for model estimation. The estimated Markov switching models result in a superior statistical fit relative to the standard (single-state) negative binomial model. It is found that the more frequent state is safer and it is correlated with better weather conditions. The less frequent state is found to be less safe and to be correlated with adverse weather conditions.     

Bayesian design of life testing plans under hybrid censoring scheme

This article describes Bayes design of hybrid‐censored life testing plans. A design criterion based on posterior variance of quantile of suitable order is proposed. The Weibull lifetime model with gamma prior distribution on model parameters is considered for illustration. Instead of using Markov chain Monte Carlo technique to compute the posterior quantities of interest, a large sample approximation is considered, which is easy to apply. Some life testing plans are presented. The effect of different prior information on the posterior quantity of interest is studied.     

Sequential optimization with particle splitting-based reliability assessment for engineering design under uncertainties

The evaluation of probabilistic constraints plays an important role in reliability-based design optimization. Traditional simulation methods such as Monte Carlo simulation can provide highly accurate results, but they are often computationally intensive to implement. To improve the computational efficiency of the Monte Carlo method, this article proposes a particle splitting approach, a rare-event simulation technique that evaluates probabilistic constraints. The particle splitting-based reliability assessment is integrated into the iterative steps of design optimization. The proposed method provides an enhancement of subset simulation by increasing sample diversity and producing a stable solution. This method is further extended to address the problem with multiple probabilistic constraints. The performance of the particle splitting approach is compared with the most probable point based method and other approximation methods through examples.     

A fast Bayesian approach using adaptive densifying approximation technique accelerated MCMC

A novel algorithm is presented in this study to improve the efficiency and accuracy of Bayesian approach for fast sampling of posterior distributions of the unknown structure parameters. This algorithm can save a computational cost by resolving the efficiency problem in Bayesian identifications. In this algorithm, an approximation model based on the radial basis function is first used to replace the actual joint posterior distribution of the unknown parameters. An adaptive densifying technique is then introduced to increase the accuracy of the approximation model by reconstructing them with densified samples. Finally, the marginal posterior distributions for each parameter with fine accuracy can be efficiently achieved using the Markov Chain Monte Carlo method based on the present densified approximation model. Two numerical examples are investigated to demonstrate that the present algorithm can achieve significant computational gains without sacrificing the accuracy.     

A new methodology for Bayesian history matching using parallel interacting Markov chain Monte Carlo

This paper presents an innovative application of a new class of parallel interacting Markov chains Monte Carlo to solve the Bayesian history matching (BHM) problem. BHM consists of sampling a posterior distribution given by the Bayesian theorem. Markov chain Monte Carlo (MCMC) is well suited for sampling, in principle, any type of distribution; however the number of iteration required by the traditional single-chain MCMC can be prohibitive in BHM applications. Furthermore, history matching is typically a highly nonlinear inverse problem, which leads in very complex posterior distributions, characterized by many separated modes. Therefore, single chain can be trapped into a local mode. Parallel interacting chains is an interesting way to overcome this problem, as shown in this paper. In addition, we presented new approaches to define starting points for the parallel chains. For validation purposes, the proposed methodology is firstly applied in a simple but challenging cross section reservoir model with many modes in the posterior distribution. Afterwards, the application to a realistic case integrated to geostatistical modelling is also presented. The results showed that the combination of parallel interacting chain with the capabilities of distributed computing commonly available nowadays is very promising to solve the BHM problem.     

Markov switching multinomial logit model: An application to accident-injury severities

In this study, two-state Markov switching multinomial logit models are proposed for statistical modeling of accident-injury severities. These models assume Markov switching over time between two unobserved states of roadway safety as a means of accounting for potential unobserved heterogeneity. The states are distinct in the sense that in different states accident-severity outcomes are generated by separate multinomial logit processes. To demonstrate the applicability of the approach, two-state Markov switching multinomial logit models are estimated for severity outcomes of accidents occurring on Indiana roads over a four-year time period. Bayesian inference methods and Markov Chain Monte Carlo (MCMC) simulations are used for model estimation. The estimated Markov switching models result in a superior statistical fit relative to the standard (single-state) multinomial logit models for a number of roadway classes and accident types. It is found that the more frequent state of roadway safety is correlated with better weather conditions and that the less frequent state is correlated with adverse weather conditions.     

Bayesian Detection of Changepoints in Finite-State Markov Chains for Multiple Sequences

We consider the analysis of sets of categorical sequences consisting of piecewise homogenous Markov segments. The sequences are assumed to be governed by a common underlying process with segments occurring in the same order for each sequence. Segments are defined by a set of unobserved changepoints where the positions and number of changepoints can vary from sequence to sequence. We propose a Bayesian framework for analyzing such data, placing priors on the locations of the changepoints and on the transition matrices and using Markov chain Monte Carlo (MCMC) techniques to obtain posterior samples given the data. Experimental results using simulated data illustrate how the methodology can be used for inference of posterior distributions for parameters and changepoints, as well as the ability to handle considerable variability in the locations of the changepoints across different sequences. We also investigate the application of the approach to sequential data from an application involving monsoonal rainfall patterns. Supplementary materials for this article are available online.     

New model diagnostics for spatio-temporal systems in epidemiology and ecology

A cardinal challenge in epidemiological and ecological modelling is to develop effective and easily deployed tools for model assessment. The availability of such methods would greatly improve understanding, prediction and management of disease and ecosystems. Conventional Bayesian model assessment tools such as Bayes factors and the deviance information criterion (DIC) are natural candidates but suffer from important limitations because of their sensitivity and complexity. Posterior predictive checks, which use summary statistics of the observed process simulated from competing models, can provide a measure of model fit but appropriate statistics can be difficult to identify. Here, we develop a novel approach for diagnosing mis-specifications of a general spatio-temporal transmission model by embedding classical ideas within a Bayesian analysis. Specifically, by proposing suitably designed non-centred parametrization schemes, we construct latent residuals whose sampling properties are known given the model specification and which can be used to measure overall fit and to elicit evidence of the nature of mis-specifications of spatial and temporal processes included in the model. This model assessment approach can readily be implemented as an addendum to standard estimation algorithms for sampling from the posterior distributions, for example Markov chain Monte Carlo. The proposed methodology is first tested using simulated data and subsequently applied to data describing the spread of Heracleum mantegazzianum (giant hogweed) across Great Britain over a 30-year period. The proposed methods are compared with alternative techniques including posterior predictive checking and the DIC. Results show that the proposed diagnostic tools are effective in assessing competing stochastic spatio-temporal transmission models and may offer improvements in power to detect model mis-specifications. Moreover, the latent-residual framework introduced here extends readily to a broad range of ecological and epidemiological models.     

Sensitivity estimations for Bayesian inference models solved by MCMC methods

The advent of Markov Chain Monte Carlo (MCMC) methods to simulate posterior distributions has virtually revolutionized the practice of Bayesian statistics. Unfortunately, sensitivity analysis in MCMC methods is a difficult task. In this paper, a computationally low-cost method to estimate local parametric sensitivities in Bayesian models is proposed. The sensitivity measure considered here is the gradient vector of a posterior quantity with respect to the parameter. The gradient vector components are estimated by using a result based on the integral/derivative interchange. The MCMC simulations used to estimate the posterior quantity can be re-used to estimate the sensitivity measures and their errors, avoiding the need for further sampling. The proposed method is easy to apply in practice as it is shown with an illustrative example.     

Reliability Estimation for Accelerated Life Tests Based on a Cox Proportional Hazard Model with Error Effect

In this paper, a Cox proportional hazard model with error effect applied on the study of an accelerated life test is investigated. Statistical inference under Bayesian methods by using the Markov chain Monte Carlo techniques is performed in order to estimate the parameters involved in the model and predict reliability in an accelerated life testing. The proposed model is applied to the analysis of the knock sensor failure time data in which some observations in the data are censored. The failure times at a constant stress level are assumed to be from a Weibull distribution. The analysis of the failure time data from an accelerated life test is used for the posterior estimation of parameters and prediction of the reliability function as well as the comparisons with the classical results from the maximum likelihood estimation. Copyright © 2017 John Wiley & Sons, Ltd.     

Non‐linear model reduction for uncertainty quantification in large‐scale inverse problems

We present a model reduction approach to the solution of large‐scale statistical inverse problems in a Bayesian inference setting. A key to the model reduction is an efficient representation of the non‐linear terms in the reduced model. To achieve this, we present a formulation that employs masked projection of the discrete equations; that is, we compute an approximation of the non‐linear term using a select subset of interpolation points. Further, through this formulation we show similarities among the existing techniques of gappy proper orthogonal decomposition, missing point estimation, and empirical interpolation via coefficient‐function approximation. The resulting model reduction methodology is applied to a highly non‐linear combustion problem governed by an advection–diffusion‐reaction partial differential equation (PDE). Our reduced model is used as a surrogate for a finite element discretization of the non‐linear PDE within the Markov chain Monte Carlo sampling employed by the Bayesian inference approach. In two spatial dimensions, we show that this approach yields accurate results while reducing the computational cost by several orders of magnitude. For the full three‐dimensional problem, a forward solve using a reduced model that has high fidelity over the input parameter space is more than two million times faster than the full‐order finite element model, making tractable the solution of the statistical inverse problem that would otherwise require many years of CPU time. Copyright © 2009 John Wiley & Sons, Ltd.     

Sequential Designs Based on Bayesian Uncertainty Quantification in Sparse Representation Surrogate Modeling

A numerical method, called overcomplete basis surrogate method (OBSM), was recently proposed, which employs overcomplete basis functions to achieve sparse representations. While the method can handle nonstationary response without the need of inverting large covariance matrices, it lacks the capability to quantify uncertainty in predictions. We address this issue by proposing a Bayesian approach that first imposes a normal prior on the large space of linear coefficients, then applies the Markov chain Monte Carlo (MCMC) algorithm to generate posterior samples for predictions. From these samples, Bayesian credible intervals can then be obtained to assess prediction uncertainty. A key application for the proposed method is the efficient construction of sequential designs. Several sequential design procedures with different infill criteria are proposed based on the generated posterior samples. Numerical studies show that the proposed schemes are capable of solving problems of positive point identification, optimization, and surrogate fitting.