Validation of reliability computational models using Bayes networks

This paper proposes a methodology based on Bayesian statistics to assess the validity of reliability computational models when full-scale testing is not possible. Sub-module validation results are used to derive a validation measure for the overall reliability estimate. Bayes networks are used for the propagation and updating of validation information from the sub-modules to the overall model prediction. The methodology includes uncertainty in the experimental measurement, and the posterior and prior distributions of the model output are used to compute a validation metric based on Bayesian hypothesis testing. Validation of a reliability prediction model for an engine blade under high-cycle fatigue is illustrated using the proposed methodology.     

Bayesian risk-based decision method for model validation under uncertainty

This paper develops a decision-making methodology for computational model validation, considering the risk of using the current model, data support for the current model, and cost of acquiring new information to improve the model. A Bayesian decision theory-based method is developed for this purpose, using a likelihood ratio as the validation metric for model assessment. An expected risk or cost function is defined as a function of the decision costs, and the likelihood and prior of each hypothesis. The risk is minimized through correctly assigning experimental data to two decision regions based on the comparison of the likelihood ratio with a decision threshold. A Bayesian validation metric is derived based on the risk minimization criterion. Two types of validation tests are considered: pass/fail tests and system response value measurement tests. The methodology is illustrated for the validation of reliability prediction models in a tension bar and an engine blade subjected to high cycle fatigue. The proposed method can effectively integrate optimal experimental design into model validation to simultaneously reduce the cost and improve the accuracy of reliability model assessment.     

Uncertainty quantification and model validation of fatigue crack growth prediction

This paper presents a methodology for uncertainty quantification and model validation in fatigue crack growth analysis. Several models – finite element model, crack growth model, surrogate model, etc. – are connected through a Bayes network that aids in model calibration, uncertainty quantification, and model validation. Three types of uncertainty are included in both uncertainty quantification and model validation: (1) natural variability in loading and material properties; (2) data uncertainty due to measurement errors, sparse data, and different inspection results (crack not detected, crack detected but size not measured, and crack detected with size measurement); and (3) modeling uncertainty and errors during crack growth analysis, numerical approximations, and finite element discretization. Global sensitivity analysis is used to quantify the contribution of each source of uncertainty to the overall prediction uncertainty and to identify the important parameters that need to be calibrated. Bayesian hypothesis testing is used for model validation and the Bayes factor metric is used to quantify the confidence in the model prediction. The proposed methodology is illustrated using a numerical example of surface cracking in a cylindrical component.     

Bayesian structural equation modeling method for hierarchical model validation

A building block approach to model validation may proceed through various levels, such as material to component to subsystem to system, comparing model predictions with experimental observations at each level. Usually, experimental data becomes scarce as one proceeds from lower to higher levels. This paper presents a structural equation modeling approach to make use of the lower-level data for higher-level model validation under uncertainty, integrating several components: lower-level data, higher-level data, computational model, and latent variables. The method proposed in this paper uses latent variables to model two sets of relationships, namely, the computational model to system-level data, and lower-level data to system-level data. A Bayesian network with Markov chain Monte Carlo simulation is applied to represent the two relationships and to estimate the influencing factors between them. Bayesian hypothesis testing is employed to quantify the confidence in the predictive model at the system level, and the role of lower-level data in the model validation assessment at the system level. The proposed methodology is implemented for hierarchical assessment of three validation problems, using discrete observations and time-series data.     

Bayesian Nonmetric Successive Categories Multidimensional Scaling

A Bayesian nonmetric successive categories multidimensional scaling (MDS) method is proposed. The proposed method can be seen as a Bayesian alternative to the maximum likelihood multidimensional successive scaling method proposed by Takane (1981), or as a nonmetric extension of Bayesian metric MDS by Oh and Raftery (2001). The model has a graded-response type measurement model part and a latent metric MDS part. All the parameters are jointly estimated using a Markov chain Monte Carlo (MCMC) estimation technique. Moreover, WinBUGS/OpenBUGS code for the proposed methodology is also given to aid applied researchers. The proposed method is illustrated through the analysis of empirical two-mode three-way similarity data.

     

Validation of models with multivariate output

This paper develops metrics for validating computational models with experimental data, considering uncertainties in both. A computational model may generate multiple response quantities and the validation experiment might yield corresponding measured values. Alternatively, a single response quantity may be predicted and observed at different spatial and temporal points. Model validation in such cases involves comparison of multiple correlated quantities. Multiple univariate comparisons may give conflicting inferences. Therefore, aggregate validation metrics are developed in this paper. Both classical and Bayesian hypothesis testing are investigated for this purpose, using multivariate analysis. Since, commonly used statistical significance tests are based on normality assumptions, appropriate transformations are investigated in the case of non-normal data. The methodology is implemented to validate an empirical model for energy dissipation in lap joints under dynamic loading.     

Model validation under epistemic uncertainty

This paper develops a methodology to assess the validity of computational models when some quantities may be affected by epistemic uncertainty. Three types of epistemic uncertainty regarding input random variables - interval data, sparse point data, and probability distributions with parameter uncertainty - are considered. When the model inputs are described using sparse point data and/or interval data, a likelihood-based methodology is used to represent these variables as probability distributions. Two approaches - a parametric approach and a non-parametric approach - are pursued for this purpose. While the parametric approach leads to a family of distributions due to distribution parameter uncertainty, the principles of conditional probability and total probability can be used to integrate the family of distributions into a single distribution. The non-parametric approach directly yields a single probability distribution. The probabilistic model predictions are compared against experimental observations, which may again be point data or interval data. A generalized likelihood function is constructed for Bayesian updating, and the posterior distribution of the model output is estimated. The Bayes factor metric is extended to assess the validity of the model under both aleatory and epistemic uncertainty and to estimate the confidence in the model prediction. The proposed method is illustrated using a numerical example.     

Calibration of adhesion models based on Bayesian inference

The use of composite materials in a myriad of applications fostered the development of reliable procedures to connect components with adhesives. This led to a demand for reliable adhesion models to be used in engineering designs that are based on computer simulations. This paper presents a strategy to be used for calibration of adhesion models. The proposed methodology is built on the formalism of Statistical Inverse Problems. Uncertainties about the unknowns are inferred using Population-Based Markov Chain Monte Carlo and Adaptive Metropolis. It is proposed to perform model assessments based on the analysis of a validation metric. Realizations of the validation metric are computed with the posterior densities of model parameters that are provided by the calibration process. The analysis of the validation metric allows for model selection to be performed. Some numerical experiments are presented with noise-contaminated data. The calibration strategy proved effective when dealing with both the nonlinearity and nondifferentiability of the adhesion constitutive equation.     

Challenges to Bayesian decision support using morphological matrices for design: empirical evidence

A novel Bayesian design support tool is empirically investigated for its potential to support the early design stages. The design support tool provides dynamic guidance with the use of morphological design matrices during the conceptual or preliminary design stages. This paper tests the appropriateness of adopting a stochastic approach for supporting the early design phase. The rationale for the stochastic approach is based on the uncertain nature of the design during this part of the design process. The support tool is based on Bayesian belief networks (BBNs) and uses a simple but effective information content–based metric to learn or induce the model structure. The dynamically interactive tool is assessed with two empirical trials. First, the laboratory-based trial with novice designers illustrates a novel emergent design search methodology. Second, the industrial-based trial with expert designers illustrates the hurdles that are faced when deploying a design support tool in a highly pressurised industrial environment. The conclusion from these trials is that there is a need for designers to better understand the stochastic methodology for them to both be able to interpret and trust the BBN model of the design domain. Further, there is a need for a lightweight domain-specific front end interface is needed to enable a better fit between the generic support tool and the domain-specific design process and associated tools.     

Advanced Scientific Computing in BASIC With Applications in Chemistry,Biology, and Pharmacology

Tuning and calibration are processes for improving the representativeness of a computer simulation code to a physical phenomenon. This article introduces a statistical methodology for simultaneously determining tuning and calibration parameters in settings where data are available from a computer code and the associated physical experiment. Tuning parameters are set by minimizing a discrepancy measure while the distribution of the calibration parameters are determined based on a hierarchical Bayesian model. The proposed Bayesian model views the output as a realization of a Gaussian stochastic process with hyper-priors. Draws from the resulting posterior distribution are obtained by the Markov chain Monte Carlo simulation. Our methodology is compared with an alternative approach in examples and is illustrated in a biomechanical engineering application. Supplemental materials, including the software and a user manual, are available online and can be requested from the first author.     

Real-time inference of stochastic damage in composite materials

This study describes a control system designed for real-time monitoring of damage in materials that employs methods and models that account for uncertainties in experimental data and parameters in continuum damage mechanics models. The methodology involves (1) developing an experimental set-up for direct and indirect measurements of damage in materials; (2) modeling damage mechanics based constitutive equations for continuum models; and (3) implementation of a Bayesian framework for statistical calibration of model with quantification of uncertainties. To provide information for real-time monitoring of damage, indirect measurement of damage is made feasible using an embedded carbon nanotube (CNT) network to perform as sensor for detecting the local damage. A software infrastructure is developed and implemented in order to integrate the various constituents, such as finite element approximation of the continuum damage models, generated experimental data, and Bayesian-based methods for model calibration and validation. The outcomes of the statistical calibration and dynamic validation of damage models are presented. The experimental program designed to provide observational data is discussed.     

Computational methods for model reliability assessment

This paper investigates various statistical approaches for the validation of computational models when both model prediction and experimental observation have uncertainties, and proposes two new methods for this purpose. The first method utilizes hypothesis testing to accept or reject a model at a desired significance level. Interval-based hypothesis testing is found to be more practically useful for model validation than the commonly used point null hypothesis testing. Both classical and Bayesian approaches are investigated. The second and more direct method formulates model validation as a limit state-based reliability estimation problem. Both simulation-based and analytical methods are presented to compute the model reliability for single or multiple comparisons of the model output and observed data. The proposed methods are illustrated and compared using numerical examples.     

Identification of stochastic loads applied to a non-linear dynamical system using an uncertain computational model and experimental responses

The paper is devoted to the identification of stochastic loads applied to a non-linear dynamical system for which experimental dynamical responses are available. The identification of the stochastic load is performed using a simplified computational non-linear dynamical model containing both model uncertainties and data uncertainties. Uncertainties are taken into account in the context of the probability theory. The stochastic load which has to be identified is modelled by a stationary non-Gaussian stochastic process for which the matrix-valued spectral density function is uncertain and is then modelled by a matrix-valued random function. The parameters to be identified are the mean value of the random matrix-valued spectral density function and its dispersion parameter. The identification problem is formulated as two optimization problems using the computational stochastic model and experimental responses. A validation of the theory proposed is presented in the context of tubes bundles in Pressurized Water Reactors.     

Generalized probabilistic approach of uncertainties in computational dynamics using random matrices and polynomial chaos decompositions

A new generalized probabilistic approach of uncertainties is proposed for computational model in structural linear dynamics and can be extended without difficulty to computational linear vibroacoustics and to computational non‐linear structural dynamics. This method allows the prior probability model of each type of uncertainties (model‐parameter uncertainties and modeling errors) to be separately constructed and identified. The modeling errors are not taken into account with the usual output‐prediction‐error method, but with the nonparametric probabilistic approach of modeling errors recently introduced and based on the use of the random matrix theory. The theory, an identification procedure and a numerical validation are presented. Then a chaos decomposition with random coefficients is proposed to represent the prior probabilistic model of random responses. The random germ is related to the prior probability model of model‐parameter uncertainties. The random coefficients are related to the prior probability model of modeling errors and then depends on the random matrices introduced by the nonparametric probabilistic approach of modeling errors. A validation is presented. Finally, a future perspective is introduced when experimental data are available. The prior probability model of the random coefficients can be improved in constructing a posterior probability model using the Bayesian approach. Copyright © 2009 John Wiley & Sons, Ltd.     

Probabilistic modeling of a nonlinear dynamical system used for producing voice

This paper is devoted to the construction of a stochastic nonlinear dynamical system for signal generation such as the production of voiced sounds. The dynamical system is highly nonlinear, and the output signal generated is very sensitive to a few parameters of the system. In the context of the production of voiced sounds the measurements have a significant variability. We then propose a statistical treatment of the experiments and we developed a probability model of the sensitive parameters in order that the stochastic dynamical system has the capability to predict the experiments in the probability distribution sense. The computational nonlinear dynamical system is presented, the Maximum Entropy Principle is used to construct the probability model and an experimental validation is shown.     

Human Reliability Analysis of Roof Bolting Operation in Underground Coal Mines

Investigations have shown that human error is the most common cause of roof bolting injuries. Human error probability estimation has become a critical issue for human reliability analysis (HRA) of roof bolting operation. Specialist judgment plays a crucial role in quantifying human error probability in the field because of limited availability of empirical data. However, the aggregation of specialist judgment is typically not carried out in a formal way in HRA. In this paper, an approach to combine Bayesian methodology and the success likelihood index method was to build a computable model using information from specialists for HRA of roof bolting operation. A numerical example was used to illustrate the application of the proposed methodology. Copyright © 2015 John Wiley & Sons, Ltd.     

Analysis of one-dimensional stochastic finite elements using neural networks

This paper explores the applicability of neural networks for analyzing the uncertainty spread of structural responses under the presence of one-dimensional random fields. Specifically, the neural network is intended to be a partial surrogate of the structural model needed in a Monte Carlo simulation, due to its associative memory properties. The network is trained with some pairs of input and output data obtained by some Monte Carlo simulations and then used in substitution of the finite element solver. In order to minimize the size of the networks, and hence the number of training pairs, the Karhunen–Loéve decomposition is applied as an optimal feature extraction tool. The Monte Carlo samples for training and validation are also generated using this decomposition. The Nyström technique is employed for the numerical solution of the Fredholm integral equation. The radial basis function (RBF) network was selected as the neural device for learning the input/output relationship due to its high accuracy and fast training speed. The analysis shows that this approach constitutes a promising method for stochastic finite element analysis inasmuch as the error with respect to the Monte Carlo simulation is negligible.     

Robust Empirical Predictions of Residual Performance of Damaged Composites with Quantified Uncertainties

Robust predictions with estimated uncertainties were made for the residual strength of impact-damaged composite laminates based on simple non-destructive measurements of the size of the damage from ultrasound C-scans. Experimental data was acquired for two sets of composite coupons, one with a crossply and the other with a quasi-isotropic layup. The laminates were subject to drop-weight impacts, non-destructively evaluated using ultrasound and then loaded to failure in bending. An empirical model of the residual strength of each laminate layup, as a function of the ultrasound measurements, was generated by fitting a Bayesian linear regression model to the normalised measured data. Bayesian linear regression was demonstrated to provide conservative estimates when only minimal data is available. Unlike classical regression, this technique provides a robust treatment of outliers, which avoids underestimation of residual strength. The Leave-One-Out-Cross-Validation (LOOCV) metric was used to assess the performance of models allowing for the quantitative comparison of the predictive power of regression models as well as being consistent in the presence of outliers in the data. The LOOCV metric indicated that predictions of residual strength are up to 25% more accurate when based on damage area than when using measurements of the damage width or length. The proposed approach provides a robust methodology for performing damage assessments in safety critical composite components based on reliable predictions with quantified uncertainties.     

Bayesian uncertainty analysis with applications to turbulence modeling

In this paper, we apply Bayesian uncertainty quantification techniques to the processes of calibrating complex mathematical models and predicting quantities of interest (QoI's) with such models. These techniques also enable the systematic comparison of competing model classes. The processes of calibration and comparison constitute the building blocks of a larger validation process, the goal of which is to accept or reject a given mathematical model for the prediction of a particular QoI for a particular scenario. In this work, we take the first step in this process by applying the methodology to the analysis of the Spalart-Allmaras turbulence model in the context of incompressible, boundary layer flows. Three competing model classes based on the Spalart-Allmaras model are formulated, calibrated against experimental data, and used to issue a prediction with quantified uncertainty. The model classes are compared in terms of their posterior probabilities and their prediction of QoI's. The model posterior probability represents the relative plausibility of a model class given the data. Thus, it incorporates the model's ability to fit experimental observations. Alternatively, comparing models using the predicted QoI connects the process to the needs of decision makers that use the results of the model. We show that by using both the model plausibility and predicted QoI, one has the opportunity to reject some model classes after calibration, before subjecting the remaining classes to additional validation challenges.     

System identification and uncertainty modeling of an electromechanical actuator

The electromechanical actuator (EMA) is of interest for applications which require easy control and high dynamics. This article deals with the analytical and uncertainties modeling, experimental identification, and model validation of a position control EMA and its “motor with harmonic drive” subsystem. They are identified and modeled as linear systems with parametric uncertainty using their experimental input–output data. The captured data, related to several working points under different conditions, were used for model estimation and validation purposes. Furthermore, the discrepancies between the linear model and the actual system, due to nonlinearities, were estimated as multiplicative uncertainty. The model is validated on the base of reproducing system behavior within acceptable bounds with minimum error. Validation results proved the derived model viability to be used for designing a robust EMA system to perform within acceptable bandwidth.