Understanding and comparisons of different sampling approaches for the Fourier Amplitudes Sensitivity Test (FAST)

Fourier Amplitude Sensitivity Test (FAST) is one of the most popular uncertainty and sensitivity analysis techniques. It uses a periodic sampling approach and a Fourier transformation to decompose the variance of a model output into partial variances contributed by different model parameters. Until now, the FAST analysis is mainly confined to the estimation of partial variances contributed by the main effects of model parameters, but does not allow for those contributed by specific interactions among parameters. In this paper, we theoretically show that FAST analysis can be used to estimate partial variances contributed by both main effects and interaction effects of model parameters using different sampling approaches (i.e., traditional search-curve based sampling, simple random sampling and random balance design sampling). We also analytically calculate the potential errors and biases in the estimation of partial variances. Hypothesis tests are constructed to reduce the effect of sampling errors on the estimation of partial variances. Our results show that compared to simple random sampling and random balance design sampling, sensitivity indices (ratios of partial variances to variance of a specific model output) estimated by search-curve based sampling generally have higher precision but larger underestimations. Compared to simple random sampling, random balance design sampling generally provides higher estimation precision for partial variances contributed by the main effects of parameters. The theoretical derivation of partial variances contributed by higher-order interactions and the calculation of their corresponding estimation errors in different sampling schemes can help us better understand the FAST method and provide a fundamental basis for FAST applications and further improvements.     

Parametric sensitivity: A case study comparison

This article presents a comparative analysis of three derivative-based parametric sensitivity approaches in multi-response regression estimation: marginal sensitivity, profile-based approach developed by [Sulieman, H., McLellan, P.J., Bacon, D.W., 2004, A Profile-based approach to parametric sensitivity in multiresponse regression models, Computational Statistics & Data Analysis, 45, 721-740] and the commonly used approach of the Fourier Amplitude Sensitivity Test (FAST). We apply the classical formulation of FAST in which Fourier sine coefficients are utilized as sensitivity measures. Contrary to marginal sensitivity, profile-based and FAST approaches provide sensitivity measures that account for model nonlinearity and are pertinent to linear and nonlinear regression models. However, the primary difference between FAST and profile-based sensitivity is that traditional FAST fails to account for parameter dependencies in the model system while these dependencies are considered in the analysis procedure of profile-based sensitivity through the re-estimation of the remaining model parameters conditional on the values of the parameter of interest. An example is discussed to illustrate the comparisons by applying the three sensitivity methods to a model described by set of non-linear differential equations. Some computational aspects are also explored.     

Global sensitivity analysis of outputs over rice-growth process in ORYZA model

Dynamic crop models usually have a complex structure and a large number of parameters. Those parameter values usually cannot be directly measured, and they vary with crop cultivars, environmental conditions and managements. Thus, parameter estimation and model calibration are always difficult issues for crop models. Therefore, the quantification of parameter sensitivity and the identification of influential parameters are very important and useful. In this work, late-season rice was simulated with meteorological data in Nanchang, China. Furthermore, we conducted a sensitivity analysis of 20 selected parameters in ORYZA_V3 using the Extended FAST method. We presented the sensitivity results for four model outputs (LAI, WAGT, WST and WSO) at four development stages and the results for yield. Meanwhile, we compared the differences among the sensitivity results for the model outputs simulated in cold, normal and hot years. The uncertainty of output variables derived from parameter variation and weather conditions were also quantified. We found that the development rates, RGRLMN and FLV0.5 had strong effects on all model outputs in all conditions, and parameters WGRMX and SPGF had relative high effects on yield in cold year. Only LAI was sensitive to ASLA. Those influential parameters had unequal effects on different outputs, and they had different effects at four development stages. With the interaction effects of parameter variation and different weather conditions, the uncertainty of model outputs varied significantly. However, the weather conditions had negligible effects on the identification of influential parameters, although they had slight effects on the ranks of the parameters' sensitivity for outputs in the panicle-formation phase and the grain-filling phase, including yield at maturity. The results suggested that the influential parameters should be recalibrated in priority and fine-tuned with higher accuracy during model calibration.     

A General Probabilistic Framework for uncertainty and global sensitivity analysis of deterministic models: A hydrological case study

The present study proposes a General Probabilistic Framework (GPF) for uncertainty and global sensitivity analysis of deterministic models in which, in addition to scalar inputs, non-scalar and correlated inputs can be considered as well. The analysis is conducted with the variance-based approach of Sobol/Saltelli where first and total sensitivity indices are estimated. The results of the framework can be used in a loop for model improvement, parameter estimation or model simplification. The framework is applied to SWAP, a 1D hydrological model for the transport of water, solutes and heat in unsaturated and saturated soils. The sources of uncertainty are grouped in five main classes: model structure (soil discretization), input (weather data), time-varying (crop) parameters, scalar parameters (soil properties) and observations (measured soil moisture). For each source of uncertainty, different realizations are created based on direct monitoring activities. Uncertainty of evapotranspiration, soil moisture in the root zone and bottom fluxes below the root zone are considered in the analysis. The results show that the sources of uncertainty are different for each output considered and it is necessary to consider multiple output variables for a proper assessment of the model. Improvements on the performance of the model can be achieved reducing the uncertainty in the observations, in the soil parameters and in the weather data. Overall, the study shows the capability of the GPF to quantify the relative contribution of the different sources of uncertainty and to identify the priorities required to improve the performance of the model. The proposed framework can be extended to a wide variety of modelling applications, also when direct measurements of model output are not available.     

Robust global sensitivity analysis under deep uncertainty via scenario analysis

Complex social-ecological systems models typically need to consider deeply uncertain long run future conditions. The influence of this deep (i.e. incalculable, uncontrollable) uncertainty on model parameter sensitivities needs to be understood and robustly quantified to reliably inform investment in data collection and model refinement. Using a variance-based global sensitivity analysis method (eFAST), we produced comprehensive model diagnostics of a complex social-ecological systems model under deep uncertainty characterised by four global change scenarios. The uncertainty of the outputs, and the influence of input parameters differed substantially between scenarios. We then developed sensitivity indicators that were robust to this deep uncertainty using four criteria from decision theory. The proposed methods can increase our understanding of the effects of deep uncertainty on output uncertainty and parameter sensitivity, and incorporate the decision maker's risk preference into modelling-related activities to obtain greater resilience of decisions to surprise.     

Variance-based sensitivity analysis for models with correlated inputs and its state dependent parameter solution

Sensitivity analysis is indispensable to structural design and optimization. This paper focuses on sensitivity analysis for models with correlated inputs. To explore the contributions of correlated inputs to the uncertainty in a model output, the universal expressions of the variance contributions of the correlated inputs are first derived in the paper based on the high dimensional model representation (HDMR) of the model function. Then by analyzing the composition of these variance contributions, the variance contributions by an individual correlated input to the model output are further decomposed into independent contribution by the individual input itself, independent contribution by interaction between the individual input and the others, contribution purely by correlation between the individual input and the others, and contribution by interaction associated with correlation between the individual input and the others. The general expressions of these components are also derived. Based on the characteristics of these general expressions, a universal framework for estimating the various variance contributions of the correlated inputs is developed by taking the efficient state dependent parameter (SDP) method as an illustration. Numerical and engineering tests show that this decomposition of the variance contributions of the correlated inputs can provide useful information for exploring the sources of the output uncertainty and identifying the structure of the model function for the complicated models with correlated inputs. The efficiency and accuracy of the SDP-based method for estimating the various variance contributions of the correlated inputs are also demonstrated by the examples.     

Assessment of the different sources of uncertainty in a SWAT model of the River Senne (Belgium)

Although rainfall input uncertainties are widely identified as being a key factor in hydrological models, the rainfall uncertainty is typically not included in the parameter identification and model output uncertainty analysis of complex distributed models such as SWAT and in maritime climate zones. This paper presents a methodology to assess the uncertainty of semi-distributed hydrological models by including, in addition to a list of model parameters, additional unknown factors in the calibration algorithm to account for the rainfall uncertainty (using multiplication factors for each separately identified rainfall event) and for the heteroscedastic nature of the errors of the stream flow. We used the Differential Evolution Adaptive Metropolis algorithm (DREAM_(zs)) to infer the parameter posterior distributions and the output uncertainties of a SWAT model of the River Senne (Belgium). Explicitly considering heteroscedasticity and rainfall uncertainty leads to more realistic parameter values, better representation of water balance components and prediction uncertainty intervals.     

A new approach to visualizing time-varying sensitivity indices for environmental model diagnostics across evaluation time-scales

Assessing the time-varying sensitivity of environmental models has become a common approach to understand both the value of different data periods for estimating specific parameters, and as part of a diagnostic analysis of the model structure itself (i.e. whether dominant processes are emerging in the model at the right times and over the appropriate time periods). It is not straightforward to visualize these results though, given that the window size over which the time-varying sensitivity is best integrated generally varies for different parameters. In this short communication we present a new approach to visualizing such time-varying sensitivity across time scales of integration. As a case study, we estimate first order sensitivity indices with the FAST (Fourier Amplitude Sensitivity Test) method for a typical conceptual rainfall–runoff model. The resulting plots can guide data selection for model calibration, support diagnostic model evaluation and help to define the timing and length of spot gauging campaigns in places where long-term calibration data are not yet available.     

Global sensitivity analysis measures the quality of parameter estimation: The case of soil parameters and a crop model

One common limitation of the use of crop models for decision making in precise crop management is the need for accurate values of soil parameters for a whole field. Estimating these parameters from data observed on the crop, using a crop model, is an interesting possibility. Nevertheless, the quality of the estimation depends on the sensitivity of model output variables to the parameters. The goal of this study is to explain the results for the quality of parameter estimation based on global sensitivity analysis (GSA). The case study consists of estimating the soil parameters by using the STICS-wheat crop model and various synthetic observations on wheat crops (LAI, absorbed nitrogen and grain yield). Suitable criteria summarizing the sensitivity indices of the observed variables were created in order to link GSA indices with the quality of parameter estimation. We illustrate this link on 16 different configurations of different soil, climatic and crop conditions. The GSA indices were computed by the Extended FAST method and a function of RMSE was computed with an importance sampling method based on Bayes theory (GLUE). The proposed GSA-based criteria are able to rank the parameters with respect to their quality of estimation and the different configurations (especially climate and observation set) with respect to their ability to estimate the whole parameter set. They may be used as a tool for predicting the performance of different observation datasets with regard to parameter estimation.     

基于EFAST方法的SEBS模型参数全局敏感性分析

应用高效、稳健的EFAST方法,以黑河流域盈科绿洲站为例,从3个方面对SEBS模型的参数敏感性进行了分析:分别以感热通量(H)、潜热通量(λE)、蒸发比(fr)作为SEBS模型的输出结果,分析其对12个输入参数的敏感性;利用气象数据驱动模型,分析H、λE和fr对6个地表特征参数的敏感性;分析了参数取值范围对敏感性分析结果的影响。研究结果表明:H、λE与fr都对参考高度处的气温和风速、地表温度以及植被特征参数的敏感性较高。参数间相互作用对H、λE的间接影响很小,而对fr的影响较大。当气象输入参数确定时,6个地表参数中地表温度对模型输出的直接贡献最大,其主敏感度指数接近0.6。参数采样范围不同时,模型输入参数的敏感性表现不同。     

Metamodels for variable importance decomposition with applications to probabilistic engineering design

It is routine in probabilistic engineering design to conduct modeling studies to determine the influence of an input variable (or a combination) on the output variable(s). The output or the response can then be fine-tuned by changing the design parameters based on this information. However, simply fine-tuning the output to the desired or target value is not adequate. Robust design principles suggest that we not only study the mean response for a given input vector but also the variance in the output attributed to noise and other unaccounted factors. Given our desire to reduce variability in any process, it is also important to understand which of the input factors affect the variability in the output the most. Given the significant computational overhead associated with most Computer Aided Engineering models, it is becoming popular to conduct such analysis through surrogate models built using a variety of metamodeling techniques. In this regard, existing literature on metamodeling and sensitivity analysis techniques provides useful insights into the various scenarios that they suit the best. However, there has been a limitation of studies that simultaneously consider the combination of metamodeling and sensitivity analysis and the environments in which they operate the best. This paper aims at contributing to reduce this limitation by basing the study on multiple metrics and using two test problems. Two test functions have been used to build metamodels, using three popular metamodeling techniques: Kriging, Radial-Basis Function (RBF) networks, and Support Vector Machines (SVMs). The metamodels are then used for sensitivity analysis, using two popular sensitivity analysis methods, Fourier Amplitude Sensitivity Test (FAST) and Sobol, to determine the influence of variance in the input variables on the variance of the output variables. The advantages and disadvantages of the different metamodeling techniques, in combination with the sensitivity analysis methods, in determining the extent to which the variabilities in the input affect the variabilities in the output are analyzed.     

A parameter estimation and identifiability analysis methodology applied to a street canyon air pollution model

Mathematical models are increasingly used in environmental science thus increasing the importance of uncertainty and sensitivity analyses. In the present study, an iterative parameter estimation and identifiability analysis methodology is applied to an atmospheric model – the Operational Street Pollution Model (OSPM^®). To assess the predictive validity of the model, the data is split into an estimation and a prediction data set using two data splitting approaches and data preparation techniques (clustering and outlier detection) are analysed. The sensitivity analysis, being part of the identifiability analysis, showed that some model parameters were significantly more sensitive than others. The application of the determined optimal parameter values was shown to successfully equilibrate the model biases among the individual streets and species. It was as well shown that the frequentist approach applied for the uncertainty calculations underestimated the parameter uncertainties. The model parameter uncertainty was qualitatively assessed to be significant, and reduction strategies were identified.     

EFAST Method for Global Sensitivity Analysisof Remote Sensing Model’s Parameters

Sensitivity analysis for parameters of remote sensing physical models is a prerequisite for inversion.The EFAST(Extended Fourier Amplitude Sensitivity Test)as a global sensitivity analysis method,can analyze not only a single parameter’s sensitivity but also the coupling effects among parameters.It is usually applied to analyse parameters’ sensitivity of the high-dimensional nonlinear models.In this paper,the SAIL model is taken as an example,the EFAST method and the field measured data of winter wheat in Shunyi district in 2001 were applied to analyze the model parameters’ sensitivity throughout the growing season and in different growth stages respectively.The results are compared with those of the USM (Uncertainty and Sensitivity Matrix) method.The results show that either the EFAST or the USM method for parameters’ sensitivity analysis of the SAIL model is feasible;but the EFAST method,which takes into account of the coupling effects among all the parameters and the analysis result is global,compared to the USM method,is more objective and comprehensive.     

利用模糊系统的直升机的自适应控制

针对直升机动力学为非线性,且存在不确定因素和状态变化,设计利用模糊系统的自适应控制器.设计的控制器是系统的输出跟踪参考模型输出的直接调整模糊控制器参数的自适应控制器.又利用Lyapunov函数保证了闭环控制系统的稳定性并推导最优的自适应规律.实验结果表明,有外部扰动的情况下所设计的自适应控制器比模糊控制器对直升机控制具有良好的动态响应和稳定性,是一种非常有效的控制方法.     

Global sensitivity analysis of yield output from the water productivity model

This study includes a global sensitivity analysis of the water productivity model AquaCrop. The study rationale consisted in a comprehensive evaluation of the model and the formulation of guidelines for model simplification and efficient calibration. The global analysis comprehended a Morris screening followed by a variance-based Extended Fourier Amplitude Sensitivity Test (EFAST) under diverse environmental conditions for maize, winter wheat and rice. The analysis involved twenty-two different climate-crop-soil-meteorology combinations. The main objectives were to distinguish the model's influential and non-influential parameters, and to examine the yield output sensitivity. For the AquaCrop model, a number of non-influential parameters could be identified. Making these parameters fixed would be a step towards model simplification. Also, a list of influential parameters was identified. Despite the dependence of parameter ranking on environmental conditions, guiding principles for priority parameters were formulated for calibration in diverse conditions, valuable to model users. For this model that focuses on modelling yield response to water, parameters describing crop responses to water stress were not often among those showing highest sensitivity. Instead, particular root and soil parameters, relevant in the determination of water availability, were influential under various conditions and merit attention during calibration. The considerations made in this study about sensitivity analysis method (Morris vs. EFAST), prior parameter ranges, target functions and ranking variation according to environmental conditions can be extrapolated to other conditions and models, if done with the necessary precaution.     

Uncertainty Analysis in Software Reliability Modeling by Bayesian Analysis with Maximum-Entropy Principle

In software reliability modeling, the parameters of the model are typically estimated from the test data of the corresponding component. However, the widely used point estimators are subject to random variations in the data, resulting in uncertainties in these estimated parameters. Ignoring the parameter uncertainty can result in grossly underestimating the uncertainty in the total system reliability. This paper attempts to study and quantify the uncertainties in the software reliability modeling of a single component with correlated parameters and in a large system with numerous components. Another characteristic challenge in software testing and reliability is the lack of available failure data from a single test, which often makes modeling difficult. This lack of data poses a bigger challenge in the uncertainty analysis of the software reliability modeling. To overcome this challenge, this paper proposes utilizing experts' opinions and historical data from previous projects to complement the small number of observations to quantify the uncertainties. This is done by combining the maximum-entropy principle (MEP) into the Bayesian approach. This paper further considers the uncertainty analysis at the system level, which contains multiple components, each with its respective model/parameter/ uncertainty, by using a Monte Carlo approach. Some examples with different modeling approaches (NHPP, Markov, Graph theory) are illustrated to show the generality and effectiveness of the proposed approach. Furthermore, we illustrate how the proposed approach for considering the uncertainties in various components improves a large-scale system reliability model.     

Design techniques of a computer program for the identification of processes and the simulation of water quality in aquatic systems

In order to support environmental scientists in finding an “adequate” model for the system they are investigating, a computer program is necessary that allows its users to perform simulations for different models, to assess the identifiability and to estimate the values of model parameters (using measured data), and to estimate prediction uncertainty. These requirements, especially that of providing much freedom in model formulation, are difficult to realize in such a program. In this paper, it is shown how object-oriented program design techniques were employed to facilitate the realization of an identification and simulation program for aquatic systems (AQUASIM) that is very flexible with regard to model formulation and that provides methods of sensitivity analysis, parameter estimation and uncertainty analysis in addition to simulation. It is the goal of this paper to encourage developers of environmental software to revise previously used program structures and to employ modern program design techniques.     

Parameter sensitivity analysis of crop growth models based on the extended Fourier Amplitude Sensitivity Test method

Sensitivity analysis (SA) has become a basic tool for the understanding, application and development of models. However, in the past, little attention has been paid to the effects of the parameter sample size and parameter variation range on the parameter SA and its temporal properties. In this paper, the corn crop planted in 2008 in the Yingke Oasis of northwest China is simulated based on meteorological observation data for the inputs and statistical data for the parameters. Furthermore, using the extended Fourier Amplitude Sensitivity (EFAST) algorithm, SA is performed on the 47 crop parameters of the WOrld FOod STudies (WOFOST) crop growth models. A deep analysis is conducted, including the effects of the parameter sample size and variation range on the parameter SA, the temporal properties and the multivariable output issues of SA. The results show that sample size highly affects the convergence of the sensitivity indices. Two types of parameter variation ranges are used for the analysis, and the results show that the sensitive parameters of the two parameter spaces are distinctly different. In addition, taking the storage organ biomasses at the different growth stages as the objective output, the time-dependent characteristics of the parameter sensitivity are discussed. The results show that several sensitive parameters exist in the grain biomass throughout the entire development stage. In addition, analyzing the twelve sensitive parameters has proven that although certain parameters have no effect on the final yield, they play key roles in certain growth stages, and the importance of these parameters gradually increases. Finally, the sensitivity analyses of different state variable outputs are performed, including the biomass, yield, leaf area index, and transpiration coefficient. The results suggest that the sensitive parameters of various variable processes differ. This study highlights the importance of considering multiple characteristics of the model parameters and the responses of the models in specific phenological stages.     

Sensitivity analysis of microbial growth parameter distributions with respect to data quality and quantity by using Monte Carlo analysis

Nowadays, most of the mathematical models used in predictive microbiology are deterministic, i.e. their model output is only one single value for the microbial load at a certain time instant. For more advanced exploitation of predictive microbiology in the context of hazard analysis and critical control points (HACCP) and risk analysis studies, stochastic models should be developed. Such models predict a probability mass function for the microbial load at a certain time instant. An excellent method to deal with stochastic variables is Monte Carlo analysis. In this research, the sensitivity of microbial growth model parameter distributions with respect to data quality and quantity is investigated using Monte Carlo analysis. The proposed approach is illustrated with experimental growth data. There appears to be a linear relation between data quality (expressed by means of the standard deviation of the normal distribution assumed on experimental data) and model parameter uncertainty (expressed by means of the standard deviation of the model parameter distribution). The quantity of data (expressed by means of the number of experimental data points) as well as the positioning of these data in time have a substantial influence on model parameter uncertainty. This has implications for optimal experiment design.