Wavelet ANOVA approach to model validation

Model validation is that critical component in the simulation development process that ensures a model is truly representative of the system that it is meant to model. Although there are numerous validation techniques described in the literature, many of these techniques still require some amount of subjective analysis in order to assess validity. This is particularly true with dynamic simulation output. To reduce or eliminate this subjectivity, this paper proposes a validation process that uses wavelet analysis of variance (WANOVA) as an effective method to statistically accept or reject a model as valid. This WANOVA validation approach performs statistical inference in the time-frequency domain to take advantage of wavelet sparsity and decorrelation. This process uses a test statistic based on thresholded wavelet coefficients to test the null hypothesis that the set of system data and model data are statistically equivalent. The validation technique is illustrated using a simulation study and empirical data from an automobile crash study.