Robust Parameter Estimation in Dynamic Systems

In this paper we present a practical method for robust parameter estimation in dynamic systems. In our study we follow the very successful approach for solving optimization problems in dynamic systems, namely the boundary value problem (BVP) approach. The suggested method combines multiple shooting for parameterizing dynamics, a flexible realization of the BVP principle, with a fast Gauss-Newton algorithm for solving the resulting constrained l ₁ problem. We give an overview of the theoretical background as well as the details of a numerical implementation. We discuss why the Gauss-Newton algorithm, which is known to perform well mainly on well-conditioned problems, is appropriate for parameter estimation problems, while quasi-Newton methods have only limited use for parameter estimation. The method is implemented on the basis of the direct multiple shooting method as implemented in PARFIT, thus inheriting all basic properties of PARFIT such as numerical stability, reliability and efficiency. The new code has been successfully applied to real-life parameter estimation problems in enzyme and chemical kinetics.     

Squeeze-and-breathe evolutionary Monte Carlo optimization with local search acceleration and its application to parameter fitting

Estimating parameters from data is a key stage of the modelling process, particularly in biological systems where many parameters need to be estimated from sparse and noisy datasets. Over the years, a variety of heuristics have been proposed to solve this complex optimization problem, with good results in some cases yet with limitations in the biological setting. In this work, we develop an algorithm for model parameter fitting that combines ideas from evolutionary algorithms, sequential Monte Carlo and direct search optimization. Our method performs well even when the order of magnitude and/or the range of the parameters is unknown. The method refines iteratively a sequence of parameter distributions through local optimization combined with partial resampling from a historical prior defined over the support of all previous iterations. We exemplify our method with biological models using both simulated and real experimental data and estimate the parameters efficiently even in the absence of a priori knowledge about the parameters.     

Deterministic inference for stochastic systems using multiple shooting and a linear noise approximation for the transition probabilities

Estimating model parameters from experimental data is a crucial technique for working with computational models in systems biology. Since stochastic models are increasingly important, parameter estimation methods for stochastic modelling are also of increasing interest. This study presents an extension to the ‘multiple shooting for stochastic systems (MSS)’ method for parameter estimation. The transition probabilities of the likelihood function are approximated with normal distributions. Means and variances are calculated with a linear noise approximation on the interval between succeeding measurements. The fact that the system is only approximated on intervals which are short in comparison with the total observation horizon allows to deal with effects of the intrinsic stochasticity. The study presents scenarios in which the extension is essential for successfully estimating the parameters and scenarios in which the extension is of modest benefit. Furthermore, it compares the estimation results with reversible jump techniques showing that the approximation does not lead to a loss of accuracy. Since the method is not based on stochastic simulations or approximative sampling of distributions, its computational speed is comparable with conventional least‐squares parameter estimation methods.Inspec keywords: stochastic systems, parameter estimation, probability, least squares approximationsOther keywords: deterministic inference, stochastic systems, multiple shooting, linear noise approximation, transition probabilities, systems biology, parameter estimation methods, likelihood function, normal distributions, intrinsic stochasticity effects, reversible jump techniques, approximative sampling, conventional least‐squares parameter estimation methods     

Data fusion algorithm for pulsed eddy current detection

A weighted data fusion algorithm based on matching pursuit (MP)-wavelet packet (WP) atomic decomposition and its applications in pulsed eddy current (PEC) non-destructive testing systems for estimation of feature parameters is presented. MP-WP atomic decomposition is used to estimate each noise-free pulse response from its noisy observation of a single-sensor PEC probe and obtain the peak value parameter from each estimated response. A weighted data fusion algorithm, on the basis of minimum mean square error (MMSE), is applied to fuse each obtained peak value together to get final optimum parameter estimation. Based on the difference of each noisy pulse response and its estimation, the variance of noise in each pulse response can be computed, respectively. Accordingly, the weight of each pulse response for data fusion is calculated by the variance of its noise. Finally, the peak value parameter is estimated by the utilised data fusion algorithm. In terms of MMSE, this weighted fusion presents an optimum estimation of the feature parameter of multi-pulse responses of PEC sensor, compared with normal averaging process.     

An efficient and unbiased method for sensitivity analysis of stochastic reaction networks

We consider the problem of estimating parameter sensitivity for Markovian models of reaction networks. Sensitivity values measure the responsiveness of an output with respect to the model parameters. They help in analysing the network, understanding its robustness properties and identifying the important reactions for a specific output. Sensitivity values are commonly estimated using methods that perform finite-difference computations along with Monte Carlo simulations of the reaction dynamics. These methods are computationally efficient and easy to implement, but they produce a biased estimate which can be unreliable for certain applications. Moreover, the size of the bias is generally unknown and hence the accuracy of these methods cannot be easily determined. There also exist unbiased schemes for sensitivity estimation but these schemes can be computationally infeasible, even for very simple networks. Our goal in this paper is to present a new method for sensitivity estimation, which combines the computational efficiency of finite-difference methods with the accuracy of unbiased schemes. Our method is easy to implement and it relies on an exact representation of parameter sensitivity that we recently proved elsewhere. Through examples, we demonstrate that the proposed method can outperform the existing methods, both biased and unbiased, in many situations.     

Efficient algorithms for ordinary differential equation model identification of biological systems

Algorithms for parameter estimation and model selection that identify both the structure and the parameters of an ordinary differential equation model from experimental data are presented. The work presented here focuses on the case of an unknown structure and some time course information available for every variable to be analysed, and this is exploited to make the algorithms as efficient as possible. The algorithms are designed to handle problems of realistic size, where reactions can be nonlinear in the parameters and where data can be sparse and noisy. To achieve computational efficiency, parameters are mostly estimated for one equation at a time, giving a fast and accurate parameter estimation algorithm compared with other algorithms in the literature. The model selection is done with an efficient heuristic search algorithm, where the structure is built incrementally. Two test systems are used that have previously been used to evaluate identification algorithms, a metabolic pathway and a genetic network. Both test systems were successfully identified by using a reasonable amount of simulated data. Besides, measurement noise of realistic levels can be handled. In comparison to other methods that were used for these test systems, the main strengths of the presented algorithms are that a fully specified model, and not only a structure, is identified, and that they are considerably faster compared with other identification algorithms.     

Field profiles of coupled surface plasmon-polaritons

We present a study of the electromagnetic field profiles for coupled surface plasmon-polariton systems. Results for both the symmetrically-clad thin metal film structure and the metal-clad dielectric cavity are given. We also consider an asymmetrically-clad thin metal-film structure and show that such a structure may also support coupled SPP modes under appropriate conditions. We describe our method for calculating the field profiles in detail. In contrast to previous methods our approach does not require the introduction of an input field, it allows straightforward computation of the field profiles associated with the optical modes of multilayer planar structures.     

Continuous analogue to iterative optimization for PDE-constrained inverse problems

The parameters of many physical processes are unknown and have to be inferred from experimental data. The corresponding parameter estimation problem is often solved using iterative methods such as steepest descent methods combined with trust regions. For a few problem classes also continuous analogues of iterative methods are available. In this work, we expand the application of continuous analogues to function spaces and consider PDE (partial differential equation)-constrained optimization problems. We derive a class of continuous analogues, here coupled ODE (ordinary differential equation)–PDE models, and prove their convergence to the optimum under mild assumptions. We establish sufficient bounds for local stability and convergence for the tuning parameter of this class of continuous analogues, the retraction parameter. To evaluate the continuous analogues, we study the parameter estimation for a model of gradient formation in biological tissues. We observe good convergence properties, indicating that the continuous analogues are an interesting alternative to state-of-the-art iterative optimization methods.     

Obtaining the phase of an interferogram by use of an evolution strategy: part I

We present a method for obtaining the phase of a noisy simulated interferogram. We find the wave-front aberrations by transforming the problem of fitting a polynomial into an optimization problem, which is then solved using an evolutionary algorithm. Our experimental results show that our method yields a more accurate solution than other methods commonly used to solve this problem.     

A Poisson shot noise model for micro-penetration of snow

As a common starting point for the interpretation of SnowMicroPenetrometer (SMP) data the penetration force is interpreted as a superposition of spatially uncorrelated ruptures of structural elements which follow an ideal elastic–brittle response. We re-state this idea and describe the fluctuating penetration force as a Poisson shot noise process. This allows us to derive simple analytical expressions for the cumulants and the covariance of the penetration force in terms of the micromechanical, force-displacement parameters of individual elements. Vice versa, the micromechanical parameters can be estimated from the statistics of the penetration force. We test our method with simulated shot noise processes and real snow profiles and reveal potential limitations of the underlying assumptions. Our model unifies different previous approaches to snow classification which are based on snow penetration resistance.     

A nonlinear programming approach for estimation of transmission parameters in childhood infectious disease using a continuous time model

Mathematical models can enhance our understanding of childhood infectious disease dynamics, but these models depend on appropriate parameter values that are often unknown and must be estimated from disease case data. In this paper, we develop a framework for efficient estimation of childhood infectious disease models with seasonal transmission parameters using continuous differential equations containing model and measurement noise. The problem is formulated using the simultaneous approach where all state variables are discretized, and the discretized differential equations are included as constraints, giving a large-scale algebraic nonlinear programming problem that is solved using a nonlinear primal–dual interior-point solver. The technique is demonstrated using measles case data from three different locations having different school holiday schedules, and our estimates of the seasonality of the transmission parameter show strong correlation to school term holidays. Our approach gives dramatic efficiency gains, showing a 40–400-fold reduction in solution time over other published methods. While our approach has an increased susceptibility to bias over techniques that integrate over the entire unknown state-space, a detailed simulation study shows no evidence of bias. Furthermore, the computational efficiency of our approach allows for investigation of a large model space compared with more computationally intensive approaches.     

Identifying Lamé parameters from time-dependent elastic wave measurements

In many sectors of today’s industry it is of utmost importance to detect defects in elastic structures contained in technical devices to guarantee their failure-free operation. As currently used signal processing techniques have natural limits with respect to accuracy and significance, modern mathematical methods are crucial to improve current algorithms. We consider in this paper a parameter identification approach for isotropic and linear elastic structures described by their Lamé parameters and a material density. This approach can be employed for non-destructive defect detection, location and characterization from time-dependent measurements of one elastic wave. To this end, we show that the operator linking the static parameters with the wave measurements is Fréchet differentiable, which allows to set up Newton-like methods for the non-linear parameter identification problem. We indicate the performance of a specific inexact Newton-like regularization method by numerical examples for a testing problem of a thin plate from measurements of the normal component of the displacement field on the boundary. As an extension, we further augment this method with a total variation regularization and thereby improve reconstructed parameters that feature edges.     

Fusion of noisy images based on joint distribution model in dual-tree complex wavelet domain

Source images are frequently corrupted by noise before fusion, which will lead to the quality decline of fused image and the inconvenience for subsequent observation. However, at present, most of the traditional medical image fusion scheme cannot be implemented in noisy environment. Besides, the existing fusion methods scarcely make full use of the dependencies between source images. In this research, a novel fusion algorithm based on the statistical properties of wavelet coefficients is proposed, which incorporates fusion and denoising simultaneously. In the proposed algorithm, the new saliency and matching measures are defined by two distributions: the marginal statistical distribution of single wavelet coefficient fit by the generalized Gaussian Distribution and joint distribution of dual source wavelet coefficients modeled by the anisotropic bivariate Laplacian model. Additionally, the bivariate shrinkage is introduced to develop a noise robust fusion method, and a moment-based parameter estimation applied in the fusion scheme is also exploited in denoising method, which allows to achieve the consistency of fusion and denoising. The experiments demonstrate that the proposed algorithm performs very well on both noisy and noise-free images from multimodal medical datasets (computerized tomography, magnetic resonance imaging, magnetic resonance angiography, etc.), outperforming the conventional methods in terms of both fusion quality and noise reduction.     

The evolution of family level sales forecasts into product level forecasts: Modeling and estimation

It is frequently the case that sales forecasts are available at the detailed product level for only a relatively short time horizon. For the rest of the forecast horizon, only aggregate sales forecasts at the product family level are available. The problem addressed in this paper is how to fit a forecast simulation model to a history of these aggregate and disaggregate forecasts. Our approach to develop such a model is to combine a forecast update model with a forecast disaggregation model. The forecast update model is called the Martingale model of forecast evolution. The parameters of the two models must be estimated from historical forecast data. It is this statistical parameter estimation problem that occupies the major part of our investigation. We recommend an estimation technique based on the method of moments.     

Model-based process optimization in the presence of parameter uncertainty

In model-based process optimization one uses a mathematical model to optimize a certain criterion, for example the product yield of a chemical process. Models often contain parameters that have to be estimated from data. Typically, a point estimate (e.g. the least squares estimate) is used to fix the model for the optimization stage. However, parameter estimates are uncertain due to incomplete and noisy data. In this article, it is shown how parameter uncertainty can be taken into account in process optimization. To quantify the uncertainty, Markov Chain Monte Carlo (MCMC) sampling, an emerging standard approach in Bayesian estimation, is used. In the Bayesian approach, the solution to the parameter estimation problem is given as a distribution, and the optimization criteria are functions of that distribution. The formulation and implementation of the optimization is studied, and numerical examples are used to show that parameter uncertainty can have a large effect in optimization results.     

误差白化与Kalman滤波的微硅加速度传感器动态补偿

通过模型参考的系统辨识方法建立微硅加速度传感器的动态补偿器。由于测量噪声和补偿器对传感器的频带扩展,使得补偿器的输入/输出信号存在严重的噪声干扰。在噪声干扰下,采用均方误差为代价函数的系统辨识方法,无法得到补偿器参数的无偏估计。补偿器参数的偏差和传感器频带的扩展将会使补偿器的输出信号出现严重失真和高频噪声干扰。为解决噪声对硅加速度传感器的动态补偿的影响,研究了一种新的动态补偿方法,该方法采用误差白化为代价函数的系统辨识方法得到补偿器的参数,可消除补偿器的参数在估计中的测量噪声影响,并通过卡尔曼实时滤波减小因传感器频带扩展所产生的高频噪声干扰。     

Index to Volume 32

ABSTRACT

Most of the recently developed methods on optimum planning for accelerated life tests (ALT) involve “guessing” values of parameters to be estimated, and substituting such guesses in the proposed solution to obtain the final testing plan. In reality, such guesses may be very different from true values of the parameters, leading to inefficient test plans. To address this problem, we propose a sequential Bayesian strategy for planning of ALTs and a Bayesian estimation procedure for updating the parameter estimates sequentially. The proposed approach is motivated by ALT for polymer composite materials, but are generally applicable to a wide range of testing scenarios. Through the proposed sequential Bayesian design, one can efficiently collect data and then make predictions for the field performance. We use extensive simulations to evaluate the properties of the proposed sequential test planning strategy. We compare the proposed method to various traditional non-sequential optimum designs. Our results show that the proposed strategy is more robust and efficient, as compared to existing non-sequential optimum designs. Supplementary materials for this article are available online.     

A fatigue model with local sensitivity analysis

The goal of this paper is two fold. First, it introduces a general parametric lifetime model for high‐cycle fatigue regime derived from physical, statistical, engineering and dimensional analysis considerations. The proposed model has two threshold parameters and three Weibull distribution parameters. A two‐step procedure is presented to estimate the parameters. In the first step, the two threshold parameters are estimated by minimizing a least squares regression function. In the second step, the parameters are estimated by the maximum likelihood method after pooling together the data from different stress levels. Since parameter estimation should always be accompanied by a sensitivity analysis of the fitted model, the second goal of this paper is to propose a method for sensitivity analysis for fatigue models. We show that the proposed sensitivity analysis methods are general and can be applied to any fatigue or lifetime model, not just to the one proposed here. Although several fatigue models have been proposed in the literature, to our knowledge this is the first attempt to produce methods for sensitivity analysis for fatigue models. The proposed method makes use of the well‐known duality property of mathematical programming, which states that the partial derivatives of the primal objective function with respect to the constraints right hand side parameters are the optimal values of the negative of the dual problem variables. For the parameters or data, for which sensitivities are sought, to appear on the right hand side, they are converted into artificial variables and set to their actual values, thus obtaining the desired constraints. Both the estimation and sensitivity analysis methods are illustrated by two examples, one application using real fatigue data and the other using simulated data. In addition, the sensitivity proposed method is also applied to an alternative fatigue model. Finally, some specific conclusions and recommendations are also given.     

Evaluation and unification of some methods for estimating reflectance spectra from RGB images

The problem of estimating spectral reflectances from the responses of a digital camera has received considerable attention recently. This problem can be cast as a regularized regression problem or as a statistical inversion problem. We discuss some previously suggested estimation methods based on critically undersampled RGB measurements and describe some relations between them. We concentrate mainly on those models that are using a priori information in the form of high-resolution measurements. We use the "kernel machine" framework in our evaluations and concentrate on the use of multiple illuminations and on the investigation of the performance of global and locally adapted estimation methods. We also introduce a nonlinear transformation of reflectance values to ensure that the estimated reflection spectra fulfill physically motivated boundary conditions. The reported experimental results are derived from measured and simulated camera responses from the Munsell Matte, NCS, and Pantone data sets.     

Nonparametric identification assuming two noise sources: adeconvolution approach

Nonparametric identification of linear systems is investigated in this paper. Nonparametric identification is the estimation of the time record of the impulse response of the system. It is a deconvolution problem, i.e., inverse operation of the convolution of the impulse response and the excitation signal. The problem is ill posed, i.e., deconvolution amplifies the measurement noise to a great extent. The noise has to be suppressed with the price of a bias in the estimate. A tradeoff has to be found between the noisy and biased estimates. Because of the need for repeatability and to reduce the subjectivity, the level of noise reduction has to be set algorithmically. This paper introduces a method that optimizes the parameter(s) of deconvolution filters and, thus, controls the level of noise reduction. The proposed method assumes observation noise sources for both the measurement of the excitation signal and the system output