Parameter estimation in nonlinear systems with auto and crosscorrelated noise

The Gohberg-Heinig explicit formula for the inversion of a block-Toeplitz matrix is used to build an estimator of the inverse of the covariance matrix of a multivariable autoregressive process. This estimator is then conveniently applied to maximum likelihood parameter estimation in nonlinear dynamical systems with output measurements corrupted by additive auto and crosscorrelated noise. An appealing computational simplification is obtained due to the particular form taken by the Gohberg-Heinig formula. The efficiency of the obtained estimation scheme is illustrated via Monte-Carlo simulations and compared with an alternative that is obtained by extending a classical technique of linear system identification to the framework of this paper. These simulations show that the proposed method improves significantly the statistical properties of the estimator in comparison with classical methods. Finally, the ability of the method to provide, in a straightforward way, an accurate confidence region around the estimated parameters is also illustrated.     

基于综合识别系统的雷达测距不确定性的分析与仿真

雷达对目标进行探测,测量精度是一个重要的性能指标。影响一部雷达测量精度的因素是多方面的,本文着重讨论,在只有加型高斯白噪声的背景下,雷达测距的不确定性,并给出相应的仿真结果.     

Computing confidence intervals for log-concave densities

In Balabdaoui, Rufibach, and Wellner (2009), pointwise asymptotic theory was developed for the nonparametric maximum likelihood estimator of a log-concave density. Here, the practical aspects of their results are explored. Namely, the theory is used to develop pointwise confidence intervals for the true log-concave density. To do this, the quantiles of the limiting process are estimated and various ways of estimating the nuisance parameter appearing in the limit are studied. The finite sample size behavior of these estimated confidence intervals is then studied via a simulation study of the empirical coverage probabilities.     

A maximum likelihood approximation method for Dirichlet's parameter estimation

Dirichlet distributions are natural choices to analyse data described by frequencies or proportions since they are the simplest known distributions for such data apart from the uniform distribution. They are often used whenever proportions are involved, for example, in text-mining, image analysis, biology or as a prior of a multinomial distribution in Bayesian statistics. As the Dirichlet distribution belongs to the exponential family, its parameters can be easily inferred by maximum likelihood. Parameter estimation is usually performed with the Newton-Raphson algorithm after an initialisation step using either the moments or Ronning's methods. However this initialisation can result in parameters that lie outside the admissible region. A simple and very efficient alternative based on a maximum likelihood approximation is presented. The advantages of the presented method compared to two other methods are demonstrated on synthetic data sets as well as for a practical biological problem: the clustering of protein sequences based on their amino acid compositions.     

Parametric inference for progressive Type-I hybrid censored data on a simple step-stress accelerated life test model

This paper considers a simple step-stress accelerated life test model under progressive Type-I hybrid censoring scheme. The progressive Type-I hybrid censoring scheme and statistical method in synthetic accelerated stresses are provided so as to decrease the lifetime and reduce the test cost. An exponentially distributed life of test units and a cumulative exposure model are assumed. The maximum likelihood estimates of the model parameters are obtained using a pivotal quantity. Two useful lemmas and a theorem are given to construct the approximate confidence intervals for the model parameters. Finally, simulation results are provided to assess the method of inference developed in this article. The simulation results show that the method does improve for large sample size.     

Modelling nonlinear count time series with local mixtures of Poisson autoregressions

A novel class of nonlinear models is studied based on local mixtures of autoregressive Poisson time series. The proposed model has the following construction: at any given time period, there exist a certain number of Poisson regression models, denoted as experts, where the vector of covariates may include lags of the dependent variable. Additionally, the existence of a latent multinomial variable is assumed, whose distribution depends on the same covariates as the experts. The latent variable determines which Poisson regression is observed. This structure is a special case of the mixtures-of-experts class of models, which is considerably flexible in modelling the conditional mean function. A formal treatment of conditions to guarantee the asymptotic normality of the maximum likelihood estimator is presented, under stationarity and nonstationarity. The performance of common model selection criteria in selecting the number of experts is explored via Monte Carlo simulations. Finally, an application to a real data set is presented, in order to illustrate the ability of the proposed structure to flexibly model the conditional distribution function.     

Bayesian system identification via Markov chain Monte Carlo techniques

The work here explores new numerical methods for supporting a Bayesian approach to parameter estimation of dynamic systems. This is primarily motivated by the goal of providing accurate quantification of estimation error that is valid for arbitrary, and hence even very short length data records. The main innovation is the employment of the Metropolis-Hastings algorithm to construct an ergodic Markov chain with invariant density equal to the required posterior density. Monte Carlo analysis of samples from this chain then provides a means for efficiently and accurately computing posteriors for model parameters and arbitrary functions of them.     

An LFT approach to parameter estimation

In this paper we consider a unified framework for parameter estimation problems. Under this framework, the unknown parameters appear in a linear fractional transformation (LFT). A key advantage of the LFT problem formulation is that it allows us to efficiently compute gradients, Hessians, and Gauss–Newton directions for general parameter estimation problems without resorting to inefficient finite-difference approximations. The generality of this approach also allows us to consider issues such as identifiability, persistence of excitation, and convergence for a large class of model structures under a single unified framework.     

GFREG: a computer program for maximum likelihood regression using the Generalized F distribution

A FORTRAN program is described for maximum likelihood estimation within the Generalized F family of distributions. It can be used to estimate regression parameters in a log-linear model for censored survival times with covariates, for which the error distribution may have a great variety of shapes, including most distributions of current use in biostatistics. The optimization is performed by an algorithm based on the generalized reduced gradient method. A stepwise variable search algorithm for covariate selection is included in the program. Output features include: model selection criteria, standard errors of parameter estimates, quantile and survival rates with their standard errors, residuals and several plots. An example based on data from Princess Margaret Hospital, Toronto, is discussed to illustrate the program's capabilities.     

Identification of process and measurement noise covariance for state and parameter estimation using extended Kalman filter

The performance of Bayesian state estimators, such as the extended Kalman filter (EKF), is dependent on the accurate characterisation of the uncertainties in the state dynamics and in the measurements. The parameters of the noise densities associated with these uncertainties are, however, often treated as ‘tuning parameters’ and adjusted in an ad hoc manner while carrying out state and parameter estimation. In this work, two approaches are developed for constructing the maximum likelihood estimates (MLE) of the state and measurement noise covariance matrices from operating input-output data when the states and/or parameters are estimated using the EKF. The unmeasured disturbances affecting the process are either modelled as unstructured noise affecting all the states or as structured noise entering the process predominantly through known, but unmeasured inputs. The first approach is based on direct optimisation of the ML objective function constructed by using the innovation sequence generated from the EKF. The second approach - the extended EM algorithm - is a derivative-free method, that uses the joint likelihood function of the complete data, i.e. states and measurements, to compute the next iterate of the decision variables for the optimisation problem. The efficacy of the proposed approaches is demonstrated on a benchmark continuous fermenter system. The simulation results reveal that both the proposed approaches generate fairly accurate estimates of the noise covariances. Experimental studies on a benchmark laboratory scale heater-mixer setup demonstrate a marked improvement in the predictions of the EKF that uses the covariance estimates obtained from the proposed approaches.     

Improved point and interval estimation for a beta regression model

In this paper we consider the beta regression model recently proposed by Ferrari and Cribari-Neto [2004. Beta regression for modeling rates and proportions. J. Appl. Statist. 31, 799-815], which is tailored to situations where the response is restricted to the standard unit interval and the regression structure involves regressors and unknown parameters. We derive the second order biases of the maximum likelihood estimators and use them to define bias-adjusted estimators. As an alternative to the two analytically bias-corrected estimators discussed, we consider a bias correction mechanism based on the parametric bootstrap. The numerical evidence favors the bootstrap-based estimator and also one of the analytically corrected estimators. Several different strategies for interval estimation are also proposed. We present an empirical application.     

Maximum likelihood stochastic gradient estimation for Hammerstein systems with colored noise based on the key term separation technique

This paper considers the identification problems of Hammerstein finite impulse response moving average (FIR-MA) systems using the maximum likelihood principle and stochastic gradient method based on the key term separation technique. In order to improve the convergence rate, a maximum likelihood multi-innovation stochastic gradient algorithm is presented. The simulation results show that the proposed algorithms can effectively estimate the parameters of the Hammerstein FIR-MA systems.     

Limits, discovery and cut optimization for a Poisson process with uncertainty in background and signal efficiency: TRolke 2.0

A C++ class was written for the calculation of frequentist confidence intervals using the profile likelihood method. Seven combinations of Binomial, Gaussian, Poissonian and Binomial uncertainties are implemented. The package provides routines for the calculation of upper and lower limits, sensitivity and related properties. It also supports hypothesis tests which take uncertainties into account. It can be used in compiled C++ code, in Python or interactively via the ROOT analysis framework.

Program summary

Program title: TRolke version 2.0Catalogue identifier: AEFT_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEFT_v1_0.htmlProgram obtainable from: CPC Program Library, Queen's University, Belfast, N. IrelandLicensing provisions: MIT licenseNo. of lines in distributed program, including test data, etc.: 3431No. of bytes in distributed program, including test data, etc.: 21 789Distribution format: tar.gzProgramming language: ISO C++.Computer: Unix, GNU/Linux, Mac.Operating system: Linux 2.6 (Scientific Linux 4 and 5, Ubuntu 8.10), Darwin 9.0 (Mac-OS X 10.5.8).RAM:∼20 MBClassification: 14.13.External routines: ROOT (http://root.cern.ch/drupal/)Nature of problem: The problem is to calculate a frequentist confidence interval on the parameter of a Poisson process with statistical or systematic uncertainties in signal efficiency or background.Solution method: Profile likelihood method, AnalyticalRunning time:<¹⁰−4 seconds per extracted limit.     

On the convergence of an extended state observer for nonlinear systems with uncertainty

The extended state observer first proposed by Jingqing Han in [J.Q. Han, A class of extended state observers for uncertain systems, Control Decis. 10 (1) (1995) 85-88 (in Chinese)] is the key link toward the active disturbance rejection control that is taking off as a technology after numerous successful applications in engineering. Unfortunately, there is no rigorous proof of convergence to date. In this paper, we attempt to tackle this long unsolved extraordinary problem. The main idea is to transform the error equation of objective system with its extended state observer into a asymptotical stable system with a small disturbance, for which the effect of total disturbance error is eliminated by the high-gain.     

A quantile estimation for massive data with generalized Pareto distribution

This paper proposes a new method of estimating extreme quantiles of heavy-tailed distributions for massive data. The method utilizes the Peak Over Threshold (POT) method with generalized Pareto distribution (GPD) that is commonly used to estimate extreme quantiles and the parameter estimation of GPD using the empirical distribution function (EDF) and nonlinear least squares (NLS). We first estimate the parameters of GPD using EDF and NLS and then, estimate multiple high quantiles for massive data based on observations over a certain threshold value using the conventional POT. The simulation results demonstrate that our parameter estimation method has a smaller Mean square error (MSE) than other common methods when the shape parameter of GPD is at least 0. The estimated quantiles also show the best performance in terms of root MSE (RMSE) and absolute relative bias (ARB) for heavy-tailed distributions.     

A nonlinear observer for on-line estimation of the cerebrospinal fluid outflow resistance

Accurate estimates of the outflow resistance of the human cerebrospinal fluid system are important for the diagnosis of a medical condition known as hydrocephalus. In this paper we design a nonlinear observer which provides on-line estimates of the outflow resistance, to the best of our knowledge the first method to do so. The output of the observer is proven to globally converge to an unbiased estimate. Its performance is experimentally verified using the same apparatus used to perform actual patient diagnoses and a specially-designed physical model of the human cerebrospinal fluid system.     

A quick method for estimation of parameters of individual pulses in a multipath signal

Consider a multipath signal whose individual signals are deterministic known increasing or decreasing pulses. The problem is to estimate the amplitudes and delay times of individual signals. Several methods have been devoted to the solution of these unknown parameters. The maximum likelihood (ML) estimation and the estimate maximize (EM) algorithm are commonly used, but they are computationally intensive and still insufficient to obtain accurate estimations. The method can provide a quick and accurate estimate of the amplitudes and arrival (delay) times, even in the closely spaced multipaths and heavy noise.     

Maximum likelihood estimation for constrained parameters of multinomial distributions—Application to Zipf-Mandelbrot models

A numerical maximum likelihood (ML) estimation procedure is developed for the constrained parameters of multinomial distributions. The main difficulty involved in computing the likelihood function is the precise and fast determination of the multinomial coefficients. For this the coefficients are rewritten into a telescopic product. The presented method is applied to the ML estimation of the Zipf-Mandelbrot (ZM) distribution, which provides a true model in many real-life cases. The examples discussed arise from ecological and medical observations. Based on the estimates, the hypothesis that the data is ZM distributed is tested using a chi-square test. The computer code of the presented procedure is available on request by the author.     

Approximation of the Feasible Parameter Set in worst-case identification of Hammerstein models

The estimation of the Feasible Parameter Set (FPS) for Hammerstein models in a worst-case setting is considered. A bounding procedure is determined both for polytopic and ellipsoidic uncertainties. It consists in the projection of the FPS of the extended parameter vector onto suitable subspaces and in the solution of convex optimization problems which provide Uncertainties Intervals of the model parameters. The bounds obtained are tighter than in the previous approaches.     

Simplified robust control for nonlinear uncertain systems: a method of projection and online estimation

Robust control based on an online estimation of uncertainty is presented for a class of nonlinear uncertain systems. The estimation is done via a robust observer after the uncertainty vector is projected onto a one-dimensional subspace. The proposed combination of dynamics projection and online estimation is to relax the knowledge about the size of uncertainty and required in the robust control design, to make robust control less conservative while being effective, and to ensure robust stability without undue complexity.