Maximum-likelihood versus maximum a posteriori parameter estimation of physiological system models: the C-peptide impulse response case study

Maximum-likelihood (ML), also given its connection to least-squares (LS), is widely adopted in parameter estimation of physiological system models, i.e., assigning numerical values to the unknown model parameters from the experimental data. A more sophisticated but less used approach is maximum a posteriori (MAP) estimation. Conceptually, while ML adopts a Fisherian approach, i.e., only experimental measurements are supplied to the estimator, MAP estimation is a Bayesian approach, i.e., a priori available statistical information on the unknown parameters is also exploited for their estimation. In this paper, after a brief review of the theory behind ML and MAP estimators, we compare their performance in the solution of a case study concerning the determination of the parameters of a sum of exponential model which describes the impulse response of C-peptide (CP), a key substance for reconstructing insulin secretion. The results show that MAP estimation always leads to parameter estimates with a precision (sometimes significantly) higher than that obtained through ML, at the cost of only a slightly worse fit. Thus, a three exponential model can be adopted to describe the CP impulse response model in place of the two exponential model usually identified in the literature by the ML/LS approach. Simulated case studies are also reported to evidence the importance of taking into account a priori information in a data poor situation, e.g., when a few or too noisy measurements are available. In conclusion, our results show that, when a priori information on the unknown model parameters is available, Bayes estimation can be of relevant interest, since it can significantly improve the precision of parameter estimates with respect to Fisher estimation. This may also allow the adoption of more complex models than those determinable by a Fisherian approach.     

Maximum likelihood estimation of exponential distribution parameters using interval data

This paper addresses the problem of estimating, by the method of maximum likelihood (ML), the location parameter (when present) and scale parameter of the exponential distribution (ED) from interval data. Interval data are defined as two data values that surround an unknown failure observation. Such observations occur naturally, during periodic inspections, for example, when only the time interval during which the failure occurred is known. The appropriate (conditional) log-likelihood functions are derived, as are expressions for the asymptotic variances and covariances of the ML parameter estimates. To illustrate the calculations involved, two numerical examples are discussed.     

Estimation of Weibull Parameters With Competing-Mode Censoring

Existing results are reviewed for the maximum likelihood (ML) estimation of the parameters of a 2-parameter Weibull life distribution for the case where the data are censored by failures due to an arbitrary number of independent 2-parameter Weibull failure modes. For the case where all distributions have a common but unknown shape parameter the joint ML estimators are derived for i) a general percentile of the j-th distribution, ii) the common shape parameter, and iii) the proportion of failures due to failure mode j. Exact interval estimates of the common shape parameter are constructable in terms of the ML estimates obtained by using i) the data without regard to failure mode, and ii) existing tables of the percentage points of a certain pivotal function. Exact interval estimates for a general percentile of failure-mode-j distribution are calculable when the failure proportion due to failure-mode-j is known; otherwise a joint s-confidence region for the percentile and failure proportion is calculable. It is shown that sudden death endurance test results can be analyzed as a special case of competing-mode censoring. Tabular values for the construction of interval estimates for the 10-th percentile of the failure-mode-j distribution are given for 17 combinations of sample size (from 5 to 30) and number of failures.     

Confidence Limits for Weibull Regression With Censored Data

The response variable in an experiment follows a 2-parameter Weibull distribution having a scale parameter that varies inversely with a power of a deterministic, externally controlled, variable generically termed a stress. The shape parameter is invariant with stress. A numerical scheme is given for solving a pair of nonlinear simultaneous equations for the maximum likelihood (ML) estimates of the common shape parameter and the stress-life exponent. Interval and median unbiased point estimates for the shape parameter, stress-life exponent and a specified percentile at any stress, are expressed in terms of percentage points of the sampling distributions of pivotal functions of the ML estimates. A numerical example is given.     

Parameter estimation for the 3-parameter Gamma distribution usingthe continuation method

This paper shows a maximum-likelihood (ML) parameter estimation algorithm for the 3-parameter Gamma distribution. The algorithm, a combination of the continuation method and the extended Gamma distribution model, can find the local ML estimates of the parameters without a careful selection of the starting point in the iterative process. This algorithm is more efficient than previous algorithms, and can find the multiple local ML estimates     

Maximum likelihood array processing for stochastic coherent sources

Maximum likelihood (ML) estimation in array signal processing for the stochastic noncoherent signal case is well documented in the literature. We focus on the equally relevant case of stochastic coherent signals. Explicit large-sample realizations are derived for the ML estimates of the noise power and the (singular) signal covariance matrix. The asymptotic properties of the estimates are examined, and some numerical examples are provided. In addition, we show the surprising fact that the ML estimates of the signal parameters obtained by ignoring the information that the sources are coherent coincide in large samples with the ML estimates obtained by exploiting the coherent source information. Thus, the ML signal parameter estimator derived for the noncoherent case (or its large-sample realizations) asymptotically achieves the lowest possible estimation error variance (corresponding to the coherent Cramer-Rao bound)     

ML parameter estimation for Markov random fields with applicationsto Bayesian tomography

Markov random fields (MRFs) have been widely used to model images in Bayesian frameworks for image reconstruction and restoration. Typically, these MRF models have parameters that allow the prior model to be adjusted for best performance. However, optimal estimation of these parameters (sometimes referred to as hyperparameters) is difficult in practice for two reasons: (i) direct parameter estimation for MRFs is known to be mathematically and numerically challenging; (ii) parameters can not be directly estimated because the true image cross section is unavailable. We propose a computationally efficient scheme to address both these difficulties for a general class of MRF models, and we derive specific methods of parameter estimation for the MRF model known as generalized Gaussian MRF (GGMRF). We derive methods of direct estimation of scale and shape parameters for a general continuously valued MRF. For the GGMRF case, we show that the ML estimate of the scale parameter, sigma, has a simple closed-form solution, and we present an efficient scheme for computing the ML estimate of the shape parameter, p, by an off-line numerical computation of the dependence of the partition function on p. We present a fast algorithm for computing ML parameter estimates when the true image is unavailable. To do this, we use the expectation maximization (EM) algorithm. We develop a fast simulation method to replace the E-step, and a method to improve the parameter estimates when the simulations are terminated prior to convergence. Experimental results indicate that our fast algorithms substantially reduce the computation and result in good scale estimates for real tomographic data sets.     

Noncausal 2-D spectrum estimation for direction finding

A two-dimensional noncausal autoregressive (NCAR) plus additive noise model-based spectrum estimation method is presented for planar array data typical of signals encountered in array processing applications. Since the likelihood function for NCAR plus noise data is nonlinear in the model parameters and is further complicated by the unknown variance of the additive noise, computationally intensive gradient search algorithms are required for computing the estimates. If a doubly periodic lattice is assumed, the complexity of the approximate maximum likelihood (ML) equation is significantly reduced without destroying the theoretical asymptotic properties of the estimates and degrading the observed accuracy of the estimated spectra. Initial conditions for starting the approximate ML computation are suggested. Experimental results that can be used to evaluate the signal-plus-noise approach and compare its performance to those of signal-only methods are presented for Gaussian and simulated planar array data. Statistics of estimated spectrum parameters are given, and estimated spectra for signals with close spatial frequencies are shown. The approximate ML parameter estimate's asymptotic properties, such as consistency and normality, are established, and lower bounds for the estimate's errors are derived, assuming that the data are Gaussian     

Bounds on maximum likelihood ratio-part I: application to antenna array detection-estimation with perfect wavefront coherence

The multiple hypothesis testing problem of the detection-estimation of an unknown number of independent Gaussian point sources is adequately addressed by likelihood ratio (LR) maximization over the set of admissible covariance matrix models. We introduce nonasymptotic lower and upper bounds for the maximum LR. Since LR optimization is generally a nonconvex multiextremal problem, any practical solution could now be tested against these bounds, enabling a high probability of recognizing nonoptimal solutions. We demonstrate that in many applications, the lower bound is quite tight, with approximate maximum likelihood (ML) techniques often unable to approach this bound. The introduced lower bound analysis is shown to be very efficient in determining whether or not performance breakdown has occurred for subspace-based direction-of-arrival (DOA) estimation techniques. We also demonstrate that by proper LR maximization, we can extend the range of signal-to-noise ratio (SNR) values and/or number of data samples wherein accurate parameter estimates are produced. Yet, when the SNR and/or sample size falls below a certain limit for a given scenario, we show that ML estimation suffers from a discontinuity in the parameter estimates: a phenomenon that cannot be eliminated within the ML paradigm.     

Solving ML equations for 2-parameter Poisson-process models forungrouped software-failure data

Existence conditions are given for maximum likelihood (ML) parameter estimates for several families of 2-parameter software-reliability Poisson-process models. For each such model, the ML equations can be expressed in terms of one equation in one unknown. Bounds are given on solutions to these one equation problems to serve as initial intervals for search algorithms like bisection. Uniqueness of the solutions is established in some cases. Solutions are also tabulated for certain simple cases. Results are given for ungrouped failure data (exact times are available for all failures). ML estimation problems for such a situation are treated as limiting cases of problems based on failure times grouped into intervals of decreasing mesh     

A model-based approach for estimation of two-dimensional maximum entropy power spectra

A stochastic model-based approach is presented for estimation of the two-dimensional maximum entropy power spectrum (MEPS) from given finite uniform array data. The method consists of fitting an appropriate two-dimensional noncausal Gaussian-Markov random field (GMRF) model to the given data using the maximum likelihood (ML) technique for parameter estimation. The nonlinear criterion function used for ML estimation is similar in structure to the function arising in the deterministic approach of Lang and McClellan. The model-based approach provides new insights into the two-dimensional MEPS estimation problem. For example, using the asymptotic normality of ML estimates, we derive simultaneous confidence bands for the estimated MEPS. It turns out that when the true correlations are generated by a noncausal GMRF model, the two-dimensional MEPS can be obtained by solving linear equations. This approach also suggests techniques for realizing two-dimensional GMRF models from the given correlation data. Several numerical examples are given to illustrate the usefulness of the approach.     

Parametric GLRT for Multichannel Adaptive Signal Detection

This paper considers the problem of detecting a multichannel signal in the presence of spatially and temporally colored disturbance. A parametric generalized likelihood ratio test (GLRT) is developed by modeling the disturbance as a multichannel autoregressive (AR) process. Maximum likelihood (ML) parameter estimation underlying the parametric GLRT is examined. It is shown that the ML estimator for the alternative hypothesis is nonlinear and there exists no closed-form expression. To address this issue, an asymptotic ML (AML) estimator is presented, which yields asymptotically optimum parameter estimates at reduced complexity. The performance of the parametric GLRT is studied by considering challenging cases with limited or no training signals for parameter estimation. Such cases (especially when training is unavailable) are of great interest in detecting signals in heterogeneous, fast changing, or dense-target environments, but generally cannot be handled by most existing multichannel detectors which rely more heavily on training at an adequate level. Compared with the recently introduced parametric adaptive matched filter (PAMF) and parametric Rao detectors, the parametric GLRT achieves higher data efficiency, offering improved detection performance in general.     

On the robustness of parameter estimation of superimposed signalsby dynamic programming

We analyze a recently proposed dynamic programming algorithm (REDP) for maximum likelihood (ML) parameter estimation of superimposed signals in noise. We show that it degrades gracefully with deviations from the key assumption of a limited interaction signal model (LISMO), providing exact estimates when the LISMO assumption holds exactly. In particular, we show that the deviations of the REDP estimates from the exact ML are continuous in the deviation of the signal model from the LISMO assumption. These deviations of the REDP estimates from the MLE are further quantified by a comparison to an ML algorithm with an exhaustive multidimensional search on a lattice in parameter space. We derive an explicit expression for the lattice spacing for which the two algorithms have equivalent optimization performance, which can be used to assess the robustness of REDP to deviations from the LISMO assumption. The values of this equivalent lattice spacing are found to be small for a classical example of superimposed complex exponentials in noise, confirming the robustness of REDP for this application     

一种新的K分布形状参数估计器

该文提出了一种将已有的U估计器和X估计器结合起来估计K分布形状参数的新估计器。仿真结果表明,与以前提出的方法相比,新估计器在我们所关心的小v值范围能够提供更加准确的形状参数估计,几乎相当于通过数值方法计算的最大似然估计器的性能。     

The expectation-maximization algorithm

A common task in signal processing is the estimation of the parameters of a probability distribution function. Perhaps the most frequently encountered estimation problem is the estimation of the mean of a signal in noise. In many parameter estimation problems the situation is more complicated because direct access to the data necessary to estimate the parameters is impossible, or some of the data are missing. Such difficulties arise when an outcome is a result of an accumulation of simpler outcomes, or when outcomes are clumped together, for example, in a binning or histogram operation. There may also be data dropouts or clustering in such a way that the number of underlying data points is unknown (censoring and/or truncation). The EM (expectation-maximization) algorithm is ideally suited to problems of this sort, in that it produces maximum-likelihood (ML) estimates of parameters when there is a many-to-one mapping from an underlying distribution to the distribution governing the observation. The EM algorithm is presented at a level suitable for signal processing practitioners who have had some exposure to estimation theory     

一种基于多模板的超宽带信道估计算法

该文针对码片率抽头间隔的脉冲超宽带(IR-UWB)系统离散信道,提出一种基于帧(frame)级采样速率的多模板信道估计算法。在该算法中引入了若干个复合信道参数,每个复合信道参数都是几个独立信道参数的线性组合。利用极大似然准则和从多个模板中获得的帧级采样数据,可以先获得所有复合信道参数的极大似然估计值。然后联合由不同模板得到的复合信道参数估计值,得到每个独立的信道参数的极大似然估计。通过仿真,验证了该方法的可行性。     

Statistical Analysis of Dynamic Tracer Data

Statistical estimation theory is applied to the problem of analyzing data obtained in dynamic tracer studies in nuclear medicine. Procedures are given for the determination of the maximum likelihood (ML), maximum a posteriori probability (MAP), and minimum meansquare-error (MMSE) estimates of parameters describing the transport of the monitored tracer in tissue. Some numerical results are given for data obtained in monitoring the natural decay of strontium-85 and oxygen-15 isotopes frequently used in nuclear medicine.     

Estimating a covariance matrix from incomplete realizations of arandom vector

It is desired to estimate the mean and the covariance matrix of a Gaussian random vector from a set of independent realizations, with the complication that not every component of each realization of the random vector is observed. Subject to some restrictions, the authors obtained an exact, noniterative solution for the maximum likelihood (ML) estimates of the mean and the covariance matrix. The ML estimate of the covariance matrix that is obtained from the set of incomplete realizations is guaranteed to be positive definite, in contrast to ad hoc approaches based on averaging products of components from the same realization. The key to obtaining the ML estimates is a tractable expression for the likelihood function in terms of the Cholesky factors of the inverse covariance matrix. With this formulation, the ML estimates are found by fitting regression operators to appropriate subsets of the data. The Cholesky formulation also leads to a simple calculation by Cramer-Rao bounds     

Performance analysis of direction finding with large arrays andfinite data

This paper considers analysis of methods for estimating the parameters of narrow-band signals arriving at an array of sensors. This problem has important applications in, for instance, radar direction finding and underwater source localization. The so-called deterministic and stochastic maximum likelihood (ML) methods are the main focus of this paper. A performance analysis is carried out assuming a finite number of samples and that the array is composed of a sufficiently large number of sensors. Several thousands of antennas are not uncommon in, e.g., radar applications. Strong consistency of the parameter estimates is proved, and the asymptotic covariance matrix of the estimation error is derived. Unlike the previously studied large sample case, the present analysis shows that the accuracy is the same for the two ML methods. Furthermore, the asymptotic covariance matrix of the estimation error coincides with the deterministic Cramer-Rao bound. Under a certain assumption, the ML methods can be implemented by means of conventional beamforming for a large enough number of sensors. We also include a simple simulation study, which indicates that both ML methods provide efficient estimates for very moderate array sizes, whereas the beamforming method requires a somewhat larger array aperture to overcome the inherent bias and resolution problem     

Blind maximum likelihood sequence estimation of digital sequencesin presence of intersymbol interference

A blind maximum likelihood (ML) sequence estimator for unknown linear dispersive channels is described. The estimator assumes a channel model with quantised parameters. A channel trellis and a data trellis are defined to search for the ML channel and data estimates using the Viterbi algorithm (VA). This approach provides a good performance/complexity tradeoff