Maximum likelihood estimation of exponential distribution parameters using interval data

This paper addresses the problem of estimating, by the method of maximum likelihood (ML), the location parameter (when present) and scale parameter of the exponential distribution (ED) from interval data. Interval data are defined as two data values that surround an unknown failure observation. Such observations occur naturally, during periodic inspections, for example, when only the time interval during which the failure occurred is known. The appropriate (conditional) log-likelihood functions are derived, as are expressions for the asymptotic variances and covariances of the ML parameter estimates. To illustrate the calculations involved, two numerical examples are discussed.     

Space-alternating generalized expectation-maximization algorithm

The expectation-maximization (EM) method can facilitate maximizing likelihood functions that arise in statistical estimation problems. In the classical EM paradigm, one iteratively maximizes the conditional log-likelihood of a single unobservable complete data space, rather than maximizing the intractable likelihood function for the measured or incomplete data. EM algorithms update all parameters simultaneously, which has two drawbacks: 1) slow convergence, and 2) difficult maximization steps due to coupling when smoothness penalties are used. The paper describes the space-alternating generalized EM (SAGE) method, which updates the parameters sequentially by alternating between several small hidden-data spaces defined by the algorithm designer. The authors prove that the sequence of estimates monotonically increases the penalized-likelihood objective, derive asymptotic convergence rates, and provide sufficient conditions for monotone convergence in norm. Two signal processing applications illustrate the method: estimation of superimposed signals in Gaussian noise, and image reconstruction from Poisson measurements. In both applications, the SAGE algorithms easily accommodate smoothness penalties and converge faster than the EM algorithms     

Approximate MLEs for the location and scale parameters of theextreme value distribution with censoring

For the extreme value distribution, the maximum likelihood method does not provide explicit estimators for the location and scale parameters. A method of deriving explicit estimators by approximating the likelihood function is provided. The authors derive the asymptotic variances, covariance, and conditional bias of these estimators, and show that they are almost as efficient as the maximum likelihood estimators and just as efficient as the best linear unbiased and the best linear invariant estimators. Two examples illustrate this method of estimation     

An exact forward-backward maximum likelihood autoregressiveparameter estimation method

A method for obtaining an exact maximum likelihood estimate (MLE) of the autoregressive (AR) parameters is proposed. The method is called the forward-backward maximum likelihood algorithm. Based on a new form of the log likelihood function for a Gaussian AR process, an iterative maximization is used to obtain an MLE of the inverse covariance matrix. The AR parameters are then determined via the normal equations. Experimental results comparing the new method with other popular AR spectrum estimation methods indicate the new method achieves low bias and low variance AR parameter estimates comparable with the existing methods     

Nonuniform image motion estimation in reduced coefficienttransformed domains

The transformed domain maximum likelihood (TDML) algorithm for image motion estimation is presented. This algorithm finds a solution which maximizes a log-likelihood function using a steepest ascent scheme. Important characteristics of the algorithm are the inclusion of noise in the signal model, the consideration of motion as a nonuniform process, and calculation of convergence parameters by means of a linear analysis. Simulation on real image sequences demonstrate the validity of the motion estimator. The experiments also verify the validity of the equations presented for the calculation of the convergence parameters. Additional experiments performed to determine the noise sensitivity of the TDML show that noise resistance can be obtained using a reduced coefficient transform (RCT) TDML algorithm. An additional benefit of using an RCT with the TDML algorithm is an increase in the speed of the algorithm without significant performance degradation. Two of the common transforms, Haar and Walsh-Hadamard, are shown to have some interesting properties when utilized with the RCT-TDML algorithm.     

基于改进单纯形算法的DOA估计

利用最大似然估计获得信号的来波方向，其统计性能要比其他算法优越，但由于该方法需要对多维参数进行优化，从而导致较大的计算量，针对该问题文中提出基于改进单纯形算法的来波方向（DOA）估计。首先利用阵列输出的协方差矩阵对来波方向进行粗略估计，并以该估计值为重心构造初始单纯形，由于待估计的参数都有一定的取值范围，所以文中通过定义罚函数，将有约束的优化问题转化为无约束的优化问题，最后结合罚函数和改进单纯形算法进行DOA估计，计算机仿真结果验证了该方法的正确性和有效性。     

Maximum Likelihood Estimation of the Parameters of the Weibull Distribution by Modified Quasilinearization

An iterative procedure is given for joint maximum likelihood estimation of the three parameters of the Weibull population. For this population, the likelihood function is forned and the maximum likelihood equations are obtained. Simultaneous solution of these equations yields joint maximum likelihood estimators for the three parameters. The proposed iterative procedure is a modified quasilinearization algorithm for solving nonlinear equations with bounded variables. Numerical results are given in which the parameters are estimated from real and from simulated life test data drawn from the Weibull population.     

Solution of the Three-Parameter Weibull Equations by Constrained Modified Quasilinearization (Progressively Censored Samples)

The location, shape, and scale parameters of the Weibull distribution are estimated from Type I progressively censored samples by the method of maximum likelihood. Nonlinear logarithmic likelihood estimating equations are derived, and the approximate asymptotic variance-covariance matrix for the maximum likelihood parameter estimates is given. The iterative procedure to solve the likelihood equations is a stable and rapidly convergent constrained modified quasilinearization algorithm which is applicable to the general case in which all three parameters are unknown. The numerical results indicate that, in terms of the number of iterations required for convergence and in the accuracy of the solution, the proposed algorithm is a very effective technique for solving systems of logarithmic likelihood equations for which all iterative approximations to the solution vector must satisfy certain intrinsic constraints on the parameters. A FORTRAN IV program implementing the maximum likelihood estimation procedure is included.     

Maximum likelihood parameter estimation of superimposed chirpsusing Monte Carlo importance sampling

We address the problem of parameter estimation of superimposed chirp signals in noise. The approach used here is a computationally modest implementation of a maximum likelihood (ML) technique. The ML technique for estimating the complex amplitudes, chirping rates, and frequencies reduces to a separable optimization problem where the chirping rates and frequencies are determined by maximizing a compressed likelihood function that is a function of only the chirping rates and frequencies. Since the compressed likelihood function is multidimensional, its maximization via a grid search is impractical. We propose a noniterative maximization of the compressed likelihood function using importance sampling. Simulation results are presented for a scenario involving closely spaced parameters for the individual signals     

Frequency-domain maximum likelihood pitch determination approach

The rate of oscillation of the vocal cords known as the pitch is an important sound feature that is useful in many speech applications. A novel approach for the automatic detection and estimation of the rate of oscillation of the vocal cords is described. The importance of this approach stems from the fact that pitch determination is conducted using three independent stages: a segmentation stage; a voiced-unvoiced classification stage; and a pitch estimation stage. Segmentation and the detection of voiced segments are implemented prior to pitch estimation in order to: exclude unvoiced sounds and silence from biasing the result of pitch estimation; employ a simple segmentation procedure with low computational complexity and time-delay; enhance the accuracy of voiced-unvoiced classification by including additional features in voicing detection; help pitch tracking by testing similarities over successive segments and to make use of a different analysis domain that enables a high resolution pitch estimation. A frequency-domain maximum likelihood procedure is used for the estimation of the pitch frequency of voiced segments by maximizing a log-likelihood function over the range of possible pitch frequencies in conversational speech. An efficient simplified realization of the generalized likelihood ratio segmentation method is also presented. Computer simulations on a number of utterances show that this approach gives an accurate, reliable and robust estimation of the pitch of voiced sounds.     

三阶多项式相位信号参数估计

三阶多项式相位信号常用于描述复杂运动目标的雷达回波,但其未知参数多,参数估计实现起来比较困难。针对此问题,本文提出了一种分两步的三阶多项式相位信号参数估计方法:首先利用Radon变换提取三阶多项式相位信号在三次相位变换后对应的直线,从而得到相位参数的初始值估计;然后将初始值代入最大似然估计函数并利用单纯形法对其进行优化得到精确估计结果。给出了单分量和多分量情况下的参数估计流程。仿真结果表明,本文算法估计性能达到CRB界,且起始工作信噪比门限低。      

Cyclic minimizers, majorization techniques, and the expectation-maximization algorithm: a refresher

Many parameter estimation problems in signal processing can be reduced to the task of minimizing a function of the unknown parameters. This task is difficult owing to the existence of possibly local minima and the sharpness of the global minimum. In this article we review three approaches that can be used to minimize functions of the type encountered in parameter estimation problems. The first two approaches, the cyclic minimization and the majorization technique, are quite general, whereas the third one, the expectation-maximization (EM) algorithm, is tied to the use of the maximum likelihood (ML) method for parameter estimation. The article provides a quick refresher of the aforementioned approaches for a wide readership.     

Empirical Bayes Estimation in the Weibull Distribution

In part I empirical Bayes estimation procedures are introduced and employed to obtain an estimator for the unknown random scale parameter of a two-parameter Weibull distribution with known shape parameter. In part II, procedures are developed for estimating both the random scale and shape parameters. These estimators use a sequence of maximum likelihood estimates from related reliability experiments to form an empirical estimate of the appropriate unknown prior probability density function. Monte Carlo simulation is used to compare the performance of these estimators with the appropriate maximum likelihood estimator. Algorithms are presented for sequentially obtaining the reduced sample sizes required by the estimators while still providing mean squared error accuracy compatible with the use of the maximum likelihood estimators. In some cases whenever the prior pdf is a member of the Pearson family of distributions, as much as a 60% reduction in total test units is obtained. A numerical example is presented to illustrate the procedures.     

Maximum likelihood parameter estimation of F-ARIMA processes using the genetic algorithm in the frequency domain

This work aims to treat the parameter estimation problem for fractional-integrated autoregressive moving average (F-ARIMA) processes under external noise. Unlike the conventional approaches from the perspective of the time domain, a maximum likelihood (ML) method is developed in the frequency domain since the power spectrum of an F-ARIMA process is in a very explicit and more simple form. However, maximization of the likelihood function is a highly nonlinear estimation problem. Conventional searching algorithms are likely to converge to local maxima under this situation. Since the genetic algorithm (GA) tends to find the globally optimal solution without being trapped at local maxima, an estimation scheme based on the GA is therefore developed to solve the ML parameter estimation problem for F-ARIMA processes from the frequency domain perspective. In the parameter estimation procedure, stability of the F-ARIMA model is ensured, and convergence to the global optimum of the likelihood function is also guaranteed. Finally, several simulation examples are presented to illustrate the proposed estimation algorithm and exhibit its performance.     

ML approaches to channel estimation for pilot-aided multirateDS/CDMA systems

This paper analyzes the asymptotic performance of maximum likelihood (ML) channel estimation algorithms in wideband code division multiple access (WCDMA) scenarios. We concentrate on systems with periodic spreading sequences (period larger than or equal to the symbol span) where the transmitted signal contains a code division multiplexed pilot for channel estimation. First, the asymptotic covariances of the training-only, semi-blind conditional maximum likelihood (CML) and semi-blind Gaussian maximum likelihood (GML) channel estimators are derived. Then, these formulas are further simplified assuming randomized spreading and training sequences under the approximation of high spreading, factors and high number of codes. The results provide a useful tool to describe the performance of the channel estimators as a function of basic system parameters such as number of codes, spreading factors, or traffic to training power ratio     

An optimum strategy for detection in the presence of randomtransient disturbance and white Gaussian noise

Detection of a deterministic signal is considered Because of the presence of the transient disturbance, the traditional totally probabilistic criteria are no longer appropriate, since they cannot be related to the physical reality, as in the case of stationary ergodic noise only. A mixed strategy is proposed for the detection where the disturbance is treated as a semideterministic object while white Gaussian noise is treated as a genuinely random object. Specifically, the author uses as the detection statistic the log-likelihood ratio of two conditional probability measures (corresponding to the signal-presence and the signal-absence hypotheses), given the disturbance, in which the disturbance is replaced by its maximum a posteriori likelihood estimate. The detection statistic takes the generalized (in function space) form of a soft-limiter or a blanker followed by the matched filter. It is an explicit detection algorithm, though improvement in computational efficiency is much desired     

Mean likelihood frequency estimation

Estimation of signals with nonlinear as well as linear parameters in noise is studied. Maximum likelihood estimation has been shown to perform the best among all the methods. In such problems, joint maximum likelihood estimation of the unknown parameters reduces to a separable optimization problem, where first, the nonlinear parameters are estimated via a grid search, and then, the nonlinear parameter estimates are used to estimate the linear parameters. We show that a grid search can be avoided by using the mean likelihood estimator for estimating the unknown nonlinear parameters and how its performance can be made equivalent to that of the maximum likelihood estimator (MLE). The mean likelihood estimator requires computation of a multidimensional integral. However, using the concepts of importance sampling, we obtain the mean likelihood estimate without using integration. The technique is computationally far less burdensome than the direct maximum likelihood method but performs just as well. Simulation examples for estimating frequencies of multiple sinusoids in noise are given. The general technique can be applied to a large class of nonlinear regression problems     

Maximum pseudo likelihood estimation in network tomography

Network monitoring and diagnosis are key to improving network performance. The difficulties of performance monitoring lie in today's fast growing Internet, accompanied by increasingly heterogeneous and unregulated structures. Moreover, these tasks become even harder since one cannot rely on the collaboration of individual routers and servers to measure network traffic directly. Even though the aggregative nature of possible network measurements gives rise to inverse problems, existing methods for solving inverse problems are usually computationally intractable or statistically inefficient. A pseudo likelihood approach is proposed to solve a group of network tomography problems. The basic idea of pseudo likelihood is to form simple subproblems and ignore the dependences among the subproblems to form a product likelihood of the subproblems. As a result, this approach keeps a good balance between the computational complexity and the statistical efficiency of the parameter estimation. Some statistical properties of the pseudo likelihood estimator, such as consistency and asymptotic normality, are established. A pseudo expectation-maximization (EM) algorithm is developed to maximize the pseudo log-likelihood function. Two examples, with simulated or real data, are used to illustrate the pseudo likelihood proposal: 1) inference of the internal link delay distributions through multicast end-to-end measurements; 2) origin-destination matrix estimation through link traffic counts.     

Improved percentile estimation for the two-parameter Weibull distribution

This paper presents an improvement of a technique recently published to estimate the parameters of the two-parameter Weibull distribution. A simple percentile method is used to estimate the two parameters. Computer simulation is employed to compare the proposed method with the maximum likelihood estimation and graphical methods results. A set of frequently-used and newer expressions for estimating the cumulative density are examined. Comparisons are made with both complete and censored data. The primary advantage of the method is its computational simplicity. Results indicate that with respect to Mean Square Error and estimation of the characteristic value with complete data, the percentile method cannot outperform the maximum likelihood method, although differences are minor in many instances. However, with censored data, improvements over the maximum likelihood are observed. When the shape parameter is estimated, the percentile method is quite competitive with that of maximum likelihood for both complete and censored data under a variety of conditions.     

A parametric method for determining the number of signals innarrow-band direction finding

A novel and more accurate method to determine the number of signals in the multisource direction finding problem is developed. The information-theoretic criteria of Y. Yin and P. Krishnaiah (1988) are applied to a set of quantities which are evaluated from the log-likelihood function. Based on proven asymptotic properties of the maximum likelihood estimation, these quantities have the properties required by the criteria. Since the information-theoretic criteria use these quantities instead of the eigenvalues of the estimated correlation matrix, this approach possesses the advantage of not requiring a subjective threshold, and also provides higher performance than when eigenvalues are used. Simulation results are presented and compared to those obtained from the nonparametric method given by H. Wax and T. Kailath (1985)