高维数据固有维数的自适应极大似然估计

如何确定高维数据的固有维数是降维成功与否的关键。基于极大似然估计（MLE）的维数估计方法是一种新近出现的方法,实现简单,选择合适的近邻能取得不错的结果。但当近邻数过小或过大时,均有比较明显的偏差。其根本原因是没有考虑每个点对固有维数的不同贡献。在充分考虑数据集的分布信息之后,提出了一种改进的MLE——自适应极大似然估计(AMLE)。实验表明,无论在合成数据集还是真实数据集上,AMLE较MLE在估计准确度上均有很大的提高,对近邻数的变化也不甚敏感。     

Robust distributed maximum likelihood estimation with dependent quantized data

In this paper, we consider the distributed maximum likelihood estimation (MLE) with dependent quantized data under the assumption that the structure of the joint probability density function (pdf) is known, but it contains unknown deterministic parameters. The parameters may include different vector parameters corresponding to marginal pdfs and parameters that describe the dependence of observations across sensors. Since MLE with a single quantizer is sensitive to the choice of thresholds due to the uncertainty of pdf, we concentrate on MLE with multiple groups of quantizers (which can be determined by the use of prior information or some heuristic approaches) to fend off against the risk of a poor/outlier quantizer. The asymptotic efficiency of the MLE scheme with multiple quantizers is proved under some regularity conditions and the asymptotic variance is derived to be the inverse of a weighted linear combination of Fisher information matrices based on multiple different quantizers which can be used to show the robustness of our approach. As an illustrative example, we consider an estimation problem with a bivariate non-Gaussian pdf that has applications in distributed constant false alarm rate (CFAR) detection systems. Simulations show the robustness of the proposed MLE scheme especially when the number of quantized measurements is small.     

A maximum likelihood approximation method for Dirichlet's parameter estimation

Dirichlet distributions are natural choices to analyse data described by frequencies or proportions since they are the simplest known distributions for such data apart from the uniform distribution. They are often used whenever proportions are involved, for example, in text-mining, image analysis, biology or as a prior of a multinomial distribution in Bayesian statistics. As the Dirichlet distribution belongs to the exponential family, its parameters can be easily inferred by maximum likelihood. Parameter estimation is usually performed with the Newton-Raphson algorithm after an initialisation step using either the moments or Ronning's methods. However this initialisation can result in parameters that lie outside the admissible region. A simple and very efficient alternative based on a maximum likelihood approximation is presented. The advantages of the presented method compared to two other methods are demonstrated on synthetic data sets as well as for a practical biological problem: the clustering of protein sequences based on their amino acid compositions.     

On the equivalence of time and frequency domain maximum likelihood estimation

Maximum likelihood estimation has a rich history. It has been successfully applied to many problems including dynamical system identification. Different approaches have been proposed in the time and frequency domains. In this paper we discuss the relationship between these approaches and we establish conditions under which the different formulations are equivalent for finite length data. A key point in this context is how initial (and final) conditions are considered and how they are introduced in the likelihood function.     

GSA-based maximum likelihood estimation for threshold vector error correction model

The log-likelihood function of threshold vector error correction models is neither differentiable, nor smooth with respect to some parameters. Therefore, it is very difficult to implement maximum likelihood estimation (MLE) of the model. A new estimation method, which is based on a hybrid algorithm and MLE, is proposed to resolve this problem. The hybrid algorithm, referred to as genetic-simulated annealing, not only inherits aspects of genetic-algorithms (GAs), but also avoids premature convergence by incorporating elements of simulated annealing (SA). Simulation experiments demonstrate that the proposed method allows to estimate the parameters of larger cointegrating systems. Additionally, numerical results show that the hybrid algorithm does a better job than either SA or GA alone.     

Parameter estimation using polynomial chaos and maximum likelihood

When modelling biological processes, there are always errors, uncertainties and variations present. In this paper, we consider the coefficients in the mathematical model to be random variables, whose distribution and moments are unknown a priori, and need to be determined by comparison with experimental data. A stochastic spectral representation of the parameters and the solution stochastic process is used, based on polynomial chaoses. The polynomial chaos representation generates a system of equations of the same type as the original model. The inverse problem of finding the parameters is reduced to establishing the best-fit values of the random variables that represent them, and this is done using maximum likelihood estimation. In particular, in modelling biofilm growth, there are variations, measurement errors and uncertainties in the processes. The biofilm growth model is given by a parabolic differential equation, so the polynomial chaos formulation generates a system of partial differential equations. Examples are presented.     

New maximum likelihood motion estimation schemes for noisy ultrasound images

When performing block-matching based motion estimation with the ML estimator, one would try to match blocks from the two images, within a predefined search area. The estimated motion vector is that which maximizes a likelihood function, formulated according to the image formation model. Two new maximum likelihood motion estimation schemes for ultrasound images are presented. The new likelihood functions are based on the assumption that both images are contaminated by a Rayleigh distributed multiplicative noise. The new approach enables motion estimation in cases where a noiseless reference image is not available. Experimental results show a motion estimation improvement with regards to other known ML estimation methods.     

Development of moving window state and parameter estimators under maximum likelihood and Bayesian frameworks

Estimation of slowly varying model parameters/unmeasured disturbances is of paramount importance in process monitoring, fault diagnosis, model based advanced control and online optimization. The conventional approach to estimate drifting parameters is to artificially model them as a random walk process and estimate them simultaneously with the states. However, this may lead to a poorly conditioned problem, where the tuning of the random walk model becomes a non-trivial exercise. In this work, the moving window parameter estimator of Huang et al. [1] is recast as a moving window maximum likelihood (ML) estimator. The state can be estimated within the window using any recursive Bayesian estimator. It is assumed that, when the model parameters are perfectly known, the innovation sequence generated by the chosen Bayesian estimator is a Gaussian white noise process and is further used to construct a likelihood function that treats the model parameters as unknowns. This leads to a well conditioned problem where the only tuning parameter is the length of the moving window, which is much easier to select than selecting the covariance of the random walk model. The ML formulation is further modified to develop a maximum a posteriori (MAP) cost function by including arrival cost for the parameter. Efficacy of the proposed ML and MAP formulations has been demonstrated by conducting simulation studies and experimental evaluation. Analysis of the simulation and experimental results reveals that the proposed moving window ML and MAP estimators are capable of tracking the drifting parameters/unmeasured disturbances fairly accurately even when the measurements are available at multiple rates and with variable time delays.     

Collaborative target tracking in WSNs using the combination of maximum likelihood estimation and Kalman filtering

Target tracking using wireless sensor networks requires efficient collaboration among sensors to tradeoff between energy consumption and tracking accuracy. This paper presents a collaborative target tracking approach in wireless sensor networks using the combination of maximum likelihood estimation and the Kalman filter. The cluster leader converts the received nonlinear distance measurements into linear observation model and approximates the covariance of the converted measurement noise using maximum likelihood estimation, then applies Kalman filter to recursively update the target state estimate using the converted measurements. Finally, a measure based on the Fisher information matrix of maximum likelihood estimation is used by the leader to select the most informative sensors as a new tracking cluster for further tracking. The advantages of the proposed collaborative tracking approach are demonstrated via simulation results.     

Frequency domain maximum likelihood estimation of linear dynamic errors-in-variables models

This paper studies the linear dynamic errors-in-variables problem for filtered white noise excitations. First, a frequency domain Gaussian maximum likelihood (ML) estimator is constructed that can handle discrete-time as well as continuous-time models on (a) part(s) of the unit circle or imaginary axis. Next, the ML estimates are calculated via a computationally simple and numerically stable Gauss-Newton minimization scheme. Finally, the Cramér-Rao lower bound is derived.     

Faster ARMA maximum likelihood estimation

A new likelihood based AR approximation is given for ARMA models. The usual algorithms for the computation of the likelihood of an ARMA model require O(n) flops per function evaluation. Using our new approximation, an algorithm is developed which requires only O(1) flops in repeated likelihood evaluations. In most cases, the new algorithm gives results identical to or very close to the exact maximum likelihood estimate (MLE). This algorithm is easily implemented in high level quantitative programming environments (QPEs) such as Mathematica, MatLab and R. In order to obtain reasonable speed, previous ARMA maximum likelihood algorithms are usually implemented in C or some other machine efficient language. With our algorithm it is easy to do maximum likelihood estimation for long time series directly in the QPE of your choice. The new algorithm is extended to obtain the MLE for the mean parameter. Simulation experiments which illustrate the effectiveness of the new algorithm are discussed. Mathematica and R packages which implement the algorithm discussed in this paper are available [McLeod, A.I., Zhang, Y., 2007. Online supplements to “Faster ARMA Maximum Likelihood Estimation”, 〈http://www.stats.uwo.ca/faculty/aim/2007/faster/〉]. Based on these package implementations, it is expected that the interested researcher would be able to implement this algorithm in other QPEs.     

Recursive maximum likelihood parameter estimation for state space systems using polynomial chaos theory

This paper combines polynomial chaos theory with maximum likelihood estimation for a novel approach to recursive parameter estimation in state-space systems. A simulation study compares the proposed approach with the extended Kalman filter to estimate the value of an unknown damping coefficient of a nonlinear Van der Pol oscillator. The results of the simulation study suggest that the proposed polynomial chaos estimator gives comparable results to the filtering method but may be less sensitive to user-defined tuning parameters. Because this recursive estimator is applicable to linear and nonlinear dynamic systems, the authors portend that this novel formulation will be useful for a broad range of estimation problems.     

Robust maximum likelihood estimation for stochastic state space model with observation outliers

The objective of this paper is to develop a robust maximum likelihood estimation (MLE) for the stochastic state space model via the expectation maximisation algorithm to cope with observation outliers. Two types of outliers and their influence are studied in this paper: namely,the additive outlier (AO) and innovative outlier (IO). Due to the sensitivity of the MLE to AO and IO, we propose two techniques for robustifying the MLE: the weighted maximum likelihood estimation (WMLE) and the trimmed maximum likelihood estimation (TMLE). The WMLE is easy to implement with weights estimated from the data; however, it is still sensitive to IO and a patch of AO outliers. On the other hand, the TMLE is reduced to a combinatorial optimisation problem and hard to implement but it is efficient to both types of outliers presented here. To overcome the difficulty, we apply the parallel randomised algorithm that has a low computational cost. A Monte Carlo simulation result shows the efficiency of the proposed algorithms.     

Frequency domain sample maximum likelihood estimation for spatially dependent parameter estimation in PDEs

The identification of the spatially dependent parameters in Partial Differential Equations (PDEs) is important in both physics and control problems. A methodology is presented to identify spatially dependent parameters from spatio-temporal measurements. Local non-rational transfer functions are derived based on three local measurements allowing for a local estimate of the parameters. A sample Maximum Likelihood Estimator (SMLE) in the frequency domain is used, because it takes noise properties into account and allows for high accuracy consistent parameter estimation. Confidence bounds on the parameters are estimated based on the noise properties of the measurements. This method is successfully applied to the simulations of a finite difference model of a parabolic PDE with piecewise constant parameters.     

GFREG: a computer program for maximum likelihood regression using the Generalized F distribution

A FORTRAN program is described for maximum likelihood estimation within the Generalized F family of distributions. It can be used to estimate regression parameters in a log-linear model for censored survival times with covariates, for which the error distribution may have a great variety of shapes, including most distributions of current use in biostatistics. The optimization is performed by an algorithm based on the generalized reduced gradient method. A stepwise variable search algorithm for covariate selection is included in the program. Output features include: model selection criteria, standard errors of parameter estimates, quantile and survival rates with their standard errors, residuals and several plots. An example based on data from Princess Margaret Hospital, Toronto, is discussed to illustrate the program's capabilities.     

基于最大伪似然准则估计的故障链路诊断*

识别网络内部的故障链路对提升网络性能具有重要参考价值。研究了树型拓扑下基于端到端测量的故障链路诊断问题,提出一种最大伪似然估计方法估计链路先验故障概率,把树型拓扑划分为一系列具有两个叶节点的子树,并使用期望最大化(EM)算法最大化每个子树的似然函数,求出链路先验概率。仿真实验表明,该方法与现有的联立方程组求解方法估计精度相当,但是大大降低了算法时间复杂度,证明了该方法的有效性。     

Intrinsic dimension estimation by maximum likelihood in isotropic probabilistic PCA

A central issue in dimension reduction is choosing a sensible number of dimensions to be retained. This work demonstrates the surprising result of the asymptotic consistency of the maximum likelihood criterion for determining the intrinsic dimension of a dataset in an isotropic version of probabilistic principal component analysis (PPCA). Numerical experiments on simulated and real datasets show that the maximum likelihood criterion can actually be used in practice and outperforms existing intrinsic dimension selection criteria in various situations. This paper exhibits and outlines the limits of the maximum likelihood criterion. It leads to recommend the use of the AIC criterion in specific situations. A useful application of this work would be the automatic selection of intrinsic dimensions in mixtures of isotropic PPCA for classification.     

Maximum likelihood estimation for constrained parameters of multinomial distributions—Application to Zipf-Mandelbrot models

A numerical maximum likelihood (ML) estimation procedure is developed for the constrained parameters of multinomial distributions. The main difficulty involved in computing the likelihood function is the precise and fast determination of the multinomial coefficients. For this the coefficients are rewritten into a telescopic product. The presented method is applied to the ML estimation of the Zipf-Mandelbrot (ZM) distribution, which provides a true model in many real-life cases. The examples discussed arise from ecological and medical observations. Based on the estimates, the hypothesis that the data is ZM distributed is tested using a chi-square test. The computer code of the presented procedure is available on request by the author.     

Causal effects in longitudinal studies: Definition and maximum likelihood estimation

Recently, a nonparametric marginal structural model (NPMSM) approach to Causal Inference has been proposed [Neugebauer, R., van der Laan, M., 2006. Nonparametric causal effects based on marginal structural models. J. Statist. Plann. Inference (in press), 〈www http://www.sciencedirect.com/science/journal/03783758〉.] as an appealing practical alternative to the original parametric MSM (PMSM) approach introduced by Robins [Robins, J., 1998a. Marginal structural models. In: 1997 Proceedings of the American Statistical Association, American Statistical Association, Alexandria, VA, pp. 1-10]. The new MSM-based causal inference methodology generalizes the concept of causal effects: the proposed nonparametric causal effects are interpreted as summary measures of the causal effects defined with PMSMs. In addition, causal inference with NPMSM does not rely on the assumed correct specification of a parametric MSM but instead defines causal effects based on a user-specified working causal model which can be willingly misspecified. The NPMSM approach was developed for studies with point treatment data or with longitudinal data where the outcome is not time-dependent (typically collected at the end of data collection). In this paper, we generalize this approach to longitudinal studies where the outcome is time-dependent, i.e. collected throughout the span of the studies, and address the subsequent estimation inconsistency which could easily arise from a hasty generalization of the algorithm for maximum likelihood estimation. More generally, we provide an overview of the multiple causal effect representations which have been developed based on MSMs in longitudinal studies.     

基于无线传感器网络的极大似然定位法的分析

对现在普遍应用的极大似然估计定位方法进行了分析,发现极大似然估计法参考方程(第n个方程)的锚节点(参照锚节点)测距误差和参照锚节点在定位区域内的位置对定位误差有重要影响:参照锚节点的测距误差越小,计算得到的定位精度越高;当所有方程所含误差相同时,参照锚节点在定位区域中心时定位误差小,在定位区域边缘时定位误差大;当参照锚...