Recursive maximum likelihood parameter estimation for state space systems using polynomial chaos theory

This paper combines polynomial chaos theory with maximum likelihood estimation for a novel approach to recursive parameter estimation in state-space systems. A simulation study compares the proposed approach with the extended Kalman filter to estimate the value of an unknown damping coefficient of a nonlinear Van der Pol oscillator. The results of the simulation study suggest that the proposed polynomial chaos estimator gives comparable results to the filtering method but may be less sensitive to user-defined tuning parameters. Because this recursive estimator is applicable to linear and nonlinear dynamic systems, the authors portend that this novel formulation will be useful for a broad range of estimation problems.     

基于变方差概率模型和进化计算的WSN 定位算法

定位是无线传感器网络技术和应用的重要基础.基于接收信号强度(received signal strength,简称RSS)的定位方法是实际应用中比较重要的定位方法考虑到实际应用中不同地点RSS测量信号的方差有所不同这一特点,运用最大概率似然理论,建立了更加符合实际的基于RSS测量的概率定位模型.对于模型中目标表达式高度非线性不好求解的特点,运用进化计算理论设计出符合传感器通信特征的定位算法(location in probability maximum with evolutionary algorithm,简称PMEA)求解概率可能性最大的位置坐标点,并用随机过程在数学上证明了算法的收敛性.最后,通过对实际公开数据集的实验,证实所提出的概率模型和PMEA算法确实能够提高RSS测距定位的精度.     

Piecewise Linear Models with Guaranteed Closeness to the Data

This paper addresses the problem of piecewise linear approximation of point sets without any constraints on the order of data points or the number of model components (line segments). We point out two problems with the maximum likelihood estimate (MLE) that present serious drawbacks in practical applications. One is that the parametric models obtained using a classical MLE framework are not guaranteed to be close to data points. It is typically impossible, in this classical framework, to detect whether a parametric model fits the data well or not. The second problem is related to accurately choosing the optimal number of model components. We first fit a nonparametric density to the data points and use it to define a neighborhood of the data. Observations inside this neighborhood are deemed informative; those outside the neighborhood are deemed uninformative for our purpose. This provides us with a means to recognize when models fail to properly fit the data. We then obtain maximum likelihood estimates by optimizing the Kullback-Leibler Divergence (KLD) between the nonparametric data density restricted to this neighborhood and a mixture of parametric models. We prove that, under the assumption of a reasonably large sample size, the inferred model components are close to their ground-truth model component counterparts. This holds independently of the initial number of assumed model components or their associated parameters. Moreover, in the proposed approach, we are able to estimate the number of significant model components without any additional computation.     

Approximate inference of the bandwidth in multivariate kernel density estimation

Kernel density estimation is a popular and widely used non-parametric method for data-driven density estimation. Its appeal lies in its simplicity and ease of implementation, as well as its strong asymptotic results regarding its convergence to the true data distribution. However, a major difficulty is the setting of the bandwidth, particularly in high dimensions and with limited amount of data. An approximate Bayesian method is proposed, based on the Expectation-Propagation algorithm with a likelihood obtained from a leave-one-out cross validation approach. The proposed method yields an iterative procedure to approximate the posterior distribution of the inverse bandwidth. The approximate posterior can be used to estimate the model evidence for selecting the structure of the bandwidth and approach online learning. Extensive experimental validation shows that the proposed method is competitive in terms of performance with state-of-the-art plug-in methods.     

高维数据固有维数的自适应极大似然估计

如何确定高维数据的固有维数是降维成功与否的关键。基于极大似然估计（MLE）的维数估计方法是一种新近出现的方法,实现简单,选择合适的近邻能取得不错的结果。但当近邻数过小或过大时,均有比较明显的偏差。其根本原因是没有考虑每个点对固有维数的不同贡献。在充分考虑数据集的分布信息之后,提出了一种改进的MLE——自适应极大似然估计(AMLE)。实验表明,无论在合成数据集还是真实数据集上,AMLE较MLE在估计准确度上均有很大的提高,对近邻数的变化也不甚敏感。     

基于自适应密度估计的系统参数辨识

针对噪声分布未知的ARMAX系统,提出了一种自适应非参数噪声密度估计方法,由估计误差动态调整高斯核函数的全局带宽和局部带宽,实现了未知噪声分布密度的自适应估计;通过极小化似然函数,给出了基于噪声密度估计的参数辨识迭代算法,分析了算法的收敛性并给出了算法收敛的充分条件.仿真结果表明本文提出的算法在系统噪声未知时具有较强的抗噪能力和良好的收敛性.     

一种新的软件可靠性模型参数估计方法

一个软件可靠性模型建立以后,需要对模型的参数进行估计,而参数估计的准确程度将直接影响到模型的预测能力。现在对参数的估计一般采用极大似然估计法。文中提出一种加权最小二乘法,根据各故障数据点对预测贡献值的不同,给与相应的权重。并以一种重要的GO软件可靠性模型为例进行分析,实验表明,该方法获得的参数模型具有更好的预测能力。     

Stepwise variable selection and maximum likelihood estimation of smoothing factors of kernel functions for nonparametric discriminant functions evaluated by different criteria

Kernel functions are used to estimate the probability density functions of variables for nonparametric discriminant analysis. In connection with stepwise variable identification a stepwise maximum likelihood estimation procedure for the estimation of smoothing factors of the kernel functions is developed. This procedure allows a step-by-step estimation of smoothing factors for every variable which is considered to be added to the model or which is examined to substitute a variable in a model. Different criteria for model evaluation in stepwise discriminant analysis are discussed. Beside criteria, like distance and dependence functions and the error and nonerror rate, a criterion which considers the ratio of probability densities of different classes at point x is proposed for stepwise variable identification. An application of the procedures described in this study to a medical decision problem shows the importance of stepwise parameter estimation of kernel functions for nonparametric discriminant analysis and the role of different model evaluation criteria for the selection of the best subset of variables.     

一种改进的期望最大图像分割方法

期望最大算法是进行极大似然估计的一种有效方法,它主要用于观测数据不完全或者似然函数不是解析时的参数估计。文中提出了一种期望最大化和贝叶斯信息准则相结合的图像分割方法。首先,运用K均值方法初始化图像分布;然后,运用期望最大算法估计输入图像参数数据,图像中类的数目由贝叶斯消息准则自动确定;最后,运用最大似然标准将像素归类于最相近的类中。实验中将此方法用于对葡萄叶部病害彩色图像的分割,其结果表明此方法有效。     

Robust diagnostics for the heteroscedastic regression model

The assumption of equal variance in the normal regression model is not always appropriate. Cook and Weisberg (1983) provide a score test to detect heteroscedasticity, while Patterson and Thompson (1971) propose the residual maximum likelihood (REML) estimation to estimate variance components in the context of an unbalanced incomplete-block design. REML is often preferred to the maximum likelihood estimation as a method of estimating covariance parameters in a linear model. However, outliers may have some effect on the estimate of the variance function. This paper incorporates the maximum trimming likelihood estimation ( [Hadi and Luce?o, 1997] and [Vandev and Neykov, 1998]) in REML to obtain a robust estimation of modelling variance heterogeneity. Both the forward search algorithm of Atkinson (1994) and the fast algorithm of Neykov et al. (2007) are employed to find the resulting estimator. Simulation and real data examples are used to illustrate the performance of the proposed approach.     

Collaborative target tracking in WSNs using the combination of maximum likelihood estimation and Kalman filtering

Target tracking using wireless sensor networks requires efficient collaboration among sensors to tradeoff between energy consumption and tracking accuracy. This paper presents a collaborative target tracking approach in wireless sensor networks using the combination of maximum likelihood estimation and the Kalman filter. The cluster leader converts the received nonlinear distance measurements into linear observation model and approximates the covariance of the converted measurement noise using maximum likelihood estimation, then applies Kalman filter to recursively update the target state estimate using the converted measurements. Finally, a measure based on the Fisher information matrix of maximum likelihood estimation is used by the leader to select the most informative sensors as a new tracking cluster for further tracking. The advantages of the proposed collaborative tracking approach are demonstrated via simulation results.     

Assessing the performance of normal-based and REML-based confidence intervals for the intraclass correlation coefficient

Using normal distribution assumptions, one can obtain confidence intervals for variance components in a variety of applications. A normal-based interval, which has exact coverage probability under normality, is usually constructed from a pivot so that the endpoints of the interval depend on the data as well as the distribution of the pivotal quantity. Alternatively, one can employ a point estimation technique to form a large-sample (or approximate) confidence interval. A commonly used approach to estimate variance components is the restricted maximum likelihood (REML) method. The endpoints of a REML-based confidence interval depend on the data and the asymptotic distribution of the REML estimator. In this paper, simulation studies are conducted to evaluate the performance of the normal-based and the REML-based intervals for the intraclass correlation coefficient under non-normal distribution assumptions. Simulated coverage probabilities and expected lengths provide guidance as to which interval procedure is favored for a particular scenario. Estimating the kurtosis of the underlying distribution plays a central role in implementing the REML-based procedure. An empirical example is given to illustrate the usefulness of the REML-based confidence intervals under non-normality.     

含时延和丢失数据不确定系统的FIR滤波器

针对状态空间模型中存在服从伯努利分布的时延和随机观测丢失的情况,基于极大似然法则,分别设计有限脉冲响应(finite impulse response, FIR)滤波器的慢速率批处理形式和快速率迭代形式.首先,将时延和数据丢失情况下的模型表述为服从伯努利分布的概率线性函数;然后,通过极大似然处理从而得到所提出极大似然FIR算法;最后,将在相同条件下的极大似然FIR估计、改进型卡尔曼滤波以及无偏FIR估计3种滤波方法进行对比,从估计误差、均方根误差和不确定性影响等角度进行比较分析.实验部分通过3-DOF直升机模型仿真,可发现所提出极大似然FIR估计方法在处理时延和数据丢失问题时更加有效,鲁棒性更高.     

Application of optimization heuristics for complex astronomical object model identification

Detection and localization of astronomical objects are two of the most fundamental topics in astronomical science where localization uses detection results. Object localization is based on modeling of point spread function and estimation of its parameters. Commonly used models as Gauss or Moffat in objects localization provide good approximation of analyzed objects but cannot be sufficient in the case of exact applications such as object energy estimation. Thus the use of sophisticated models is upon the place. One of the key roles plays also the way of the objective function estimation. The least square method is often used, but it expects data with normal distribution, thus there is a question of a maximum likelihood method application. Another important factor of presented problem is choice of the right optimization method. Classical methods for objective function minimization usually require a good initial estimate for all parameters and differentiation of the objective function with respect to model parameters. The results indicated that stochastic methods such as simulated annealing or harmony search achieved better results than the classical optimization methods.     

带测量误差的非线性退化过程建模与剩余寿命估计

剩余寿命(Remaining useful lifetime, RUL)估计是设备视情维护和预测与健康管理(Prognostics and health management, PHM)中的一项关键问题. 采用退化过程建模进行剩余寿命估计的研究中,现有方法仅考虑了具有线性或可以线性化的退化轨迹的问题.本 文提出了一种基于扩散过程的非线性退化过程建模方法,在首达时间的意义下,推导出了剩余寿命的分布.该方法可以描述一般的非线性退化轨迹, 现有的线性退化建模方法是其特例.在参数的推断中,考虑到真实的退化过程受到测量误差的影响,难以直接测量得到, 因此,在退化建模的过程中引入了测量误差对退化观测数据的影响,通过观测数据,提出了一种退化模型未知参数的极大似然估计方法. 最后,通过激光发生器和陀螺仪的退化测量数据验证了本文方法明显优于线性建模方法,具有潜在的工程应用价值.     

Computing all roots of the likelihood equations of seemingly unrelated regressions

Seemingly unrelated regressions are statistical regression models based on the Gaussian distribution. They are popular in econometrics but also arise in graphical modeling of multivariate dependencies. In maximum likelihood estimation, the parameters of the model are estimated by maximizing the likelihood function, which maps the parameters to the likelihood of observing the given data. By transforming this optimization problem into a polynomial optimization problem, it was recently shown that the likelihood function of a simple bivariate seemingly unrelated regressions model may have several stationary points. Thus local maxima may complicate maximum likelihood estimation. In this paper, we study several more complicated seemingly unrelated regression models, and show how all stationary points of the likelihood function can be computed using algebraic geometry.     

Fuzzy clustering to estimate the parameters of block mixture models

Finite mixture models are being increasingly used to provide model-based cluster analysis. To tackle the problem of block clustering which aims to organize the data into homogeneous blocks, recently we have proposed a block mixture model; we have considered this model under the classification maximum likelihood approach and we have developed a new algorithm for simultaneous partitioning based on the classification EM algorithm. From the estimation point of view, classification maximum likelihood approach yields inconsistent estimates of the parameters and in this paper we consider the block clustering problem under the maximum likelihood approach; unfortunately, the application of the classical EM algorithm for the block mixture model is not direct: difficulties arise due to the dependence structure in the model and approximations are required. Considering the block clustering problem under a fuzzy approach, we propose a fuzzy block clustering algorithm to approximate the EM algorithm. To illustrate our approach, we study the case of binary data by using a Bernoulli block mixture.     

核密度估计的聚类算法*

相似性度量是聚类分析的重要基础,如何有效衡量类属型符号间的相似性是相似性度量的一个难点.文中根据离散符号的核概率密度衡量符号间的相似性,与传统的简单符号匹配及符号频度估计方法不同,该相似性度量在核函数带宽的作用下,不再依赖同一属性上符号间独立性假设.随后建立类属型数据的贝叶斯聚类模型,定义基于似然的类属型对象-簇间相似性度量,给出基于模型的聚类算法.采用留一估计和最大似然估计,提出3种求解方法在聚类过程中动态确定最优的核带宽.实验表明,相比使用特征加权或简单匹配距离的聚类算法,文中算法可以获得更高的聚类精度,估计的核函数带宽在重要特征识别等应用中具有实际意义.     

Maximum likelihood kernel density estimation: On the potential of convolution sieves

Methods for improving the basic kernel density estimator include variable locations, variable bandwidths and variable weights. Typically these methods are implemented separately and via pilot estimation of variation functions derived from asymptotic considerations. The starting point here is a simple maximum likelihood procedure which allows (in its greatest generality) variation of all these quantities at once, bypassing asymptotics and explicit pilot estimation. One special case of this approach is the density estimator associated with nonparametric maximum likelihood estimation (NPMLE) in a normal location mixture model. Another, closely associated with the NPMLE, is a kernel convolution sieve estimator proposed in 1982 but little used in practice to date. Simple algorithms are utilised, a simulation study is reported on, a method for bandwidth selection is investigated and an illustrative example is given. The simulations and other considerations suggest that the kernel convolution sieve provides an especially promising framework for further practical utilisation and development. And the method has a further advantage: it automatically reduces, where appropriate, to a few-component mixture model which indicates and initialises parametric mixture modelling of the data.     

Robust accelerated failure time regression

Robust estimators for accelerated failure time models with asymmetric (or symmetric) error distribution and censored observations are proposed. It is assumed that the error model belongs to a log-location-scale family of distributions and that the mean response is the parameter of interest. Since scale is a main component of mean, scale is not treated as a nuisance parameter. A three steps procedure is proposed. In the first step, an initial high breakdown point S estimate is computed. In the second step, observations that are unlikely under the estimated model are rejected or down weighted. Finally, a weighted maximum likelihood estimate is computed. To define the estimates, functions of censored residuals are replaced by their estimated conditional expectation given that the response is larger than the observed censored value. The rejection rule in the second step is based on an adaptive cut-off that, asymptotically, does not reject any observation when the data are generated according to the model. Therefore, the final estimate attains full efficiency at the model, with respect to the maximum likelihood estimate, while maintaining the breakdown point of the initial estimator. Asymptotic results are provided. The new procedure is evaluated with the help of Monte Carlo simulations. Two examples with real data are discussed.