Competitive EM algorithm for finite mixture models

In this paper, we present a novel competitive EM (CEM) algorithm for finite mixture models to overcome the two main drawbacks of the EM algorithm: often getting trapped at local maxima and sometimes converging to the boundary of the parameter space. The proposed algorithm is capable of automatically choosing the clustering number and selecting the “split” or “merge” operations efficiently based on the new competitive mechanism we propose. It is insensitive to the initial configuration of the mixture component number and model parameters.Experiments on synthetic data show that our algorithm has very promising performance for the parameter estimation of mixture models. The algorithm is also applied to the structure analysis of complicated Chinese characters. The results show that the proposed algorithm performs much better than previous methods with slightly heavier computation burden.     

一种半监督机器学习的EM算法改进方法

EM(Expectation Maximization)算法是含有隐变量(latent variable)的概率参数模型最大似然估计、极大后验概率估计最有效的算法,但很容易进入局部最优现象,对此提出基于半监督机器学习机制的EM算法.本文方法是在最大似然函数中加入惩罚最小二乘因子,同时引入非负约束作为先验信息,结合半监督机器学习方法,将EM算法改进转化为最小化求解问题,再采用最大似然方法求解EM模型,有效估计了混合矩阵和高斯混合模型参数,实现EM算法的改进.仿真结果表明,该方法能够很好地解决了EM算法容易局部最优化问题.     

融入邻域作用的高斯混合分割模型及简化求解

目的 基于高斯混合模型（GMM）的图像分割方法易受噪声影响,为此采用马尔可夫随机场（MRF）将像素邻域关系引入GMM,提高算法抗噪性。针对融入邻域作用的高斯混合分割模型结构复杂、参数估计困难,难以获得全局最优分割解等问题,提出一种融入邻域作用的高斯混合分割模型及其简化求解方法。方法 首先,构建融入邻域作用的GMM。为了提高GMM的抗噪性,采用MRF建模混合模型权重系数的先验分布。然后,利用贝叶斯理论建立图像分割模型,即品质函数;由于品质函数中参数较多（包括权重系数,均值,协方差）、函数结构复杂,导致参数求解困难。因此,将品质函数中的均值和协方差定义为权重系数的函数,由此简化模型结构并方便其求解;虽然品质函数中仅包含参数权重系数,但结构比较复杂,难以求得参数的解析式。最后,采用非线性共轭梯度法（CGM）求解参数,该方法仅需利用品质函数值和参数梯度值,降低了参数求解的复杂性,并且收敛快,可以得到全局最优解。结果 为了有效而准确地验证提出的分割方法,分别采用本文算法和对比算法对合成图像和高分辨率遥感图像进行分割实验,并定性和定量地评价和分析了实验结果。实验结果表明本文方法的有效抗噪性,并得到很好的分割结果。从参数估计结果可以看出,本文算法有效简化了模型参数,并获得全局最优解。结论 提出一种融入邻域作用的高斯混合分割模型及其简化求解方法,实验结果表明,本文算法提高了算法的抗噪性,有效地简化了模型参数,并得到全局最优参数解。本文算法对具有噪声的高分辨率遥感影像广泛适用。     

基于高斯马尔可夫随机场的MAP估计在高斯噪声滤波中的应用

提出了基于高斯马尔可夫随机场(GMRF)的最大后验概率(MAP)估计在图像高斯噪声滤波中的应用方法。根据高斯噪声的先验特点,建立基于高斯马尔可夫随机场的退化图像恢复模型,从而将图像高斯噪声滤波问题转化为求解最大后验概率问题。先验概率可以根据马尔可夫随机场(MRF)和吉布斯分布(GD)的等效性, 用GD的概率估计。为了求解最大后验概率,第一,通过期望最大化(EM）算法对GMRF模型进行参数估计。第二,用共轭梯度法将目标函数最小化。实验结果表明,与其他滤波器（如高斯滤波、维纳滤波等）相比,本文所阐述的方法在滤除高斯噪声、保持图像原有结构方面效果更好。     

基于高斯混合模型聚类的非均匀采样系统的多模型切换辨识

从概率统计方法出发,提出一种基于高斯混合模型聚类与递推最小二乘算法的非均匀采样系统的多模型建模方法.首先,采用高斯混合模型作为调度函数,使用最大期望(EM)算法迭代更新估计高斯混合模型中参数,从而通过每个子系统的高斯概率密度函数计算和比较来确定子系统的激活情况; 其次,采用递推最小二乘算法估计局部子系统参数;然后,使用鞅收敛定理对所提出的算法性能进行分析; 最后,通过非均匀采样系统的多模型建模来证明所提出方法的有效性.     

基于高斯混合模型的海面运动目标检测

提出了一种基于变化检测的高斯混合模型参数估计方法,建立了象素点背景模型并用于海面运动目标的检测。在实验部分,将该方法估计的高斯混合背景模型的参数与基于迭代的EM算法估计的模型参数做比较,模拟实验的结果表明两者估计的参数值相差不大,而在对视频流中的象素点灰度值分布的逼近中,该文的方法比EM算法更接近真实的分布,并且在一定程度上减少了建立背景模型的所需的内存和计算时间。运动目标检测的结果表明,使用该方法建立的背景模型可以比较准确地检测到海面上的运动船只。     

嵌套删失数据期望最大化的高斯混合聚类算法

针对聚类问题中的非随机性缺失数据, 本文基于高斯混合聚类模型, 分析了删失型数据期望最大化算法的有效性, 并揭示了删失数据似然函数对模型算法的作用机制. 从赤池弘次信息准则、信息散度等指标, 比较了所提出方法与标准的期望最大化算法的优劣性. 通过删失数据划分及指示变量, 推导了聚类模型参数后验概率及似然函数, 调整了参数截尾正态函数的一阶和二阶估计量. 并根据估计算法的有效性理论, 通过关于得分向量期望的方程得出算法估计的最优参数. 对于同一删失数据集, 所提出的聚类算法对数据聚类中心估计更精准. 实验结果证实了所提出算法在高斯混合聚类的性能上优于标准的随机性缺失数据期望最大化算法.     

基于线性模型的信道质量估计算法

针对自适应卫星通信中对信道质量估计的需求,在加性高斯白噪声信道条件下,以信干噪比(SINR)作为表征信道质量的参数,提出一种信道质量估计算法.给出矩估计法和判决数据估计法的数学分析,利用指数加权因子对噪声加干扰的功率进行平滑.提出基于矩估计和判决数据估计的线性模型对SINR估计的算法,分析该模型的均方误差,同时搜索最佳...     

基于遗传算法的SAR图像多尺度分割

基于最大期望（EM）算法与遗传算法（GA）,提出一种有效的多尺度SAR图像无监督分割方法。该方法首先利用混合多尺度自回归（MMAR）模型描述SAR图像中由于雷达斑点所引起的不同尺度和同一尺度内像素之间的统计相依性; 然后将GA与EM结合给出MMAR模型的参数估计算法。这种算法利用最小描述长度（MDL）准则,能够选择模型的分量数;最后利用Bayes分类器实现图像的分割。该方法集遗传算法和EM算法的优点,对初始值有较少的敏感性,避免局部最优解,提高了分割精度。实验结果表明GA EM方法优于EM算法。     

基于自适应表面模型的概率视频跟踪算法

提出一种鲁棒自适应表面模型,该模型中每个像素值的变化过程由一混合高斯分布描述．为了适应目标表面的变化,这些高斯参数在跟踪期间通过在线的EM算法自适应更新;在估计目标状态时。采用了粒子滤波算法。设计了基于自适应表面模型的观测模型;在处理遮挡时,采用了一种鲁棒估计技术．多组试验结果表明,该算法对光照变化、姿态变化、部分或完全遮挡下的跟踪具有较强的鲁棒性．     

Surrogate modeling approximation using a mixture of experts based on EM joint estimation

An automatic method to combine several local surrogate models is presented. This method is intended to build accurate and smooth approximation of discontinuous functions that are to be used in structural optimization problems. It strongly relies on the Expectation−Maximization (EM) algorithm for Gaussian mixture models (GMM). To the end of regression, the inputs are clustered together with their output values by means of parameter estimation of the joint distribution. A local expert is then built (linear, quadratic, artificial neural network, moving least squares) on each cluster. Lastly, the local experts are combined using the Gaussian mixture model parameters found by the EM algorithm to obtain a global model. This method is tested over both mathematical test cases and an engineering optimization problem from aeronautics and is found to improve the accuracy of the approximation.     

Directionlet-based method using the Gaussian mixture prior to SAR image despeckling

In this article, a new denoising algorithm is proposed based on the directionlet transform and the maximum a posteriori (MAP) estimation. The detailed directionlet coefficients of the logarithmically transformed noise-free image are considered to be Gaussian mixture probability density functions (PDFs) with zero means, and the speckle noise in the directionlet domain is modelled as additive noise with a Gaussian distribution. Then, we develop a Bayesian MAP estimator using these assumed prior distributions. Because the estimator that is the solution of the MAP equation is a function of the parameters of the assumed mixture PDF models, the expectation-maximization (EM) algorithm is also utilized to estimate the parameters, including weight factors and variances. Finally, the noise-free SAR image is restored from the estimated coefficients yielded by the MAP estimator. Experimental results show that the directionlet-based MAP method can be successfully applied to images and real synthetic aperture radar images to denoise speckle.     

Gaussian mixture model classification: A projection pursuit approach

Gaussian mixture models (GMM) are commonly employed in nonparametric supervised classification. In high-dimensional problems it is often the case that information relevant to the separation of the classes is contained in a few directions. A GMM fitting procedure oriented to supervised classification is proposed, with the aim of reducing the number of free parameters. It resorts to projection pursuit as a dimension reduction method and combines it with GM modelling of class-conditional densities. In its derivation, issues regarding the forward and backward projection pursuit algorithms are discussed. The proposed procedure avoids the “curse of dimensionality”, is able to model structure in subspaces and regularizes the classification model. Its performance is illustrated on a simulation experiment and on a real data set, in comparison with other GMM-based classification methods.     

Robust Gaussian process modeling using EM algorithm

Gaussian process (GP) regression is a fully probabilistic method for performing non-linear regression. In a Bayesian framework, regression models can be made robust by using heavy-tailed distributions instead of using normal distribution for modeling noise. This work focuses on estimation of parameters for robust GP regression. In literature, these are learned by maximizing the approximate marginal likelihood of data. However, gradient-based optimization algorithms which are used for this purpose can be unstable or may require tuning. In this work, an EM algorithm based approach is derived and implemented to infer the parameters. The pros and cons of the two approaches are analyzed. The advantage of EM algorithm lies in its ease of implementation and theoretical guarantees of numerical stability and convergence while its prediction performance is still comparable to gradient-based approaches. In some cases EM algorithm may be slow to converge. To circumvent this issue a faster EM based approach known as Expectation Conjugate Gradient (ECG) is implemented on robust GP regression. Finally, the proposed EM approach to robust GP regression is validated using an industrial data set.     

空间约束层次加权Gamma混合模型的SAR图像分割

目的 合成孔径雷达（SAR）图像中像素强度统计分布呈现出复杂的特性,而传统混合模型难以建模非对称、重尾或多峰等特性的分布。为了准确建模SAR图像统计分布并得到高精度分割结果,本文提出一种利用空间约束层次加权Gamma混合模型（HWGaMM）的SAR图像分割算法。方法 采用Gamma分布的加权和定义混合组份;考虑到同质区域内像素强度的差异性和异质区域间像素强度的相似性,采用混合组份加权和定义HWGaMM结构。采用马尔可夫随机场（MRF）建模像素空间位置关系,利用中心像素及其邻域像素的后验概率定义混合权重以将像素邻域关系引入HWGaMM,构建空间约束HWGaMM,以降低SAR图像内固有斑点噪声的影响。提出算法结合M-H（Metropolis-Hastings）和期望最大化算法（EM）求解模型参数,以实现快速SAR图像分割。该求解方法避免了M-H算法效率低的缺陷,同时克服了EM算法难以求解Gamma分布中形状参数的问题。结果 采用3种传统混合模型分割算法作为对比算法进行分割实验。拟合直方图结果表明本文算法具有准确建模复杂统计分布的能力。在分割精度上,本文算法比基于高斯混合模型（GMM）、Gamma分布和Gamma混合模型（GaMM）分割算法分别提高33%,29%和9%。在分割时间上,本文算法虽然比GMM算法多64 s,但与基于Gamma分布和GaMM算法相比较分别快600 s和420 s。因此,本文算法比传统M-H算法的分割效率有很大的提高。结论 提出一种空间约束HWGaMM的SAR图像分割算法,实验结果表明提出的HWGaMM算法具有准确建模复杂统计分布的能力,且具有较高的精度和效率。     

Algorithms for deterministic balanced subspace identification

New algorithms for identification of a balanced state space representation are proposed. They are based on a procedure for the estimation of impulse response and sequential zero input responses directly from data. The proposed algorithms are more efficient than the existing alternatives that compute the whole Hankel matrix of Markov parameters. It is shown that the computations can be performed on Hankel matrices of the input-output data of various dimensions. By choosing wider matrices, we need persistency of excitation of smaller order. Moreover, this leads to computational savings and improved statistical accuracy when the data is noisy. Using a finite amount of input-output data, the existing algorithms compute finite time balanced representation and the identified models have a lower bound on the distance to an exact balanced representation. The proposed algorithm can approximate arbitrarily closely an exact balanced representation. Moreover, the finite time balancing parameter can be selected automatically by monitoring the decay of the impulse response. We show what is the optimal in terms of minimal identifiability condition partition of the data into “past” and “future”.     

双重高斯混合模型的EM算法的聚类问题研究

EM算法是参数估计的重要方法,其算法核心是根据已有的数据来迭代计算似然函数,使之收敛于某个最优值.半监督聚类是利用少部分标签的数据辅助大量未标签的数据进行的聚类分析.文章介绍了一种基于双重高斯混合模型的EM算法,在无监督学习中增加一些已标记的样本,利用已标记的样本得到初始参数,研究了半监督条件下的双重高斯混合模型的EM聚类算法.实验表明,该算法较无监督学习而言,提升了样本的识别率,有良好的聚类性能.这种算法模型也可以作为一种基础模型,具有一定的应用领域.     

一种带有色量测噪声的非线性系统辨识方法

利用最大似然判据, 本文提出了一种带有色量测噪声的非线性系统辨识方法. 首先, 利用量测差分方法将有色量测噪声白色化, 获得新的量测方程, 从而将带有色量测噪声的非线性系统辨识问题转化成带白色量测噪声和一步延迟状态的非线性系统辨识问题. 其次, 利用期望最大化(Expectation maximization, EM)算法提出了一种新的基于最大似然估计的非线性系统辨识方法, 该算法由期望步骤(Expectation step, E-step)和最大化步骤(Maximization step, M-step)两部分组成. 在期望步骤中, 基于当前估计的参数并利用带有色量测噪声的高斯近似滤波器和平滑器, 近似计算完整的对数似然函数的期望. 在最大化步骤中, 近似计算的似然函数期望值被最大化, 并且通过解析更新获得噪声参数估计, 通过Newton更新方法获得模型参数的估计. 最后, 数值仿真验证了本文提出算法的有效性.     

FOE estimation: Can image measurement errors be totally “corrected” by the geometric method?

For the FOE estimation, there are basically three kinds of estimation methods in the literature: algebraic, geometric, and the maximum likelihood-based ones. In this paper, our attention is focused on the geometric method. The computational complexity of the classical geometric method is usually very high because it needs to solve a non-linear minimum problem with many variables. In this work, such a minimum problem is converted into an equivalent one with only two variables and accordingly a simplified geometric method is proposed. Based on the equivalence of the classical geometric method and the proposed simplified geometric method, we show that the measurement errors can at most be “corrected” only in one of the two images by geometric methods. In other words, it is impossible to correct the measurement errors in both of the two images. In addition, we show that the “corrected” corresponding pairs by geometric methods cannot in general meet some of the inherent constraints of corresponding pairs under pure camera translations. Hence, it is not proper to consider the “corrected” corresponding pairs as “faithful” corresponding pairs in geometric methods, and the estimated FOE from such pairs is not necessarily trustworthier. Finally, a new geometric algorithm, which automatically enforces the inherent constraints, is proposed in this work, and better FOE estimation and more faithful corresponding pairs are obtained.     

Polarimetric SAR statistical analysis using alpha-stable distribution and its application in optimal despeckling

In this article, the statistical model of the polarimetric synthetic aperture radar (SAR) single-look complex image is analysed using alpha-stable distribution. It is better to use alpha-stable distribution than Gaussian distribution to represent the statistical characteristics of the polarimetric SAR image. A polarimetric SAR covariance matrix estimation method based on fractional lower-order statistics (FLOS) is proposed. Based on this model, an adaptive polarimetric SAR optimal despeckling method based on FLOS is developed. This algorithm adaptively estimates the characteristic exponents of each channel and uses these estimated alphas to calculate the parameters for the optimal despeckling adaptively. The experiments using polarimetric SAR data demonstrate that the proposed method not only reduces the blurs that occur in the area of impulsive reflectors in the result of the original optimal despeckling method, but also maintains the speckle reduction ability (equivalent number of looks).