Probabilistic Modeling and Recognition of 3-D Objects

This paper introduces a uniform statistical framework for both 3-D and 2-D object recognition using intensity images as input data. The theoretical part provides a mathematical tool for stochastic modeling. The algorithmic part introduces methods for automatic model generation, localization, and recognition of objects. 2-D images are used for learning the statistical appearance of 3-D objects; both the depth information and the matching between image and model features are missing for model generation. The implied incomplete data estimation problem is solved by the Expectation Maximization algorithm. This leads to a novel class of algorithms for automatic model generation from projections. The estimation of pose parameters corresponds to a non-linear maximum likelihood estimation problem which is solved by a global optimization procedure. Classification is done by the Bayesian decision rule. This work includes the experimental evaluation of the various facets of the presented approach. An empirical evaluation of learning algorithms and the comparison of different pose estimation algorithms show the feasibility of the proposed probabilistic framework.     

面向大规模数据集的贝叶斯网络参数学习算法

贝叶斯网络的学习可以分为结构学习和参数学习。期望最大化（EM）算法通常用于不完整数据的参数学习,但是由于EM算法计算相对复杂,存在收敛速度慢和容易局部最大化等问题,传统的EM算法难于处理大规模数据集。研究了EM算法的主要问题,采用划分数据块的方法将大规模数据集划分为小的样本集来处理,降低了EM算法的计算量,同时也提高了计算精度。实验证明,该改进的EM算法具有较高的性能。     

Estimating time-varying densities using a stochastic learning automaton

The popular Expectation Maximization technique suffers a major drawback when used to approximate a density function using a mixture of Gaussian components; that is the number of components has to be a priori specified. Also, Expectation Maximization by itself cannot estimate time-varying density functions. In this paper, a novel stochastic technique is introduced to overcome these two limitations. Kernel density estimation is used to obtain a discrete estimate of the true density of the given data. A Stochastic Learning Automaton is then used to select the number of mixture components that minimizes the distance between the density function estimated using the Expectation Maximization and discrete estimate of the density. The validity of the proposed approach is verified using synthetic and real univariate and bivariate observation data.     

信息熵在网络流量矩阵估算中的应用

提出一种网络流量矩阵估算方法,在已知网络拓扑结构和网络链路流量的情况下,根据网络链路流量计算出信息熵,利用期望最大化算法对网络源和目的对之间的流量需求进行估计。以校园网为实验环境,对骨干网络采集网络流量数据,与通用重力模型方法的比较结果表明,利用该方法进行估算有更高的准确性。     

Nested Monte Carlo EM algorithm for switching state-space models

Switching state-space models have been widely used in many applications arising from science, engineering, economic, and medical research. In this paper, we present a Monte Carlo Expectation Maximization (MCEM) algorithm for learning the parameters and classifying the states of a state-space model with a Markov switching. A stochastic implementation based on the Gibbs sampler is introduced in the expectation step of the MCEM algorithm. We study the asymptotic properties of the proposed algorithm, and we also describe how a nesting approach and the Rao-Blackwellized forms can be employed to accelerate the rate of convergence of the MCEM algorithm. Finally, the performance and the effectiveness of the proposed method are demonstrated by applications to simulated and physiological experimental data.     

Variational EM method for blur estimation using the spike-and-slab image prior

Most state-of-the-art blind image deconvolution methods rely on the Bayesian paradigm to model the deblurring problem and estimate both the blur kernel and latent image. It is customary to model the image in the filter space, where it is supposed to be sparse, and utilize convenient priors to account for this sparsity. In this paper, we propose the use of the spike-and-slab prior together with an efficient variational Expectation Maximization (EM) inference scheme to estimate the blur in the image. The spike-and-slab prior, which constitutes the gold standard in sparse machine learning, selectively shrinks irrelevant variables while mildly regularizing the relevant ones. The proposed variational Expectation Maximization algorithm is more efficient than usual Markov Chain Monte Carlo (MCMC) inference and, also, proves to be more accurate than the standard mean-field variational approximation. Additionally, all the prior model parameters are estimated by the proposed scheme. After blur estimation, a non-blind restoration method is used to obtain the actual estimation of the sharp image. We investigate the behavior of the prior in the experimental section together with a series of experiments with synthetically generated and real blurred images that validate the method's performance in comparison with state-of-the-art blind deconvolution techniques.     

Recursive Bayesian Recurrent Neural Networks for Time-Series Modeling

This paper develops a probabilistic approach to recursive second-order training of recurrent neural networks (RNNs) for improved time-series modeling. A general recursive Bayesian Levenberg–Marquardt algorithm is derived to sequentially update the weights and the covariance (Hessian) matrix. The main strengths of the approach are a principled handling of the regularization hyperparameters that leads to better generalization, and stable numerical performance. The framework involves the adaptation of a noise hyperparameter and local weight prior hyperparameters, which represent the noise in the data and the uncertainties in the model parameters. Experimental investigations using artificial and real-world data sets show that RNNs equipped with the proposed approach outperform standard real-time recurrent learning and extended Kalman training algorithms for recurrent networks, as well as other contemporary nonlinear neural models, on time-series modeling.      

相关分块衰落信道下基于因子图EM算法的解码方法

针对相关分块衰落信道模型,提出一种基于因子图期望最大化(FGEM)算法的联合估计解码方法。在接收机中,采用因子图消息传递方法进行信道估计和迭代解码, 并引入期望最大化(EM)算法来消除因子图模型中存在环路对消息传递的影响,同时解决了消息传递中的混合高斯消息计算的问题。采用卡尔曼前后向算法代替最大化步消息更新过程,简化了消息的迭代计算,降低了联合解码和估计的复杂性。实验结果表明,与只有导频辅助方法和判决方法相比,该算法进一步提高了信道估计的准确度和接收机的解码性能。     

A method for mixed states texture segmentation with simultaneous parameter estimation

In this work a method for mixed-state model motion texture segmentation and parameter estimation is presented. We use the Expectation Maximization algorithm for mixture parameter estimation, introducing the Gibbs distribution for moving points, excluding zero discrete component associated with no motion regions. We use then the a posteriori probabilities to generate an alternative field to segment the textures according to its statistical parameters.     

基于梯形模型和支撑向量机的非结构化道路检测

在H.Jeong的梯形模型的基础上,提出了基于梯形模型和支撑向量机——SVM（Support Vector Machine）的道路检测算法。算法先对视频中提取的图像帧进行预处理,然后采用Kalman滤波及EM算法进行处理,接着用SVM得到道路检测结果,并进行滤波处理得到最终的检测结果。由于算法采用了比BP（Back Propagation）网络具有更好的分类识别效果的SVM,所以比采用BP网络的H.Jeong等人提出的模型具有更好的检测效果。该算法在预处理部分采用脉冲耦合神经网络即（PCNN-Pulse Coupled Neural Network）消除道路上的阴影,减少了光照变化对最终检测结果的不利影响。实验表明,与H.Jeong的梯形及BP算法相比,道路的检测效果更好。     

Mixture periodic autoregressive conditional heteroskedastic models

Mixture Periodically Correlated Autoregressive Conditionally Heteroskedastic (MPARCH) model, which extends the ARCH model, is proposed. The primary motivation behind this extension is to make the model consistent with high kurtosis, outliers and extreme events, and at the same time, able to capture the periodicity feature exhibited by the autocovariance structure. The second and the fourth moment periodically stationary conditions and their closed-forms are derived. Maximum likelihood estimation is obtained via the iterative Expectation Maximization algorithm and the performance of this algorithm is shown via a simulation studies and the MPARCH models are fitted to a real data set.     

Maximum likelihood estimation in vector long memory processes via EM algorithm

We present an approach for exact maximum likelihood estimation of parameters from univariate and multivariate autoregressive fractionally integrated moving average models with Gaussian errors using the Expectation Maximization (EM) algorithm. The method takes advantage of the relation between the VARFIMA(0,d,0) process and the corresponding VARFIMA(p,d,q) process in the computation of the likelihood.     

基于EM算法的混合模型医学图像分割

医学图像分割中的期望最大化(EM)算法在求解混合模型参数时存在局限性。为此,提出一种模糊约束的混合模型图像分割算法。该算法以像素的独立性假设为前提,在采用EM算法对模型参数进行求解的过程中,通过模糊集合论方法,引入像素空间信息。实验结果表明,该算法没有引入新的模型参数,能够保持独立混合模型的简单性,且具有自动模型选择能力,可以获得较理想的分割结果。     

Mixture periodic autoregressive conditional heteroskedastic models

Mixture Periodically Correlated Autoregressive Conditionally Heteroskedastic (MPARCH) model, which extends the ARCH model, is proposed. The primary motivation behind this extension is to make the model consistent with high kurtosis, outliers and extreme events, and at the same time, able to capture the periodicity feature exhibited by the autocovariance structure. The second and the fourth moment periodically stationary conditions and their closed-forms are derived. Maximum likelihood estimation is obtained via the iterative Expectation Maximization algorithm and the performance of this algorithm is shown via a simulation studies and the MPARCH models are fitted to a real data set.     

基于MRF模型和EM算法的多源图像融合方法

提出了两种图像融合方法.该方法首先利用EM-MRF算法与模糊分类方法的等价性,将EM-MRF算法引入到图像融合领域.在此基础上,利用统计模型对图像进行非监督分类的模型参数估计转化通过EM算法从不完全数据中估计模型参数的问题,并利用Markov随机场模型建立类别的先验概率、EM迭代算法进行图像分类的方法有较高的分类精度和鲁棒性,导出了基于分布式和集中式多传感器图像融合模型的两种融合方法.最后仿真试验表明,这两种融合方法既可以提高分类精度,又可以加强对噪声的抗干扰能力.     

一类非线性周期时间序列模型

为了描述周期时间序列中的偏倚和多峰等非线性特征,结合有限混合模型方法,提出混合周期自回归滑动平均时间序列模型（MPARMA）,给出了MPARMA模型的平稳性条件,讨论了期望最大化（EM）算法的应用,通过PM10浓度序列分析,评估了MPARMA模型的表现。     

Semi-supervised learning with an imperfect supervisor

Real-life applications may involve huge data sets with misclassified or partially classified training data. Semi-supervised learning and learning in the presence of label noise have recently emerged as new paradigms in the machine learning community to cope with this kind of problems. This paper describes a new discriminant algorithm for semi-supervised learning. This algorithm optimizes the classification maximum likelihood (CML) of a set of labeled–unlabeled data, using a discriminant extension of the Classification Expectation Maximization algorithm. We further propose to extend this algorithm by modeling imperfections in the estimated class labels for unlabeled data. The parameters of this label-error model are learned together with the semi-supervised classifier parameters. We demonstrate the effectiveness of the approach using extensive experiments on different datasets. Massih R. Amini is currently assistant professor in the University of Pierre and Marie Curie (Paris 6). He received an engineering degree in computer science from the Ecole Supérieure d'Informatique (Computer science engineering school) in Paris in 1995. He then accomplished his master thesis in science in artificial intelligence in 1997 and obtained his PhD in 2001 at University of Pierre and Marie Curie. His research interests include Statistical Learning and Text-Mining. Patrick Gallinari is currently professor in the University of Pierre and Marie Curie (Paris 6) and head of the Computer Science laboratory (LIP6). His main research activity has been in the field of statistical machine learning for the last 15 years. He has also contributed in developing machine learning techniques for different application domains like information retrieval and text mining, user modelling, man–machine interaction and pen interfaces.     

一种基于 Bernoulli混合模型的不完整数据文本分类方法

在Bernoulli混合模型和期望最大化(EM)算法的基础上给出了一种基于不完整数据的改进方法。首先在已标记数据的基础上通过Bernoulli混合模型和朴素贝叶斯算法得到似然函数参数估计初始值, 然后利用含有权值的EM算法对分类器的先验概率模型进行参数估计,得到最终的分类器。实验结果表明,该方法在准确率和查全率方面要优于朴素贝叶斯文本分类。     

拉普拉斯混合分布语音模型研究

针对具有超重尾特性的语音信号,提出了混合拉普拉斯分布语音模型。从理论上探讨了混合拉普拉斯分布模型的参数估计,从原理与算法得以实现。通过最大期望（Expectation Maximization,EM）算法取得了良好效果。创新运用混合拉普拉斯模型研究语音信号处理。     

基于贝叶斯网络的航班保障服务时间动态估计

针对航班保障服务时间估计的问题,考虑到航班保障服务流程的特殊性、复杂性以及影响因素的不确定性,提出了一种基于贝叶斯网络（BN）的航班保障服务时间估计模型。该模型把航空领域的专家知识与历史数据的机器学习相结合,使用贝叶斯网络的增量学习特性动态地调整BN模型,使其适应新的变化,进而不断更新航班保障服务时间的估计值。使用国内某大型枢纽机场信息系统内提取的数据,通过期望最大化（EM）方法对模型进行训练,得到了测试结果。实验结果分析与模型评价表明,所提方法能有效估计航班保障服务时间且具有较高的准确度。敏感性分析表明,航班到达时段的航班密度对航班保障服务时间影响最强。