Expectation-maximization algorithms for inference in Dirichlet processes mixture

Mixture models are ubiquitous in applied science. In many real-world applications, the number of mixture components needs to be estimated from the data. A popular approach consists of using information criteria to perform model selection. Another approach which has become very popular over the past few years consists of using Dirichlet processes mixture (DPM) models. Both approaches are computationally intensive. The use of information criteria requires computing the maximum likelihood parameter estimates for each candidate model whereas DPM are usually trained using Markov chain Monte Carlo (MCMC) or variational Bayes (VB) methods. We propose here original batch and recursive expectation-maximization algorithms to estimate the parameters of DPM. The performance of our algorithms is demonstrated on several applications including image segmentation and image classification tasks. Our algorithms are computationally much more efficient than MCMC and VB and outperform VB on an example.     

后向散射系数的贝叶斯估计

后向散射系数是合成孔径雷达图像中重要的物理参数.由于合成孔径雷达测量系统的噪声干扰和其他不确定因素影响使得测量数据往往不够精确,这就需要对测量数据进行合理估计.为了对后向散射系数做出准确合理的估计,文章将后向散射系数的先验知识考虑进去,给出了后向散射系数的三种贝叶斯估计算法.贝叶斯估计的关键是概率密度模型的选取.例中选用贝塔(Beta)分布作为先验概率密度模型,伽玛(Gamma)分布作为条件概率密度模型得到了合理的估计结果,并与最大似然估计(ML)算法进行了比较,比较结果表明在对后向散射系数的估计中,贝叶斯估计算法要明显优于最大似然估计算法.     

Independent factor discriminant analysis

In the general classification context the recourse to the so-called Bayes decision rule requires to estimate the class conditional probability density functions. A mixture model for the observed variables which is derived by assuming that the data have been generated by an independent factor model is proposed. Independent factor analysis is in fact a generative latent variable model whose structure closely resembles the one of the ordinary factor model, but it assumes that the latent variables are mutually independent and not necessarily Gaussian. The method therefore provides a dimension reduction together with a semiparametric estimate of the class conditional probability density functions. This density approximation is plugged into the classic Bayes rule and its performance is evaluated both on real and simulated data.     

TOA source localization and DOA estimation algorithms using prior distribution for calibrated source

This paper presents an a priori probability density function (pdf)-based time-of-arrival (TOA) source localization algorithms. Range measurements are used to estimate the location parameter for TOA source localization. Previous information on the position of the calibrated source is employed to improve the existing likelihood-based localization method. The cost function where the prior distribution was combined with the likelihood function is minimized by the adaptive expectation maximization (EM) and space-alternating generalized expectation–maximization (SAGE) algorithms. The variance of the prior distribution does not need to be known a priori because it can be estimated using Bayes inference in the proposed adaptive EM algorithm. Note that the variance of the prior distribution should be known in the existing three-step WLS method [1]. The resulting positioning accuracy of the proposed methods was much better than the existing algorithms in regimes of large noise variances. Furthermore, the proposed algorithms can also effectively perform the localization in line-of-sight (LOS)/non-line-of-sight (NLOS) mixture situations.     

A Bayesian hybrid Huberized support vector machine and its applications in high-dimensional medical data

A hybrid Huberized support vector machine (HHSVM) with an elastic-net penalty has been developed for cancer tumor classification based on thousands of gene expression measurements. In this paper, we develop a Bayesian formulation of the hybrid Huberized support vector machine for binary classification. For the coefficients of the linear classification boundary, we propose a new type of prior, which can select variables and group them together simultaneously. Our proposed prior is a scale mixture of normal distributions and independent gamma priors on a transformation of the variance of the normal distributions. We establish a direct connection between the Bayesian HHSVM model with our special prior and the standard HHSVM solution with the elastic-net penalty. We propose a hierarchical Bayes technique and an empirical Bayes technique to select the penalty parameter. In the hierarchical Bayes model, the penalty parameter is selected using a beta prior. For the empirical Bayes model, we estimate the penalty parameter by maximizing the marginal likelihood. The proposed model is applied to two simulated data sets and three real-life gene expression microarray data sets. Results suggest that our Bayesian models are highly successful in selecting groups of similarly behaved important genes and predicting the cancer class. Most of the genes selected by our models have shown strong association with well-studied genetic pathways, further validating our claims.     

Recursive algorithms for Bayes smoothing with uncertain observations

Recursive algorithms for the Bayes solution of fixed-interval, fixed-point, and fixed-lag smoothing under uncertain observations are presented. The Bayes smoothing algorithms are obtained for a Markovian system model with Markov uncertainty, a model more general than the one used in linear smoothing algorithms. The Bayes fixed-interval smoothing algorithm is applied to a Gauss-Markov example. The simulation results for this example indicate that the MSE performance of the Bayes smoother is significantly better than that of the linear smoother.     

Bayesian analysis of stochastic volatility models with mixture-of-normal distributions

Stochastic volatility (SV) models usually assume that the distribution of asset returns conditional on the latent volatility is normal. This article analyzes SV models with a mixture-of-normal distributions in order to compare with other heavy-tailed distributions such as the Student-t distribution and generalized error distribution (GED). A Bayesian method via Markov-chain Monte Carlo (MCMC) techniques is used to estimate parameters and Bayes factors are calculated to compare the fit of distributions. The method is illustrated by analyzing daily data from the Yen/Dollar exchange rate and the Tokyo stock price index (TOPIX). According to Bayes factors, we find that while the t distribution fits the TOPIX better than the normal, the GED and the normal mixture, the mixture-of-normal distributions give a better fit to the Yen/Dollar exchange rate than other models. The effects of the specification of error distributions on the Bayesian confidence intervals of future returns are also examined. Comparison of SV with GARCH models shows that there are cases that the SV model with the normal distribution is less effective to capture leptokurtosis than the GARCH with heavy-tailed distributions.     

基于混合概率模型的无监督离散化算法

现实应用中常常涉及许多连续的数值属性，而且前许多机器学习算法则要求所处理的属性取离散值，根据在对数值属性的离散化过程中，是否考虑相关类别属性的值，离散化算法可分为有监督算法和无监督算法两类。基于混合概率模型，该文提出了一种理论严格的无监督离散化算法，它能够在无先验知识，无类别是属性的前提下，将数值属性的值域划分为若干子区间，再通过贝叶斯信息准则自动地寻求最佳的子区间数目和区间划分方法。     

Latent Classification Models

One of the simplest, and yet most consistently well-performing set of classifiers is the Naïve Bayes models. These models rely on two assumptions: (i) All the attributes used to describe an instance are conditionally independent given the class of that instance, and (ii) all attributes follow a specific parametric family of distributions. In this paper we propose a new set of models for classification in continuous domains, termed latent classification models. The latent classification model can roughly be seen as combining the Naïve Bayes model with a mixture of factor analyzers, thereby relaxing the assumptions of the Naïve Bayes classifier. In the proposed model the continuous attributes are described by a mixture of multivariate Gaussians, where the conditional dependencies among the attributes are encoded using latent variables. We present algorithms for learning both the parameters and the structure of a latent classification model, and we demonstrate empirically that the accuracy of the proposed model is significantly higher than the accuracy of other probabilistic classifiers.Editors: Pedro Larrañaga, Jose A. Lozano, Jose M. Peña and Iñaki Inza     

Kernel mixture model for probability density estimation in Bayesian classifiers

Estimating reliable class-conditional probability is the prerequisite to implement Bayesian classifiers, and how to estimate the probability density functions (PDFs) is also a fundamental problem for other probabilistic induction algorithms. The finite mixture model (FMM) is able to represent arbitrary complex PDFs by using a mixture of mutimodal distributions, but it assumes that the component mixtures follows a given distribution, which may not be satisfied for real world data. This paper presents a non-parametric kernel mixture model (KMM) based probability density estimation approach, in which the data sample of a class is assumed to be drawn by several unknown independent hidden subclasses. Unlike traditional FMM schemes, we simply use the k-means clustering algorithm to partition the data sample into several independent components, and the regional density diversities of components are combined using the Bayes theorem. On the basis of the proposed kernel mixture model, we present a three-step Bayesian classifier, which includes partitioning, structure learning, and PDF estimation. Experimental results show that KMM is able to improve the quality of estimated PDFs of conventional kernel density estimation (KDE) method, and also show that KMM-based Bayesian classifiers outperforms existing Gaussian, GMM, and KDE-based Bayesian classifiers.