Mixture Poisson regression models for heterogeneous count data based on latent and fuzzy class analysis

In this paper we propose a new approach, called a fuzzy class model for Poisson regression, in the analysis of heterogeneous count data. On the basis of fuzzy set concept and fuzzy classification maximum likelihood (FCML) procedures we create an FCML algorithm for fuzzy class Poisson regression models. Traditionally, the EM algorithm had been used for latent class regression models. Thus, the accuracy and effectiveness of EM and FCML algorithms for estimating the parameters are compared. The results show that the proposed FCML algorithm presents better accuracy and effectiveness and can be used as another good tool to regression analysis for heterogeneous count data.This work was supported in part by the National Science Council of Taiwan under Grant NSC-89-2213-E-033-007.     

Multi-model multivariate Gaussian process modelling with correlated noises

A composite multiple-model approach based on multivariate Gaussian process regression (MGPR) with correlated noises is proposed in this paper. In complex industrial processes, observation noises of multiple response variables can be correlated with each other and process is nonlinear. In order to model the multivariate nonlinear processes with correlated noises, a dependent multivariate Gaussian process regression (DMGPR) model is developed in this paper. The covariance functions of this DMGPR model are formulated by considering the “between-data” correlation, the “between-output” correlation, and the correlation between noise variables. Further, owing to the complexity of nonlinear systems as well as possible multiple-mode operation of the industrial processes, to improve the performance of the proposed DMGPR model, this paper proposes a composite multiple-model DMGPR approach based on the Gaussian Mixture Model algorithm (GMM-DMGPR). The proposed modelling approach utilizes the weights of all the samples belonging to each sub-DMGPR model which are evaluated by utilizing the GMM algorithm when estimating model parameters through expectation and maximization (EM) algorithm. The effectiveness of the proposed GMM-DMGPR approach is demonstrated by two numerical examples and a three-level drawing process of Carbon fiber production.     

A novel framework for motion segmentation and tracking by clustering incomplete trajectories

In this paper, we present a framework for visual object tracking based on clustering trajectories of image key points extracted from an image sequence. The main contribution of our method is that the trajectories are automatically extracted from the image sequence and they are provided directly to a model-based clustering approach. In most other methodologies, the latter constitutes a difficult part since the resulting feature trajectories have a short duration, as the key points disappear and reappear due to occlusion, illumination, viewpoint changes and noise. We present here a sparse, translation invariant regression mixture model for clustering trajectories of variable length. The overall scheme is converted into a maximum a posteriori approach, where the Expectation–Maximization (EM) algorithm is used for estimating the model parameters. The proposed method detects the different objects in the input image sequence by assigning each trajectory to a cluster, and simultaneously provides their motion. Numerical results demonstrate the ability of the proposed method to offer more accurate and robust solutions in comparison with other tracking approaches, such as the mean shift tracker, the camshift tracker and the Kalman filter.     

Score tests for zero-inflated generalized Poisson mixed regression models

Zero-inflated Poisson (ZIP) regression model is a popular approach to the analysis of count data with excess zeros. For correlated count data where the observations are either repeated or clustered outcomes from individual subjects, ZIP mixed regression model may be appropriate. However, ZIP model may often fail to fit such data either because of over-dispersion or because of under-dispersion in relation to the Poisson distribution. In this paper, we extend the ZIP mixed regression model to zero-inflated generalized Poisson (ZIGP) mixed regression model, where the base-line discrete distribution is generalized Poisson (GP) distribution, which is a natural extension of standard Poisson distribution. Furthermore, the random effects are considered in both zero-inflated and GP components throughout the paper. An EM algorithm for estimating parameters is proposed based on the best linear unbiased prediction-type (BLUP) log-likelihood and the residual maximum likelihood (REML). Meanwhile, several score tests are presented for testing the ZIP mixed regression model against the ZIGP mixed regression model, and for testing the significance of regression coefficients in zero-inflation and generalized Poisson portion. A numerical example is given to illustrate our methodology and the properties of score test statistics are investigated through Monte Carlo simulations.     

一种快速、贪心的高斯混合模型EM算法研究

针对传统EM算法存在初始模型成分数目需要预先指定以及收敛速度随样本数目的增长而急剧减慢等问题,提出了一种快速、贪心的高斯混合模型EM算法。该算法采用贪心的策略以及对隐含参数设置适当阈值的方法,使算法能够快速收敛,从而在很少的迭代次数内获取高斯混合模型的模型成分数。该算法通过与传统EM算法、无监督EM算法和鲁棒EM算法的聚类结果进行比较,实验结果证明该算法具有很强的鲁棒性,并且能够提高算法的效率以及模型成分数的准确性。     

Regression analysis of belief functions on interval-valued variables: comparative studies

In real engineering, the observations of process variables are usually imprecise, uncertain, or both. In such cases, the general process modeling approaches cannot be implemented. In this paper, we investigate on the parametric and nonparametric evidential regression of imprecise and uncertain data, represented as belief function on interval-valued variables. The parametric evidential regression includes both multiple linear and nonlinear evidential regression models. The nonlinear evidential regression model is derived by introducing kernel function into the multiple linear evidential regression model. The parametric evidential regression models are identified by using evidential EM algorithm, an evidential extension of the EM algorithm. In the nonparametric evidential regression, the prediction for a given input vector is computed using a nonparametric, instance-based approach: the training samples in the neighborhood of the given input vector provide pieces of evidence reflecting the values taken by such input vector, these pieces of evidence are combined to form the prediction. Some unreliable sensor experiments are designed to validate the performances of the proposed parametric and nonparametric evidential regression models. With comparative studies, we get some interesting results.     

Ordered subsets Bayesian tomographic reconstruction using 2-D smoothing splines as priors

The ordered subsets expectation maximization (OS-EM) algorithm has enjoyed considerable interest for accelerating the well-known EM algorithm for emission tomography. The OS principle has also been applied to several regularized EM algorithms, such as nonquadratic convex minimization-based maximum a posteriori (MAP) algorithms. However, most of these methods have not been as practical as OS-EM due to their complex optimization methods and difficulties in hyperparameter estimation. We note here that, by relaxing the requirement of imposing sharp edges and using instead useful quadratic spline priors, solutions are much easier to compute, and hyperparameter calculation becomes less of a problem. In this work, we use two-dimensional smoothing splines as priors and apply a method of iterated conditional modes for the optimization. In this case, step sizes or line-search algorithms necessary for gradient-based descent methods are avoided. We also accelerate the resulting algorithm using the OS approach and propose a principled way of scaling smoothing parameters to retain the strength of smoothing for different subset numbers. Our experimental results show that the OS approach applied to our quadratic MAP algorithms provides a considerable acceleration while retaining the advantages of quadratic spline priors.     

A multi-threshold segmentation approach based on Artificial Bee Colony optimization

This paper explores the use of the Artificial Bee Colony (ABC) algorithm to compute threshold selection for image segmentation. ABC is an evolutionary algorithm inspired by the intelligent behavior of honey-bees which has been successfully employed to solve complex optimization problems. In this approach, an image 1-D histogram is approximated through a Gaussian mixture model whose parameters are calculated by the ABC algorithm. In the model, each Gaussian function represents a pixel class and therefore a threshold point. Unlike the Expectation-Maximization (EM) algorithm, the ABC method shows fast convergence and low sensitivity to initial conditions. Remarkably, it also improves complex time-consuming computations commonly required by gradient-based methods. Experimental results over multiple images with different range of complexity validate the efficiency of the proposed technique with regard to segmentation accuracy, speed, and robustness. The paper also includes an experimental comparison to the EM and to one gradient-based method which ultimately demonstrates a better performance from the proposed algorithm.     

Image thresholding based on the EM algorithm and the generalized Gaussian distribution

In this paper, a novel parametric and global image histogram thresholding method is presented. It is based on the estimation of the statistical parameters of “object” and “background” classes by the expectation-maximization (EM) algorithm, under the assumption that these two classes follow a generalized Gaussian (GG) distribution. The adoption of such a statistical model as an alternative to the more common Gaussian model is motivated by its attractive capability to approximate a broad variety of statistical behaviors with a small number of parameters. Since the quality of the solution provided by the iterative EM algorithm is strongly affected by initial conditions (which, if inappropriately set, may lead to unreliable estimation), a robust initialization strategy based on genetic algorithms (GAs) is proposed. Experimental results obtained on simulated and real images confirm the effectiveness of the proposed method.     

A new a priori SNR estimator based on multiple linear regression technique for speech enhancement

We propose a new approach to estimate the a priori signal-to-noise ratio (SNR) based on a multiple linear regression (MLR) technique. In contrast to estimation of the a priori SNR employing the decision-directed (DD) method, which uses the estimated speech spectrum in previous frame, we propose to find the a priori SNR based on the MLR technique by incorporating regression parameters such as the ratio between the local energy of the noisy speech and its derived minimum along with the a posteriori SNR. In the experimental step, regression coefficients obtained using the MLR are assigned according to various noise types, for which we employ a real-time noise classification scheme based on a Gaussian mixture model (GMM). Evaluations using both objective speech quality measures and subjective listening tests under various ambient noise environments show that the performance of the proposed algorithm is better than that of the conventional methods.     

Stream water temperature prediction based on Gaussian process regression

The prediction of stream water temperature presents an interesting topic since the water temperature has a significant ecological and economical role, such as in species distribution, fishery, industry and agriculture water exploitation. The prediction of stream water temperature is usually based on appropriate mathematical model and measurements of different atmospheric factors. In this paper, a probabilistic approach to daily mean water temperature prediction is proposed. The resulting model is a combination of two Gaussian process regression models where the first model describes the long-term component of water temperature and the other model describes the short-term variations in water temperature. The proposed approach is developed even further by modeling the short-term variations with multiple Gaussian process regression models instead with a single one. Apart from that, variable selection procedure based on mutual information is presented which is suitable for input variable selection when nonlinear models for stream water prediction are developed. The proposed approach is compared with traditional modeling approaches on the measurements obtained on the Drava river in Croatia. The presented methodology can be used as a basis of the predictive tools for water resource managers.     

An electromagnetism metaheuristic for solving the Maximum Betweenness Problem

In this paper we present an electromagnetism (EM) metaheuristic for solving NP hard Maximum Betweenness Problem (MBP). A new encoding scheme with appropriate objective functions is implemented. Specific representation of the individuals enables the EM operators to explore the searching space in a way that achieves high quality solutions. An effective 1-swap based local search procedure improved by the specific caching technique is performed on each EM point. The algorithm is tested both on real and artificial instances from the literature. Experimental results show that the proposed EM approach achieves all previously known optimal solutions, except one, and achieves the best-known solutions or outperforms other approaches on all large-scale instances, except two. Provided statistical analysis indicates that the EM approach is significantly better than other approaches.     

Gaussian process internal model control

Nonlinear modelling approaches such as neural networks, fuzzy models and multiple model networks have been proposed for model based control, to improve the poor transient response of adaptive control techniques. The quality of control is known to be strongly related to the accuracy of the model which represents the process. A Bayesian Gaussian process (GP) approach provides an analytic prediction of the model uncertainty, which makes the GP model an ideal candidate for model based control strategies. This article extends the use of the GP model for nonlinear internal model control. The invertibility of the GP model is discussed and the use of predicted variance is illustrated on a simulated example.     

Linear search applied to global motion estimation

Gradient-based algorithms for global motion estimation are effective in many image-processing tasks. However, when analytical estimation of derivatives of objective function is not possible, linear search based algorithms such as Powell perform better than the gradient-based ones. In this paper we propose global motion estimation algorithm that exploits linear search based algorithm, particularly Powell, instead of commonly used gradient-based one. We also introduce a new approach for extracting global motion parameters called Two Step Powell-based GME. Using this approach we further improve the Powell-based GME. The proposed Powell-based GME outperforms Gauss–Newton algorithm (gradient-based) in terms of PSNR. The proposed Two Step Powell GME algorithm outperforms Powell-based GME in terms of PSNR and computational time.     

Fuzzy neural network with general parameter adaptation for modelingof nonlinear time-series

By taking advantage of fuzzy systems and neural networks, a fuzzy-neural network with a general parameter (GP) learning algorithm and heuristic model structure determination is proposed in this paper. Our network model is based on the Gaussian radial basis function network (RBFN). We use the flexible GP approach both for initializing the off-line training algorithm and fine-tuning the nonlinear model efficiently in online operation. A modification of the robust unbiasedness criterion using distorter (UCD) is utilized for selecting the structural parameters of this adaptive model. The UCD approach provides the desired modeling accuracy and avoids the risk of over-fitting. In order to illustrate the operation of the proposed modeling scheme, it is experimentally applied to a fault detection application.     

Robust multiple-model LPV approach to nonlinear process identification using mixture t distributions

In this paper, we propose a robust multiple-model linear parameter varying (LPV) approach to identification of the nonlinear process contaminated with outliers. The identification problem is formulated and solved under the EM framework. Instead of assuming that the measurement noise comes from the Gaussian distribution like conventional LPV approaches, the proposed robust algorithm formulates the LPV solution using mixture t distributions and thus naturally addresses the robust identification problem. By modulating the distribution tails through degrees of freedom, the proposed algorithm can handle various outliers. Two simulated examples and an experiment are studied to verify the effectiveness of the proposed approach.     

Improving human robot collaboration through Force/Torque based learning for object manipulation

Human–Robot Collaboration (HRC) is a term used to describe tasks in which robots and humans work together to achieve a goal. Unlike traditional industrial robots, collaborative robots need to be adaptive; able to alter their approach to better suit the situation and the needs of the human partner. As traditional programming techniques can struggle with the complexity required, an emerging approach is to learn a skill by observing human demonstration and imitating the motions; commonly known as Learning from Demonstration (LfD). In this work, we present a LfD methodology that combines an ensemble machine learning algorithm (i.e. Random Forest (RF)) with stochastic regression, using haptic information captured from human demonstration. The capabilities of the proposed method are evaluated using two collaborative tasks; co-manipulation of an object (where the human provides the guidance but the robot handles the objects weight) and collaborative assembly of simple interlocking parts. The proposed method is shown to be capable of imitation learning; interpreting human actions and producing equivalent robot motion across a diverse range of initial and final conditions. After verifying that ensemble machine learning can be utilised for real robotics problems, we propose a further extension utilising Weighted Random Forest (WRF) that attaches weights to each tree based on its performance. It is then shown that the WRF approach outperforms RF in HRC tasks.     

Semiparametric Regression Using Student t Processes

In this paper, we propose a latent factor regression model, in which priors are assigned to both the latent regression vector and the error term, by using reproducing kernels. The resulting regression function follows a stochastic process known as a student process. The model is attractive because its implementation is based on a tractable posterior predictive distribution and a simple expectation-maximization (EM) estimation algorithm. In addition, treating the transductive inference as a missing data problem, we devise the EM algorithm to deal with the parameter estimation as well as the response prediction in a single paradigm. The model is also elaborated for multivariate-response regression problems. For this purpose, we present a generalization of multivariate models and some of its properties. Experimental results show our approaches to be effective.     

自适应平方根无迹卡尔曼滤波算法

将高斯过程回归融入平方根无迹卡尔曼滤波(SRUKF)算法,本文提出了一种不确定系统模型协方差自适应调节滤波算法.该算法分为学习和估计两部分:学习阶段用高斯过程对训练数据进行学习,得到系统回归模型及噪声协方差;估计阶段由回归模型代替状态方程和观测方程,相应的噪声协方差实时自适应调整.该方法克服了传统方法容易受系统动态模型不确定性和噪声协方差不准确限制的问题,仿真结果验证了算法的有效性.     

混合高斯模型的自适应前景提取

复杂场景下的运动前景提取是计算机视觉研究领域的研究重点。为解决复杂场景中的前景目标提取问题,本文提出一种应用于复杂变化场景中的基于混合高斯模型的自适应前景提取方法。本方法可以对视频帧中每个像素的高斯分布数进行动态控制,并且通过在线EM算法对高斯分布的各参数进行学习,此外每个像素的权值更新速率可根据策略进行调整。实验结果表明本方法对复杂变化场景具有较好的适应性,可有效、快速地提取前景目标,提取结果具有较好的查准率和查全率。