基于GMM的多工况过程监测方法

传统基于主元分析的故障检测方法大多假设工业过程只运行在1个稳定工况,数据服从单一的高斯分布。若这些方法直接用于多工况过程则将会产生大量的误检。为此,本文提出了1种基于高斯混合模型的多工况过程监测方法。首先利用PCA变换对过程数据集进行降维,在主元空间建立高斯混合模型对过程数据进行聚类,自动获取工况数和相关分布特性。然后对每个工况建立主元分析(principal component analysis,PCA)模型来描述整个运行过程数据分布的统计特性。最后在过程监测中,根据监测样本属于各个工况的概率构造综合统计量,实现对多工况过程的故障检测。TE过程的仿真结果表明,本文提出的方法与传统的PCA方法相比,能自动获取工况和精确估计各个工况的统计特性,从而能更准确及时地检测出多工况过程的各种故障。     

Process monitoring through manifold regularization-based GMM with global/local information

The nonlinear and multimodal characteristics in many manufacturing processes have posed some difficulties to regular multivariate statistical process control (MSPC) (e.g., principal component analysis (PCA)-based monitoring method) because a fundamental assumption is that the process data follow unimodal and Gaussian distribution. To explicitly address these important data distribution characteristics in some complicated processes, a novel manifold learning algorithm, joint local intrinsic and global/local variance preserving projection (JLGLPP) is proposed for information extraction from process data. Based on the features extracted by JLGLPP, local/nonlocal manifold regularization-based Gaussian mixture model (LNGMM) is proposed to estimate process data distributions with nonlinear and multimodal characteristics. A probabilistic indicator for quantifying process states is further developed, which effectively combines local and global information extracted from a baseline GMM. Thus, the JLGLPP and LNGMM-based monitoring model can be used effectively for online process monitoring under complicated working conditions. The experimental results illustrate that the proposed method effectively captures meaningful information hidden in the process signals and shows superior process monitoring performance compared to regular monitoring methods.     

基于混合变分自编码器回归模型的软测量建模方法EI北大核心CSCD

近年来,变分自编码器(Variational auto-encoder,VAE)模型由于在概率数据描述和特征提取能力等方面的优越性,受到了学术界和工业界的广泛关注,并被引入到工业过程监测、诊断和软测量建模等应用中.然而,传统基于VAE的软测量方法使用高斯分布作为潜在变量的分布,限制了其对复杂工业过程数据,尤其是多模态数据的建模能力.为了解决这一问题,本论文提出了一种混合变分自编码器回归模型(Mixture variational autoencoder regression,MVAER),并将其应用于复杂多模态工业过程的软测量建模.具体来说,该方法采用高斯混合模型来描述VAE的潜在变量分布,通过非线性映射将复杂多模态数据映射到潜在空间,学习各模态下的潜在变量,获取原始数据的有效特征表示.同时,建立潜在特征表示与关键质量变量之间的回归模型,实现软测量应用.通过一个数值例子和一个实际工业案例,对所提模型的性能进行了评估,验证了该模型的有效性和优越性.     

Multi-model multivariate Gaussian process modelling with correlated noises

A composite multiple-model approach based on multivariate Gaussian process regression (MGPR) with correlated noises is proposed in this paper. In complex industrial processes, observation noises of multiple response variables can be correlated with each other and process is nonlinear. In order to model the multivariate nonlinear processes with correlated noises, a dependent multivariate Gaussian process regression (DMGPR) model is developed in this paper. The covariance functions of this DMGPR model are formulated by considering the “between-data” correlation, the “between-output” correlation, and the correlation between noise variables. Further, owing to the complexity of nonlinear systems as well as possible multiple-mode operation of the industrial processes, to improve the performance of the proposed DMGPR model, this paper proposes a composite multiple-model DMGPR approach based on the Gaussian Mixture Model algorithm (GMM-DMGPR). The proposed modelling approach utilizes the weights of all the samples belonging to each sub-DMGPR model which are evaluated by utilizing the GMM algorithm when estimating model parameters through expectation and maximization (EM) algorithm. The effectiveness of the proposed GMM-DMGPR approach is demonstrated by two numerical examples and a three-level drawing process of Carbon fiber production.     

Maximum likelihood estimation of Gaussian mixture models using stochastic search

Gaussian mixture models (GMM), commonly used in pattern recognition and machine learning, provide a flexible probabilistic model for the data. The conventional expectation–maximization (EM) algorithm for the maximum likelihood estimation of the parameters of GMMs is very sensitive to initialization and easily gets trapped in local maxima. Stochastic search algorithms have been popular alternatives for global optimization but their uses for GMM estimation have been limited to constrained models using identity or diagonal covariance matrices. Our major contributions in this paper are twofold. First, we present a novel parametrization for arbitrary covariance matrices that allow independent updating of individual parameters while retaining validity of the resultant matrices. Second, we propose an effective parameter matching technique to mitigate the issues related with the existence of multiple candidate solutions that are equivalent under permutations of the GMM components. Experiments on synthetic and real data sets show that the proposed framework has a robust performance and achieves significantly higher likelihood values than the EM algorithm.     

基于GMM的间歇过程故障检测

对间歇过程的多操作阶段进行划分时,往往会被离群点和噪声干扰,影响建模的精确性,针对此问题提出一种新的方法:主元分析--多方向高斯混合模型(Principal component analysis-multiple Gaussian mixture model, PCA-MGMM)建模方法.首先用最短长度法对数据进行等长处理,融合不同展开方法相结合的处理方式消除数据预估问题;利用主元分析方法将数据转换到对故障较为敏感的低维子空间中,得到主元的同时消除了离群点和噪声的干扰;通过改进的高斯混合模型(Gaussian mixture model, GMM)算法对各阶段主元进行聚类,减少了运算量的同时自动得到最佳高斯成分和对应的统计分布参数;最后将局部指标融合为全局概率监控指标,实现了连续的在线监控.通过一个实际的半导体制造过程的仿真研究验证了所提方法的有效性.     

Wavelet fuzzy LVQ based speaker verification system

This work evaluates the performance of speaker verification system based on Wavelet based Fuzzy Learning Vector Quantization (WLVQ) algorithm. The parameters of Gaussian mixture model (GMM) are designed using this proposed algorithm. Mel Frequency Cepstral Coefficients (MFCC) are extracted from the speech data and vector quantized through Wavelet based FLVQ algorithm. This algorithm develops a multi resolution codebook by updating both winning and nonwinning prototypes through an unsupervised learning process. This codebook is used as mean vector of GMM. The other two parameters, weight and covariance are determined from the clusters formed by the WLVQ algorithm. The multi resolution property of wavelet transform and ability of FLVQ in regulating the competition between prototypes during learning are combined in this algorithm to develop an efficient codebook for GMM. Because of iterative nature of Expectation Maximization (EM) algorithm, the applicability of alternative training algorithms is worth investigation. In this work, the performance of speaker verification system using GMM trained by LVQ, FLVQ and WLVQ algorithms are evaluated and compared with EM algorithm. FLVQ and WLVQ based training algorithms for modeling speakers using GMM yields better performance than EM based GMM.     

基于GMM和贝叶斯推理的多模态过程运行状态评价

为使综合经济效益最大化,生产过程应保持在最优运行状态等级.针对多模态过程运行状态等级优劣判断问题,提出一种运行状态等级评价方法.该方法对同一运行状态等级的多模态数据建立一个高斯混合模型(Gaussian mixture model,GMM),确保特征提取的准确性,避免模态划分问题.至于在线评价策略,本文采用贝叶斯推理,确定当前运行状态属于各等级的后验概率.并引入滑动窗口,判定当前运行状态等级,有效解决多模态过程运行状态在线评价问题.针对"非优"运行状态,本文提出一种基于变量偏导数的贡献计算方法,对导致过程运行状态等级"非优"的原因变量进行追溯.最后,通过田纳西–伊斯曼(Tennessee–Eastman,TE)过程验证所提方法的有效性.     

基于GMM的多模态过程模态识别与过程监测

多模态复杂过程的多变量、多工序、变量时变性以及模态转换时间不确定等多种原因, 导致面向多模态生产过程的监测问题十分复杂. 对此, 基于高斯混合模型的监测方法, 结合定性知识和定量知识, 解决了多模态过程监测中离线数据模态划分、稳定模态和过渡模态的监测模型建立以及在线数据的模态识别等关键问题, 最终实现了对多模态过程的监测.     

A new fault diagnosis method of multimode processes using Bayesian inference based Gaussian mixture contribution decomposition

This paper presents a novel Bayesian inference based Gaussian mixture contribution (BIGMC) method to isolate and diagnose the faulty variables in chemical processes with multiple operating modes. The statistical confidence intervals of traditional principal component analysis (PCA) based T² and SPE diagnostics rely upon the assumption that the operating data follow a multivariate Gaussian distribution approximately and therefore may not be able to determine the faulty variables in multimode non-Gaussian processes accurately. As an alternative solution, the proposed BIGMC method first identifies the multiple Gaussian modes corresponding to different operating conditions and then integrates the Mahalanobis distance based variable contributions across all the Gaussian clusters through Bayesian inference strategy. The derived BIGMC index is of probabilistic feature and includes all operation scenarios with posterior probabilities as weighting factors. The Tennessee Eastman process (TEP) is used to demonstrate the utility of the proposed BIGMC method for fault diagnosis of multimode processes. The comparison of the single-PCA and multi-PCA based contribution approaches shows that the BIGMC method can effectively identify the leading faulty variables with superior diagnosis capability.     

A particle filter driven dynamic Gaussian mixture model approach for complex process monitoring and fault diagnosis

Complex non-Gaussian processes may have dynamic operation scenario shifts so that the conventional monitoring methods become ill-suited. In this article, a new particle filter based dynamic Gaussian mixture model (DGMM) is developed by adopting particle filter re-sampling method to update the mixture model parameters in a dynamic fashion. Then the particle filtered Bayesian inference probability index is established for process fault detection. Furthermore, the particle filtered Bayesian inference contributions are decomposed among different process variables for fault diagnosis. The proposed DGMM monitoring approach is applied to the Tennessee Eastman Chemical process with dynamic mode changes and the results show its superiority to the dynamic principal component analysis (DPCA) and regular Gaussian mixture model (GMM) in terms of fault detection and diagnosis accuracy.     

Active curve axis Gaussian mixture models

Gaussian Mixture Models (GMM) have been broadly applied for the fitting of probability density function. However, due to the intrinsic linearity of GMM, usually many components are needed to appropriately fit the data distribution, when there are curve manifolds in the data cloud.

In order to solve this problem and represent data with curve manifolds better, in this paper we propose a new nonlinear probability model, called active curve axis Gaussian model. Intuitively, this model can be imagined as Gaussian model being bent at the first principal axis. For estimating parameters of mixtures of this model, the EM algorithm is employed.

Experiments on synthetic data and Chinese characters show that the proposed nonlinear mixture models can approximate distributions of data clouds with curve manifolds in a more concise and compact way than GMM does. The performance of the proposed nonlinear mixture models is promising.     

基于GLRGMM的间歇过程在线监控策略

针对间歇过程的非线性和动态性,提出了全局—局部正则化高斯混合模型 （GLRGMM）算法。首先引入邻域保持嵌入算法提取局部流形结构,通过寻求一种低维投影对非线性过程进行全局结构保持,同时最大限度地保留局部流形特征;然后通过对高斯混合模型引入正则项来在线监控更新高斯模型,获取非线性数据流形结构,解决数据动态性问题;最后集成全局—局部监控指标实现在线监控。通过青霉素发酵过程进行了验证,结果表明所提算法比DPCA、GLNPE具有更好的在线监控效果。     

基于在线分裂合并EM算法的高斯混合模型分类方法*

为了解决传统高斯混合模型中期望值EM处理必须具备足够数量的样本才能开始训练的问题,提出了一种新的高斯混合模型在线增量训练算法。本算法在Ueda等人提出的Split-and-Merge EM方法基础上对分裂合并准则的计算进行了改进,能够有效避免陷入局部极值并减少奇异值出现的情况;通过引入时间序列参数提出了增量EM训练方法,能够实现增量式的期望最大化训练,从而能够逐样本在线更新GMM模型参数。对合成数据和实际语音识别应用的实验结果表明,本算法具有较好的运算效率和分类准确性。     

基于CUDA的GMM模型快速训练方法及应用

由于能够很好地近似描述任何分布,混合高斯模型(GMM)在模式在识别领域得到了广泛的应用.GMM模型参数通常使用迭代的期望最大化(EM)算法训练获得,当训练数据量非常庞大及模型混合数很大时,需要花费很长的训练时间.NVIDIA公司推出的统一计算设备架构(Computed unified device architecture,CUDA)技术通过在图形处理单元(GPU)并发执行多个线程能够实现大规模并行快速计算.本文提出一种基于CUDA,适用于特大数据量的GMM模型快速训练方法,包括用于模型初始化的K-means算法的快速实现方法,以及用于模型参数估计的EM算法的快速实现方法.文中还将这种训练方法应用到语种GMM模型训练中.实验结果表明,与Intel DualCore PentiumⅣ3.0 GHz CPU的一个单核相比,在NVIDIA GTS250 GPU上语种GMM模型训练速度提高了26倍左右.     

一种基于EM算法的图像分割改进技术

图像分割是计算机视觉的基础,该文结合EM算法和PCA降维技术,给出了一种有效快速的进行图象分割的方法。该方法利用高斯混合模型对原始图像进行建模,通过EM算法将分割问题转化为参数最大似然估计的问题,同时采用PCA降维技术和随机采样来降低计算量。通过人工合成图象及真实图象的实际测试结果,验证了该算法的有效性和快速性。     

一种基于EM算法的图像分割改进技术

图像分割是计算机视觉的基础,该文结合EM算法和PCA降维技术,给出了一种有效快速的进行图象分割的方法。该方法利用高斯混合模型对原始图像进行建模,通过EM算法将分割问题转化为参数最大似然估计的问题,同时采用PCA降维技术和随机采样来降低计算量。通过人工合成图象及真实图象的实际测试结果,验证了该算法的有效性和快速性。     

Speech Enhancement Using Gaussian Scale Mixture Models

This paper presents a novel probabilistic approach to speech enhancement. Instead of a deterministic logarithmic relationship, we assume a probabilistic relationship between the frequency coefficients and the log-spectra. The speech model in the log-spectral domain is a Gaussian mixture model (GMM). The frequency coefficients obey a zero-mean Gaussian whose covariance equals to the exponential of the log-spectra. This results in a Gaussian scale mixture model (GSMM) for the speech signal in the frequency domain, since the log-spectra can be regarded as scaling factors. The probabilistic relation between frequency coefficients and log-spectra allows these to be treated as two random variables, both to be estimated from the noisy signals. Expectation-maximization (EM) was used to train the GSMM and Bayesian inference was used to compute the posterior signal distribution. Because exact inference of this full probabilistic model is computationally intractable, we developed two approaches to enhance the efficiency: the Laplace method and a variational approximation. The proposed methods were applied to enhance speech corrupted by Gaussian noise and speech-shaped noise (SSN). For both approximations, signals reconstructed from the estimated frequency coefficients provided higher signal-to-noise ratio (SNR) and those reconstructed from the estimated log-spectra produced lower word recognition error rate because the log-spectra fit the inputs to the recognizer better. Our algorithms effectively reduced the SSN, which algorithms based on spectral analysis were not able to suppress.     

A unified neurofuzzy model for classification

This work proposes a unified neurofuzzy modelling scheme. To begin with, the initial fuzzy base construction method is based on fuzzy clustering utilising a Gaussian mixture model (GMM) combined with the analysis of covariance (ANOVA) decomposition in order to obtain more compact univariate and bivariate membership functions over the subspaces of the input features. The mean and covariance of the Gaussian membership functions are found by the expectation maximisation (EM) algorithm with the merit of revealing the underlying density distribution of system inputs. The resultant set of membership functions forms the basis of the generalised fuzzy model (GFM) inference engine. The model structure and parameters of this neurofuzzy model are identified via the supervised subspace orthogonal least square (OLS) learning. Finally, instead of providing deterministic class label as model output by convention, a logistic regression model is applied to present the classifier’s output, in which the sigmoid type of logistic transfer function scales the outputs of the neurofuzzy model to the class probability. Experimental validation results are presented to demonstrate the effectiveness of the proposed neurofuzzy modelling scheme.     

基于高斯混合模型的遥感影像连续型朴素贝叶斯网络分类器

提出了一种新的嵌入高斯混合模型(GMM,Gaussian Mixture Model)遥感影像朴素贝叶斯网络模型GMM-NBC(GMMbased Na ve Bayesian Classifier)。针对连续型朴素贝叶斯网络分类器中假设地物服从单一高斯分布的缺点,该方法将地物在特征空间的分布用高斯混合模型来模拟,用改进EM算法自动获取高斯混合模型的参数;高斯混合模型整体作为一个子节点嵌入朴素贝叶斯网络中,将其输出作为节点(特征)的中间类后验概率,在朴素贝叶斯网络的框架下进行融合获得最终的类后验概率。对多光谱和高光谱数据的分类实验结果表明,该方法较传统贝叶斯分类器分类效果要好,且有较强的鲁棒性。