Multimode process monitoring with PCA mixture model

For multimode processes, Gaussian mixture model (GMM) has been applied to estimate the probability density function of the process data under normal-operational condition in last few years. However, learning GMM with the expectation maximization (EM) algorithm from process data can be difficult or even infeasible for high-dimensional and collinear process variables. To address this issue, a novel multimode process monitoring approach based on PCA mixture model is proposed. First, the PCA technique is directly applied to the covariance matrix of each Gaussian component to reduce the dimension of process variables and to obtain nonsingular covariance matrices. Then the Bayesian Ying-Yang incremental EM algorithm is adopted to automatically optimize the number of mixture components. With the obtained PCA mixture model, a novel process monitoring scheme is derived for fault detection of multimode processes. Three case studies are provided to evaluate the monitoring performance of the proposed method.     

A particle filter driven dynamic Gaussian mixture model approach for complex process monitoring and fault diagnosis

Complex non-Gaussian processes may have dynamic operation scenario shifts so that the conventional monitoring methods become ill-suited. In this article, a new particle filter based dynamic Gaussian mixture model (DGMM) is developed by adopting particle filter re-sampling method to update the mixture model parameters in a dynamic fashion. Then the particle filtered Bayesian inference probability index is established for process fault detection. Furthermore, the particle filtered Bayesian inference contributions are decomposed among different process variables for fault diagnosis. The proposed DGMM monitoring approach is applied to the Tennessee Eastman Chemical process with dynamic mode changes and the results show its superiority to the dynamic principal component analysis (DPCA) and regular Gaussian mixture model (GMM) in terms of fault detection and diagnosis accuracy.     

Progressive multi-block modelling for enhanced fault isolation in batch processes

A multi-block progressive modelling approach is proposed for enhanced fault isolation in batch processes. The unfolding of batch data typically leads to matrices with a large number of columns and this complicates contribution analysis. In order to rapidly focus fault isolation in batch processes, it would be desirable to employ multi-block modelling techniques. Multi-block model such as consensus principal component analysis (CPCA) can produce multiple monitoring charts for sub-blocks and block loadings and block scores can be obtained which can represent unique behaviour of each sub-block. CPCA model uses super score which is the same as score from normal principal component analysis (PCA) model and it does not produce enhanced monitoring performance. Multi-block PCA (MBPCA) model using block score for model calculation can represent sub-blocks’ character but block scores are obtained from super loading so it may not be the best way to describe sub-blocks. A new MBPCA model is proposed for better expression of each sub-block. Through progressive modelling and contribution analysis, variables related to or affected by the fault, as well as the associated time information, are gradually identified. This enables a fault propagation path being established. The proposed method is applied to a benchmark simulated penicillin production process, PenSim.     

Fixed-structure Gaussian process model

This article describes a method for modelling non-linear dynamic systems from measurement data. The method merges the linear local model blending approach in the velocity-based linearisation form with Bayesian Gaussian process (GP) modelling. The new Fixed-Structure GP (FSGP) model has a predetermined linear model structure with varying and probabilistic parameters represented by GP models. These models have several advantages for the modelling of local model parameters as they give us adequate results, even with small data sets. Furthermore, they provide a measure of the confidence in the prediction of the varying parameters and information about the dependence of the parameters on individual inputs. The FSGP model can be applied for the extended local linear equivalence class of non-linear systems. The obtained non-linear system model can be, for example, used for control-system design. The proposed modelling method is illustrated with a simple example of non-linear system modelling for control design.     

Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting

In this paper, we present an incremental method for model selection and learning of Gaussian mixtures based on the recently proposed variational Bayes approach. The method adds components to the mixture using a Bayesian splitting test procedure: a component is split into two components and then variational update equations are applied only to the parameters of the two components. As a result, either both components are retained in the model or one of them is found to be redundant and is eliminated from the model. In our approach, the model selection problem is treated locally, in a region of the data space, so we can set more informative priors based on the local data distribution. A modified Bayesian mixture model is presented to implement this approach, along with a learning algorithm that iteratively applies a splitting test on each mixture component. Experimental results and comparisons with two other techniques testify for the adequacy of the proposed approach     

Flexible modelling of random effects in linear mixed models—A Bayesian approach

Flexible modelling of random effects in linear mixed models has attracted some attention recently. In this paper, we propose the use of finite Gaussian mixtures as in Verbeke and Lesaffre [A linear mixed model with heterogeneity in the random-effects population, J. Amu. Statist. Assoc. 91, 217-221]. We adopt a fully Bayesian hierarchical framework that allows simultaneous estimation of the number of mixture components together with other model parameters. The technique employed is the Reversible Jump MCMC algorithm (Richardson and Green [On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion). J. Roy. Statist. Soc. Ser. B 59, 731-792]). This approach has the advantage of producing a direct comparison of different mixture models through posterior probabilities from a single run of the MCMC algorithm. Moreover, the Bayesian setting allows us to integrate over different mixture models to obtain a more robust density estimate of the random effects. We focus on linear mixed models with a random intercept and a random slope. Numerical results on simulated data sets and a real data set are provided to demonstrate the usefulness of the proposed method.     

Gaussian process approach for modelling of nonlinear systems

Parametric modelling principals such as neural networks, fuzzy models and multiple model techniques have been proposed for modelling of nonlinear systems. Research effort has focused on issues such as the selection of the structure, constructive learning techniques, computational issues, the curse of dimensionality, off-equilibrium behaviour, etc. To reduce these problems, the use of non-parametrical modelling approaches have been proposed. This paper introduces the Gaussian process (GP) prior approach for the modelling of nonlinear dynamic systems. The relationship between the GP model and the radial basis function neural network is explained. Issues such as selection of the dimension of the input space and the computation load are also discussed. The GP modelling technique is demonstrated on an example of the nonlinear hydraulic positioning system.     

基于GMM和贝叶斯推理的多模态过程运行状态评价

为使综合经济效益最大化,生产过程应保持在最优运行状态等级.针对多模态过程运行状态等级优劣判断问题,提出一种运行状态等级评价方法.该方法对同一运行状态等级的多模态数据建立一个高斯混合模型(Gaussian mixture model,GMM),确保特征提取的准确性,避免模态划分问题.至于在线评价策略,本文采用贝叶斯推理,确定当前运行状态属于各等级的后验概率.并引入滑动窗口,判定当前运行状态等级,有效解决多模态过程运行状态在线评价问题.针对"非优"运行状态,本文提出一种基于变量偏导数的贡献计算方法,对导致过程运行状态等级"非优"的原因变量进行追溯.最后,通过田纳西–伊斯曼(Tennessee–Eastman,TE)过程验证所提方法的有效性.     

Variational learning for Dirichlet process mixtures of Dirichlet distributions and applications

In this paper, we propose a Bayesian nonparametric approach for modeling and selection based on a mixture of Dirichlet processes with Dirichlet distributions, which can also be seen as an infinite Dirichlet mixture model. The proposed model uses a stick-breaking representation and is learned by a variational inference method. Due to the nature of Bayesian nonparametric approach, the problems of overfitting and underfitting are prevented. Moreover, the obstacle of estimating the correct number of clusters is sidestepped by assuming an infinite number of clusters. Compared to other approximation techniques, such as Markov chain Monte Carlo (MCMC), which require high computational cost and whose convergence is difficult to diagnose, the whole inference process in the proposed variational learning framework is analytically tractable with closed-form solutions. Additionally, the proposed infinite Dirichlet mixture model with variational learning requires only a modest amount of computational power which makes it suitable to large applications. The effectiveness of our model is experimentally investigated through both synthetic data sets and challenging real-life multimedia applications namely image spam filtering and human action videos categorization.     

基于GMM的间歇过程故障检测

对间歇过程的多操作阶段进行划分时,往往会被离群点和噪声干扰,影响建模的精确性,针对此问题提出一种新的方法:主元分析--多方向高斯混合模型(Principal component analysis-multiple Gaussian mixture model, PCA-MGMM)建模方法.首先用最短长度法对数据进行等长处理,融合不同展开方法相结合的处理方式消除数据预估问题;利用主元分析方法将数据转换到对故障较为敏感的低维子空间中,得到主元的同时消除了离群点和噪声的干扰;通过改进的高斯混合模型(Gaussian mixture model, GMM)算法对各阶段主元进行聚类,减少了运算量的同时自动得到最佳高斯成分和对应的统计分布参数;最后将局部指标融合为全局概率监控指标,实现了连续的在线监控.通过一个实际的半导体制造过程的仿真研究验证了所提方法的有效性.     

A genetic expectation-maximization method for unsupervised change detection in multitemporal SAR imagery

In this paper, we present a novel, automatic and unsupervised change-detection approach to the analysis of single-channel single-polarization multitemporal SAR images. The statistical parameters of the changed and unchanged classes, which are assumed to follow a generalized Gaussian (GG) distribution in the analysed log-ratio image, are explicitly estimated by the expectation-maximization (EM) algorithm initialized with a robust strategy based on genetic algorithms (GAs). In addition, the proposed approach integrates two further processing capabilities. The first one intends to cope with the problem of the automatic detection of multiple changes in the scene. This is carried out by modelling the log-ratio image histogram with a multimodal GG mixture whose number of components is estimated basing on the Bayesian information criterion (BIC). The second processing capability allows exploitation of spatial contextual information in the change detection process through a Markovian formulation. Results obtained on both simulated and real data are reported and discussed.     

基于高斯混合模型的遥感影像连续型朴素贝叶斯网络分类器

提出了一种新的嵌入高斯混合模型(GMM,Gaussian Mixture Model)遥感影像朴素贝叶斯网络模型GMM-NBC(GMMbased Na ve Bayesian Classifier)。针对连续型朴素贝叶斯网络分类器中假设地物服从单一高斯分布的缺点,该方法将地物在特征空间的分布用高斯混合模型来模拟,用改进EM算法自动获取高斯混合模型的参数;高斯混合模型整体作为一个子节点嵌入朴素贝叶斯网络中,将其输出作为节点(特征)的中间类后验概率,在朴素贝叶斯网络的框架下进行融合获得最终的类后验概率。对多光谱和高光谱数据的分类实验结果表明,该方法较传统贝叶斯分类器分类效果要好,且有较强的鲁棒性。     

基于数据关联狄利克雷混合模型的电网净负荷不确定性表征研究EI北大核心CSCD

针对电网净负荷时序数据关联的特点,提出基于数据关联的狄利克雷混合模型(Data-relevance Dirichlet process mixture model,DDPMM)来表征净负荷的不确定性.首先,使用狄利克雷混合模型对净负荷的观测数据与预测数据进行拟合,得到其混合概率模型;然后,提出考虑数据关联的变分贝叶斯推断方法,改进后验分布对该混合概率模型进行求解,从而得到混合模型的最优参数;最后,根据净负荷预测值的大小得到其对应的预测误差边缘概率分布,实现不确定性表征.本文基于比利时电网的净负荷数据进行检验,算例结果表明:与传统的狄利克雷混合模型和高斯混合模型(Gaussian mixture model,GMM)等方法相比,所提出的基于数据关联狄利克雷混合模型可以更为有效地表征净负荷的不确定性.     

Bayesian learning of finite generalized inverted Dirichlet mixtures: Application to object classification and forgery detection

The advent of mixture models has opened the possibility of flexible models which are practical to work with. A common assumption is that practitioners typically expect that data are generated from a Gaussian mixture. The inverted Dirichlet mixture has been shown to be a better alternative to the Gaussian mixture and to be of significant value in a variety of applications involving positive data. The inverted Dirichlet is, however, usually undesirable, since it forces an assumption of positive correlation. Our focus here is to develop a Bayesian alternative to both the Gaussian and the inverted Dirichlet mixtures when dealing with positive data. The alternative that we propose is based on the generalized inverted Dirichlet distribution which offers high flexibility and ease of use, as we show in this paper. Moreover, it has a more general covariance structure than the inverted Dirichlet. The proposed mixture model is subjected to a fully Bayesian analysis based on Markov Chain Monte Carlo (MCMC) simulation methods namely Gibbs sampling and Metropolis–Hastings used to compute the posterior distribution of the parameters, and on Bayesian information criterion (BIC) used for model selection. The adoption of this purely Bayesian learning choice is motivated by the fact that Bayesian inference allows to deal with uncertainty in a unified and consistent manner. We evaluate our approach on the basis of two challenging applications concerning object classification and forgery detection.     

基于混合变分自编码器回归模型的软测量建模方法EI北大核心CSCD

近年来,变分自编码器(Variational auto-encoder,VAE)模型由于在概率数据描述和特征提取能力等方面的优越性,受到了学术界和工业界的广泛关注,并被引入到工业过程监测、诊断和软测量建模等应用中.然而,传统基于VAE的软测量方法使用高斯分布作为潜在变量的分布,限制了其对复杂工业过程数据,尤其是多模态数据的建模能力.为了解决这一问题,本论文提出了一种混合变分自编码器回归模型(Mixture variational autoencoder regression,MVAER),并将其应用于复杂多模态工业过程的软测量建模.具体来说,该方法采用高斯混合模型来描述VAE的潜在变量分布,通过非线性映射将复杂多模态数据映射到潜在空间,学习各模态下的潜在变量,获取原始数据的有效特征表示.同时,建立潜在特征表示与关键质量变量之间的回归模型,实现软测量应用.通过一个数值例子和一个实际工业案例,对所提模型的性能进行了评估,验证了该模型的有效性和优越性.     

Probabilistic characterisation of model error using Gaussian mixture model—With application to Charpy impact energy prediction for alloy steel

A novel approach to characterise the model prediction errors using a Gaussian mixture model is proposed. The motivation for this work lies behind many data models that are developed through prediction error minimisation with the assumption of a normal noise distribution. When the noise is non-normal, which may often be the case in complicated data modelling scenarios, the model prediction errors may contain rich information, which can be further exploited for model refinement and improvement. The key contents presented in this paper include: choosing the relevant variables to form the error data, optimising the number of Gaussian components required for the error data modelling, and fitting the Gaussian mixture parameters using an expectation-maximisation algorithm. Application of the proposed method for further model improvement, within the framework of hybrid deterministic/stochastic modelling, is also discussed. Preliminary results on the real industrial Charpy impact energy data for heat-treated steels show its effectiveness for model error characterisation, and the potential for model performance improvement in terms of prediction accuracy as well as providing accurate prediction confidence intervals.     

基于狄利克雷过程混合模型的内外先验融合

近年来,使用高斯混合模型作为块先验的贝叶斯方法取得了优秀的图像复原性能,针对这类模型分量固定及主要依赖外部学习的缺点,提出了一种新的基于狄利克雷过程混合模型的图像先验模型。该模型从干净图像数据库中学习外部通用先验,从退化图像中学习内部先验,借助模型中统计量的可累加性自然实现内外部先验融合。通过聚类的新增及归并机制,模型的复杂度随着数据的增大或缩小而自适应地变化,可以学习到可解释及紧凑的模型。为了求解所有隐变量的变分后验分布,提出了一种结合新增及归并机制的批次更新可扩展变分算法,解决了传统坐标上升算法在大数据集下效率较低、容易陷入局部最优解的问题。在图像去噪及填充实验中,相比传统方法,所提模型无论在客观质量评价还是视觉观感上都更有优势,验证了该模型的有效性。     

Multi-model multivariate Gaussian process modelling with correlated noises

A composite multiple-model approach based on multivariate Gaussian process regression (MGPR) with correlated noises is proposed in this paper. In complex industrial processes, observation noises of multiple response variables can be correlated with each other and process is nonlinear. In order to model the multivariate nonlinear processes with correlated noises, a dependent multivariate Gaussian process regression (DMGPR) model is developed in this paper. The covariance functions of this DMGPR model are formulated by considering the “between-data” correlation, the “between-output” correlation, and the correlation between noise variables. Further, owing to the complexity of nonlinear systems as well as possible multiple-mode operation of the industrial processes, to improve the performance of the proposed DMGPR model, this paper proposes a composite multiple-model DMGPR approach based on the Gaussian Mixture Model algorithm (GMM-DMGPR). The proposed modelling approach utilizes the weights of all the samples belonging to each sub-DMGPR model which are evaluated by utilizing the GMM algorithm when estimating model parameters through expectation and maximization (EM) algorithm. The effectiveness of the proposed GMM-DMGPR approach is demonstrated by two numerical examples and a three-level drawing process of Carbon fiber production.     

Multivariate statistical monitoring of two-dimensional dynamic batch processes utilizing non-Gaussian information

Dynamics are inherent characteristics of batch processes, and they may exist not only within a particular batch, but also from batch to batch. To model and monitor such two-dimensional (2D) batch dynamics, two-dimensional dynamic principal component analysis (2D-DPCA) has been developed. However, the original 2D-DPCA calculates the monitoring control limits based on the multivariate Gaussian distribution assumption which may be invalid because of the existence of 2D dynamics. Moreover, the multiphase features of many batch processes may lead to more significant non-Gaussianity. In this paper, Gaussian mixture model (GMM) is integrated with 2D-DPCA to address the non-Gaussian issue in 2D dynamic batch process monitoring. Joint probability density functions (pdf) are estimated to summarize the information contained in 2D-DPCA subspaces. Consequently, for online monitoring, control limits can be calculated based on the joint pdf. A two-phase fed-batch fermentation process for penicillin production is used to verify the effectiveness of the proposed method.     

A screening-based gradient-enhanced Gaussian process regression model for multi-fidelity data fusion

The prediction accuracy of multi-fidelity models can be enhanced by incorporating gradient formation. However, the computational complexity would increase dramatically as the number of design variables increase. In this work, a gradient-enhanced multi-fidelity Gaussian process model using a portion of gradients (PGEMFGP) is proposed. To be specific, a Bayesian Gaussian process regression model for multi-fidelity (MF) data fusion is developed, which incorporates high-fidelity (HF) and low-fidelity (LF) responses, as well as the corresponding gradients. A screening technique based on distance correlation is applied to select a portion of gradients of the low-fidelity model so that the modeling complexity can be greatly reduced. The merit of the proposed method is tested with six numerical examples ranging from 10-D to 30-D, as well as an aerodynamic airfoil case with 18 design variables. The proposed method is compared to two other existing gradient-enhanced Gaussian process-based models. It is shown that the modeling efficiency of the proposed model is dramatically improved compared to the original gradient-enhanced multi-fidelity Gaussian process model, while the loss of the prediction accuracy can be almost negligible. In consequence, it can be a promising approach for gradient-enhanced models dealing with multi-fidelity data.