Hidden Markov models combining local and global information for nonlinear and multimodal process monitoring

Multivariate statistical process monitoring techniques have been successfully used to detect and identify deviation from normal operation within industrial processes. However, the nonlinear and multimodal characteristics in some processes have posed difficulties to the conventional approaches, because a fundamental assumption is often that the operating data is unimodal and Gaussian distributed. To explicitly address these important characteristics in processes, hidden Markov models (HMM)-based process monitoring models have been developed in this paper. A novel quantification indication for process state is proposed, which effectively combines local information (Mahalanobis distance) and global information (negative log likelihood probability) in HMM. In addition, a Bayesian inference-based process failure probability indication is developed, where the posterior probabilities of each new sample belonging to each Gaussian component in a hidden state of HMM are calculated. HMM is capable to estimate data distribution from normal operation with nonlinear and multimodal characteristics, under the assumption that predictable fault patterns are not available. Thus, the HMM-based monitoring models can be used for online process monitoring without too much human intervention. The validity and effectiveness of the proposed models are illustrated through simulated and real-world processes. The experimental results clearly demonstrate that the proposed approaches effectively captured the nonlinear and multimodal relationship in process variables and showed superior process monitoring performance compared to those conventional process monitoring approaches.     

基于GLRGMM的间歇过程在线监控策略

针对间歇过程的非线性和动态性,提出了全局—局部正则化高斯混合模型 （GLRGMM）算法。首先引入邻域保持嵌入算法提取局部流形结构,通过寻求一种低维投影对非线性过程进行全局结构保持,同时最大限度地保留局部流形特征;然后通过对高斯混合模型引入正则项来在线监控更新高斯模型,获取非线性数据流形结构,解决数据动态性问题;最后集成全局—局部监控指标实现在线监控。通过青霉素发酵过程进行了验证,结果表明所提算法比DPCA、GLNPE具有更好的在线监控效果。     

基于LNS-DEWKECA算法的多模态工业过程故障检测

受市场需求主导,工业过程需要在多种工作模态下切换,数据往往呈现多模态复杂分布特性,研究多模态的故障检测技术对于保障工业过程的安全运行具有重要意义.为此,提出一种基于局部近邻标准化(LNS)和方向熵加权核熵成分分析(DEWKECA)的故障检测算法.利用LNS实现多模态数据的标准化,相比于全局标准化, LNS可以有效消除多模态特性;考虑到故障样本与正常样本在变化趋势上的差异,定义样本变化方向的信息熵为方向熵,用来衡量样本变化方向的无序程度,从而利用DEWKECA实现数据降维,可以更有效提取数据变化方向特征;考虑到多模态数据往往服从非高斯分布,采用局部离群因子(LOF)算法建立监控统计量,根据核密度估计确定其控制限.最后,通过数值例子及TE过程仿真验证所提出算法的有效性.     

基于局部熵双子空间的多模态过程故障检测

为了提高非高斯工业过程的检测性能, 提出局部熵双子空间(LEDS)的多模态过程故障检测方法. 运用局部 概率密度估计构建数据的局部熵矩阵, 消除数据的多模态特性. 用Kolmogorov-Smirnov (KS)检验局部熵数据中变 量的正态分布特性, 对高斯分布和非高斯分布的数据分别建立基于PCA的高斯子空间和ICA的非高斯子空间故障 检测模型. 利用Bayesian决策将检测结果转化成发生故障概率的形式, 将检测结果组合成最终的统计信息, 进行故 障检测. 将该方法应用于数值例子和田纳西–伊斯曼多模态过程, 仿真结果表明, 该方法在误报率较低的情况下, 故 障检测率最高, 优于PCA、局部熵PCA(LEPCA)和局部熵ICA(LEICA)方法.     

Local and global principal component analysis for process monitoring

In this paper, a novel data projection method, local and global principal component analysis (LGPCA) is proposed for process monitoring. LGPCA is a linear dimensionality reduction technique through preserving both of local and global information in the observation data. Beside preservation of the global variance information of Euclidean space that principal component analysis (PCA) does, LGPCA is characterized by capturing a good linear embedding that preserves local structure to find meaningful low-dimensional information hidden in the high-dimensional process data. LGPCA-based T² (D) and squared prediction error (Q) statistic control charts are developed for on-line process monitoring. The validity and effectiveness of LGPCA-based monitoring method are illustrated through simulation processes and Tennessee Eastman process (TEP). The experimental results demonstrate that the proposed method effectively captures meaningful information hidden in the observations and shows superior process monitoring performance compared to those regular monitoring methods.     

基于相对密度和流形上k近邻的聚类算法

针对传统的基于欧氏距离的相似性度量不能完全反映复杂结构的数据分布特性的问题,提出了一种基于相对密度和流形上k近邻的聚类算法。基于能描述全局一致性信息的流形距离,及可体现局部相似性和紧密度的k近邻概念,通过流形上k近邻相似度度量数据对象间的相似性,采用k近邻的相对紧密度发现不同密度下的类簇,设计近邻点对约束规则搜寻k近邻点对构成的近邻链,归类数据对象及识别离群点。与标准k-means算法、流形距离改进的k-means算法进行了性能比较,在人工数据集和UCI数据集上的仿真实验结果均表明,该算法能有效地处理复杂结构的数据聚类问题,且聚类效果更好。     

Nonlinear process monitoring and fault isolation using extended maximum variance unfolding

Kernel principal component analysis (KPCA) has become a popular technique for process monitoring, owing to its capability of handling nonlinearity. Nevertheless, KPCA suffers from two major disadvantages. First, the underlying manifold structure of data is not considered in process modeling. Second, the selection of kernel parameters is problematic. To avoid such deficiencies, a manifold learning technique named maximum variance unfolding (MVU) is considered as an alternative. However, such method is merely able to deal with the training data, but has no means to handle new samples. Therefore, MVU cannot be applied to process monitoring directly. In this paper, an extended MVU (EMVU) method is proposed, extending the utilization of MVU to new samples by approximating the nonlinear mapping between the input space and the output space with a Gaussian process model. Consequently, EMVU is suitable to nonlinear process monitoring. A cross-validation algorithm is designed to determine the dimensionality of the EMVU output space. For online monitoring, three different types of monitoring indices are developed, including squared prediction error (SPE), Hotelling-T², and the prediction variance of the outputs. In addition, a fault isolation algorithm based on missing data analysis is designed for EMVU to identify the variables contributing most to the faults. The effectiveness of the proposed methods is verified by the case studies on a numerical simulation and the benchmark Tennessee Eastman (TE) process.     

黎曼法坐标流形学习扩展算法

LOGMAP是最近提出的一种黎曼流形学习算法,它能够有效地学习出高维数据的低维嵌入坐标.然而该算法只能处理单类数据的流形学习问题,当存在多类数据时往往不能得到理想的嵌入结果.为解决这个问题,提出了一种扩展的LOGMAP算法(Extended LOGMA PAlgorithm,简称ELOGMAP).该算法通过计算全局基准点所在类到其他类的最短距离找出各类的局部基准点,然后逐个计算各类数据相对于局部基准点的局部黎曼法坐标,最后通过扩展的全局基准点与局部基准点之间测地距离关系得到多类数据的整体嵌入坐标.实验结果验证了该算法在处理多类数据流形学习上的有效性.     

黎曼流形上的多模态医学图像配准

针对传统多模态配准方法忽视图像的结构信息和像素间的空间关系,并假定灰度全局一致的前提。本文提出了一种在黎曼流形上的多模态医学图像配准算法。首先采用线性动态模型捕捉图像的高维空间的非线性结构和局部信息,然后通过参数化动态模型构造出一种李群群元,形成黎曼流形,继而将流形嵌入到高维的再生核希尔伯特空间,再在核空间上学习出相似性测度。仿真和临床数据实验结果表明本文算法在刚体配准和仿射配准精度上均优于传统互信息方法和基于邻域的相似性测度学习方法。     

一种改进的局部线性嵌入算法

局部线性嵌入算法(Local Linear Embedding,简称LLE)是一种非线性流形学习算法,能有效地学习出高维采样数据的低维嵌入坐标,但也存在一些不足,如不能处理稀疏的样本数据.针对这些缺点,提出了一种基于局部映射的线性嵌入算法(Local Project Linear Embedding,简称LPLE).通过假定目标空间的整体嵌入函数,重新构造样本点的局部邻域特征向量,最后将问题归结为损失矩阵的特征向量问题从而构造出目标空间的全局坐标.LPLE算法解决了传统LLE算法在源数据稀疏情况下的不能有效进行降维的问题,这也是其他传统的流形学习算法没有解决的.通过实验说明了LPLE算法研究的有效性和意义.     

基于判别正则化的局部保留投影

局部保留投影（Locality preserving projections,LPP）是一种常用的线性化流形学习方法,其通过线性嵌入来保留基于图所描述的流形数据本质结构特征,因此LPP对图的依赖性强,且在嵌入过程中缺少对图描述的进一步分析和挖掘。当图对数据本质结构特征描述不恰当时,LPP在嵌入过程中不易实现流形数据本质结构的有效提取。为了解决这个问题,本文在给定流形数据图描述的条件下,通过引入局部相似度阈值进行局部判别分析,并据此建立判别正则化局部保留投影（简称DRLPP）。该方法能够在现有图描述的条件下,有效突出不同流形结构在线性嵌入空间中的可分性。在人造合成数据集和实际标准数据集上对DRLPP以及相关算法进行对比实验,实验结果证明了DRLPP的有效性。     

一种基于局部模型的非线性多工况过程监测方法

针对复杂工业过程中的非线性、非高斯特性以及多工况问题, 提出了一种基于局部模型的在线统计监测新方法. 首先利用局部最小二乘支持向量机回归 (Least square support vector regression, LSSVR) 模型对过程输出进行预测, 与真实的输出相比较构成残差序列. 然后利用 ICA-PCA 两步特征提取策略, 完整地提取残差的高斯和非高斯信息, 最后用三个统计量 (I2、T2 和 SPE) 对过程进行监测, 建立了一种具有非线性、非高斯特性的多工况过程在线监测算法. 通过对 TE (Tennessee Eastman) 过程的仿真研究, 验证提出的方法是可行、有效的, 并显示出了一定的故障检测能力.     

基于流形的微粒群优化

起源于群体智能的微粒群优化技术已经得到广泛的应用.一般情况下,我们假定微粒处于均匀分布的线性空间内.流形是几何学中的概念,概括地说,它是一个非线性空间.提出了一种基于流形即非线性空间上的微粒群优化框架MPSO,它用于非线性、非均匀数据分布,并对其进行了收敛性分析和算法性能评估.     

Stochastic neighbor projection on manifold for feature extraction

This paper develops a manifold-oriented stochastic neighbor projection (MSNP) technique for feature extraction. MSNP is designed to find a linear projection for the purpose of capturing the underlying pattern structure of observations that actually lie on a nonlinear manifold. In MSNP, the similarity information of observations is encoded with stochastic neighbor distribution based on geodesic distance metric, then the same distribution is required to be hold in feature space. This learning criterion not only empowers MSNP to extract nonlinear feature through a linear projection, but makes MSNP competitive as well by reason that distribution preservation is more workable and flexible than rigid distance preservation. MSNP is evaluated in three applications: data visualization for faces image, face recognition and palmprint recognition. Experimental results on several benchmark databases suggest that the proposed MSNP provides a unsupervised feature extraction approach with powerful pattern revealing capability for complex manifold data.     

复杂高维数据的密度峰值快速搜索聚类算法

机器学习的无监督聚类算法已被广泛应用于各种目标识别任务。基于密度峰值的快速搜索聚类算法(DPC)能快速有效地确定聚类中心点和类个数,但在处理复杂分布形状的数据和高维图像数据时仍存在聚类中心点不容易确定、类数偏少等问题。为了提高其处理复杂高维数据的鲁棒性,文中提出了一种基于学习特征表示的密度峰值快速搜索聚类算法(AE-MDPC)。该算法采用无监督的自动编码器(AutoEncoder)学出数据的最优特征表示,结合能刻画数据全局一致性的流形相似性,提高了同类数据间的紧致性和不同类数据间的分离性,促使潜在类中心点的密度值成为局部最大。在4个人工数据集和4个真实图像数据集上将AE-MDPC与经典的K-means,DBSCAN,DPC算法以及结合了PCA的DPC算法进行比较。实验结果表明,在外部评价指标聚类精度、内部评价指标调整互信息和调整兰德指数上,AE-MDPC的聚类性能优于对比算法,而且提供了更好的可视化性能。总之,基于特征表示学习且结合流形距离的AE-MDPC算法能有效地处理复杂流形数据和高维图像数据。     

一种在源数据稀疏情况下的流形学习算法研究

传统的流形学习算法能有效地学习出高维采样数据的低维嵌入坐标,但也存在一些不足,如不能处理稀疏的样本数据.针对这些缺点,提出了一种基于局部映射的直接求解线性嵌入算法(Solving Directly Linear Embedding,简称SDLE).通过假定低维流形的整体嵌入函数,将流形映射赋予局部光滑的约束,应用核方法将高维空间的坐标投影到特征空间,最后构造出在低维空间的全局坐标.SDLE算法解决了在源数据稀疏情况下的非线性维数约简问题,这是传统的流形学习算法没有解决的问题.通过实验说明了SDLE算法研究的有效性.     

基于互信息的多块k近邻故障监测及诊断

由于传统的k近邻故障监测不考虑过程的局部信息,只建立一个全局模型,因此提出一种基于互信息的多块k近邻故障监测方法。首先,考虑建模数据的非线性和非高斯等特性,基于变量间的互信息进行子块构建;然后,利用k近邻方法对每个子块进行建模与监测,子块中的k近邻模型反映了更多的过程局部特征;最后,将所有子块的监测结果通过贝叶斯推断方法进行融合,并采用基于马氏距离的故障诊断方法辨识故障源。通过对田纳西-伊斯曼过程和高炉炼铁过程中的应用仿真,监测结果表明所提方法的可行性和有效性。     

An improved local tangent space alignment method for manifold learning

Principal component analysis (PCA) is widely used in recently proposed manifold learning algorithms to provide approximate local tangent spaces. However, such approximations provided by PCA may be inaccurate when local neighborhoods of the data manifold do not lie in or close to a linear subspace. Furthermore, the approximated tangent spaces can not fit the change in data distribution density. In this paper, a new method is proposed for providing faithful approximations to the local tangent spaces of a data manifold, which is proved to be more accurate than PCA. With this new method, an improved local tangent space alignment (ILTSA) algorithm is developed, which can efficiently recover the geometric structure of data manifolds even in the case when data are sparse or non-uniformly distributed. Experimental results are presented to illustrate the better performance of ILTSA on both synthetic data and image data.     

改进的有监督的局部线性嵌入算法及实验演示

流形学习方法中的LLE算法可以将高维数据在保持局部邻域结构的条件下降维到低维流形子空间中．并得到与原样本集具有相似局部结构的嵌入向量集合。LLE算法在数据降维处理过程中没有考虑样本的分类信息。针对这些问题进行研究,提出改进的有监督的局部线性嵌人算法（MSLLE）,并利用MatLab对该改进算法的实现效果同LLE进行实验演示比较。通过实验演示表明,MSLLE算法较LLE算法可以有利于保持数据点本身内部结构。