Pattern recognition of manufacturing process signals using Gaussian mixture models-based recognition systems

Unnatural patterns exhibited in manufacturing processes can be associated with certain assignable causes for process variation. Hence, accurate identification of various process patterns (PPs) can significantly narrow down the scope of possible causes that must be investigated, and speed up the troubleshooting process. This paper proposes a Gaussian mixture models (GMM)-based PP recognition (PPR) model, which employs a collection of several GMMs trained for PPR. By using statistical features and wavelet energy features as the input features, the proposed PPR model provides more simple training procedure and better generalization performance than using single recognizer, and hence is easier to be used by quality engineers and operators. Furthermore, the proposed model is capable of adapting novel PPs through using a dynamic modeling scheme. The simulation results indicate that the GMM-based PPR model shows good detection and recognition of current PPs and adapts further novel PPs effectively. Analysis from this study provides guidelines in developing GMM – based SPC recognition systems.     

高斯超向量-支持向量机鉴别性语种识别系统

支持向量机在语种识别技术中获得了广泛的研究和应用,并且达到和传统混合高斯模型相当的性能。高斯超向量-支持向量机系统将高斯混合模型与支持向量机有效地结合起来,采用高斯超向量核函数,以支持向量机作为后端分类器。重点介绍基于高斯超向量-支持向量机的语种识别系统,并和传统的高斯混合模型系统进行比较。在美国国家标准技术研究院2003年和2007年语种识别评测数据集上进行实验。实验结果表明,高斯超向量-支持向量机系统相对于混合高斯模型建模的方法,在长时数据上有较明显的性能优势。     

Pattern recognition using Markov random field models

In this paper, we propose Markov random field models for pattern recognition, which provide a flexible and natural framework for modelling the interactions between spatially related random variables in their neighbourhood systems. The proposed approach is superior to conventional approaches in many aspects. This paper introduces the concept of states into Markov random filed models, presents a theoretic analysis of the approach, discusses issues of designing neighbourhood system and cliques, and analyses properties of the models. We have applied our method to the recognition of unconstrained handwritten numerals. The experimental results show that the proposed approach can achieve high performance.     

Pattern recognition categories

We present a categorical definition of pattern recognition, which unifies algebraic and topological formalisms. We state and prove a recognition theorem, and show how to define equivalence and invariants for both settings.     

Mixture control chart patterns recognition using independent component analysis and support vector machine

Effective recognition of control chart patterns (CCPs) is an important issue since abnormal patterns exhibited in control charts can be associated with certain assignable causes which affect the process. Most of the existing studies assume that the observed process data which needs to be recognized are basic types of abnormal CCPs. However, in practical situations, the observed process data could be mixture patterns, which consist of two basic CCPs combined together. In this study, a hybrid scheme using independent component analysis (ICA) and support vector machine (SVM) is proposed for CCPs recognition. The proposed hybrid ICA-SVM scheme initially applies an ICA to the mixture patterns in order to generate independent components (ICs). The hidden basic patterns of the mixture patterns can be discovered in these ICs. The ICs can then serve as the input variables of the SVM for building a CCP recognition model. Experimental results revealed that the proposed scheme is able to effectively recognize mixture control chart patterns and outperform the single SVM models, which did not use an ICA as a preprocessor.     

Pattern recognition of collections

The task of simultaneously classifying a set of objects has unusual problems. Three categories of such problems are identified. Characteristics of context-dependent and context-independent classifications are considered. The effects of having parallepipeds or ellipsoids as decision regions are compared.     

Pattern classes: A technique for recovering their distributions

Many statistical pattern-recognition techniques depend for their application on the generation of one or more prototype patterns for each decision class. In turn, the determination of prototypes is dependent on the underlying probability distribution associated with a given class and that distribution's relationship to the distributions associated with the remaining classes. If these distributions are known, the problem of classification is considerably less complex than if they are unknown. The problem of recovering an unknown underlying distribution is one that has received considerable attention. The results thus far, however, are nonpractical. A practical technique that makes use of certain parameters related to sample size is presented and verified.This work was supported in part by the Office of Naval Research under Contract No. N00014-72-C-0459.Former Member of the Professional Staff, Ultrasystems, Inc.     

Pattern recognition using type-II fuzzy sets

Type II fuzzy sets are a generalization of the ordinary fuzzy sets in which the membership value for each member of the set is itself a fuzzy set in [0,1]. We introduce a similarity measure for measuring the similarity, or compatibility, between two type-II fuzzy sets. With this new similarity measure we show that type-II fuzzy sets provide us with a natural language for formulating classification problems in pattern recognition.     

Conception of complex probabilistic neural network system for classification of partial discharge patterns using multifarious inputs

Pattern recognition has a long history within electrical engineering but has recently become much more widespread as the automated capture of signal and images has been cheaper. Very many of the application of neural networks are to classification, and so are within the field of pattern recognition and classification. In this paper, we explore how probabilistic neural networks fit into the earlier framework of pattern recognition of partial discharge patterns since the PD patterns are an important tool for diagnosis of HV insulation systems. Skilled humans can identify the possible insulation defects in various representations of partial discharge (PD) data. One of the most widely used representation is phase resolved PD (PRPD) patterns. Also this paper describes a method for the automated recognition of PRPD patterns using a novel complex probabilistic neural network system for the actual classification task. The efficacy of composite neural network developed using probabilistic neural network is examined.     

Source cell-phone recognition from recorded speech using non-speech segments

In a recent study, we have introduced the problem of identifying cell-phones using recorded speech and shown that speech signals convey information about the source device, making it possible to identify the source with some accuracy. In this paper, we consider recognizing source cell-phone microphones using non-speech segments of recorded speech. Taking an information-theoretic approach, we use Gaussian Mixture Model (GMM) trained with maximum mutual information (MMI) to represent device-specific features. Experimental results using Mel-frequency and linear frequency cepstral coefficients (MFCC and LFCC) show that features extracted from the non-speech segments of speech contain higher mutual information and yield higher recognition rates than those from speech portions or the whole utterance. Identification rate improves from 96.42% to 98.39% and equal error rate (EER) reduces from 1.20% to 0.47% when non-speech parts are used to extract features. Recognition results are provided with classical GMM trained both with maximum likelihood (ML) and maximum mutual information (MMI) criteria, as well as support vector machines (SVMs). Identification under additive noise case is also considered and it is shown that identification rates reduces dramatically in case of additive noise.     

Pattern classification of non-sparse data using optimal interpolative nets

In this paper we introduce two pattern classifiers for non-sparse data (i.e. data with overlapping class distributions) which use the optimal interpolative neural network (OI-net), derived by one of the authors based on a generalized Fock (GF) space formulation. We present a statistical pattern classifier operating as a two-stage algorithm. The first stage consists of a pre-processing operation involving a k-N N editing of the original training set T. The operation results in a new training set, T_e, which in the second stage is classified by an OI-net constructed by the recursive least squares algorithm. We also propose a new data specific classifier which has an additional third computational stage, in which samples of the original training set are added to the network piece by piece until satisfactory classification results are obtained. During the computation process the training set is iteratively updated until the number of mis-classified samples is minimized. The performance of these two classifiers has been evaluated in some illustrative examples.     

模式识别的主要方法及其应用

模式识别技术现在已经在各个领域得到广泛应用。本文对其理论基础与应用作了详细介绍与阐述。介绍了模式识别的基本概念、主要方法、模式识别的应用及其发展趋势。     

High-dimensional data clustering

Clustering in high-dimensional spaces is a difficult problem which is recurrent in many domains, for example in image analysis. The difficulty is due to the fact that high-dimensional data usually exist in different low-dimensional subspaces hidden in the original space. A family of Gaussian mixture models designed for high-dimensional data which combine the ideas of subspace clustering and parsimonious modeling are presented. These models give rise to a clustering method based on the expectation-maximization algorithm which is called high-dimensional data clustering (HDDC). In order to correctly fit the data, HDDC estimates the specific subspace and the intrinsic dimension of each group. Experiments on artificial and real data sets show that HDDC outperforms existing methods for clustering high-dimensional data.     

Stochastic learning-based weak estimation of multinomial random variables and its applications to pattern recognition in non-stationary environments

In this paper, we formally present a novel estimation method, referred to as the Stochastic Learning Weak Estimator (SLWE), which yields the estimate of the parameters of a binomial distribution, where the convergence of the estimate is weak, i.e. with regard to the first and second moments. The estimation is based on the principles of stochastic learning. The mean of the final estimate is independent of the scheme's learning coefficient, λ, and both the variance of the final distribution and the speed decrease with λ. Similar results are true for the multinomial case, except that the equations transform from being of a scalar type to be of a vector type. Amazingly enough, the speed of the latter only depends on the same parameter, λ, which turns out to be the only non-unity eigenvalue of the underlying stochastic matrix that determines the time-dependence of the estimates. An empirical analysis on synthetic data shows the advantages of the scheme for non-stationary distributions. The paper also briefly reports (without detailed explanation) conclusive results that demonstrate the superiority of SLWE in pattern-recognition-based data compression, where the underlying data distribution is non-stationary. Finally, and more importantly, the paper includes the results of two pattern recognition exercises, the first of which involves artificial data, and the second which involves the recognition of the types of data that are present in news reports of the Canadian Broadcasting Corporation (CBC). The superiority of the SLWE in both these cases is demonstrated.     

The use of kernel principal component analysis to model data distributions

We describe the use of kernel principal component analysis (KPCA) to model data distributions in high-dimensional spaces. We show that a previous approach to representing non-linear data constraints using KPCA is not generally valid, and introduce a new ‘proximity to data’ measure that behaves correctly. We investigate the relation between this measure and the actual density for various low-dimensional data distributions. We demonstrate the effectiveness of the method by applying it to the higher-dimensional case of modelling an ensemble of images of handwritten digits, showing how it can be used to extract the digit information from noisy input images.     

基于说话人模型聚类的说话人识别

为了提高说话人识别系统的识别效率,提出一种基于说话人模型聚类的说话人识别方法,通过近似KL距离将相似的说话人模型聚类,为每类确定类中心和类代表,构成分级说话人识别模型。测试时先通过计算测试矢量与类中心或类代表之间的距离选择类,再通过计算测试矢量与选中类中的说话人模型之间对数似然度确定目标说话人,这样可以大大减少计算量。实验结果显示,在相同条件下,基于说话人模型聚类的说话人识别的识别速度要比传统的GMM的识别速度快4倍,但是识别正确率只降低了0.95%。因此,与传统GMM相比,基于说话人模型聚类的说话人识别能在保证识别正确率的同时大大提高识别速度。     

Variance estimation and ranking of target tracking position errors modeled using Gaussian mixture distributions

In this paper, variance estimation and ranking methods are developed for stochastic processes modeled by Gaussian mixture distributions. It is shown that the variance estimate from a Gaussian mixture distribution has the same properties as the variance estimate from a single Gaussian distribution based on a reduced number of samples. Hence, well-known tools for variance estimation and ranking of single Gaussian distributions can be applied to Gaussian mixture distributions. As an application example, we present optimization of sensor processing order in the sequential multi-target multi-sensor joint probabilistic data association (MSJPDA) algorithm.     

Symbol recognition in electrical diagrams using probabilistic graph matching

In this paper a method is proposed to recognize symbols in electrical diagrams based on probabilistic matching. The skeletons of the symbols are represented by graphs. After finding the pose of the graph (orientation, translation, scale) by a bounded search for a minimum error transformation, the observed graph is matched to the class models and the likelihood of the match is calculated. Results are given for computer-generated symbols and hand drawn symbols with and without a template. Error rates range from <1% to 8%.     

Effective recognition of control chart patterns in autocorrelated data using a support vector machine based approach

The effective recognition of unnatural control chart patterns (CCPs) is a critical issue in statistical process control, as unnatural CCPs can be associated with specific assignable causes adversely affecting the process. Machine learning techniques, such as artificial neural networks (ANNs), have been widely used in the research field of CCP recognition. However, ANN approaches can easily overfit the training data, producing models that can suffer from the difficulty of generalization. This causes a pattern misclassification problem when the training examples contain a high level of background noise (common cause variation). Support vector machines (SVMs) embody the structural risk minimization, which has been shown to be superior to the traditional empirical risk minimization principle employed by ANNs. This research presents a SVM-based CCP recognition model for the on-line real-time recognition of seven typical types of unnatural CCP, assuming that the process observations are AR(1) correlated over time. Empirical comparisons indicate that the proposed SVM-based model achieves better performance in both recognition accuracy and recognition speed than the model based on a learning vector quantization network. Furthermore, the proposed model is more robust toward background noise in the process data than the model based on a back propagation network. These results show the great potential of SVM methods for on-line CCP recognition.