Identification of concurrent control chart patterns with singular spectrum analysis and learning vector quantization

Identification of unnatural control chart patterns (CCPs) from manufacturing process measurements is a critical task in quality control as these patterns indicate that the manufacturing process is out-of-control. Recently, there have been numerous efforts in developing pattern recognition and classification methods based on artificial neural network to automatically recognize unnatural patterns. Most of them assume that a single type of unnatural pattern exists in process data. Due to this restrictive assumption, severe performance degradations are observed in these methods when unnatural concurrent CCPs present in process data. To address this problem, this paper proposes a novel approach based on singular spectrum analysis (SSA) and learning vector quantization network to identify concurrent CCPs. The main advantage of the proposed method is that it can be applied to the identification of concurrent CCPs in univariate manufacturing processes. Moreover, there are no permutation and scaling ambiguities in the CCPs recovered by the SSA. These desirable features make the proposed algorithm an attractive alternative for the identification of concurrent CCPs. Computer simulations and a real application for aluminium smelting processes confirm the superior performance of proposed algorithm for sets of typical concurrent CCPs.     

Mixture control chart patterns recognition using independent component analysis and support vector machine

Effective recognition of control chart patterns (CCPs) is an important issue since abnormal patterns exhibited in control charts can be associated with certain assignable causes which affect the process. Most of the existing studies assume that the observed process data which needs to be recognized are basic types of abnormal CCPs. However, in practical situations, the observed process data could be mixture patterns, which consist of two basic CCPs combined together. In this study, a hybrid scheme using independent component analysis (ICA) and support vector machine (SVM) is proposed for CCPs recognition. The proposed hybrid ICA-SVM scheme initially applies an ICA to the mixture patterns in order to generate independent components (ICs). The hidden basic patterns of the mixture patterns can be discovered in these ICs. The ICs can then serve as the input variables of the SVM for building a CCP recognition model. Experimental results revealed that the proposed scheme is able to effectively recognize mixture control chart patterns and outperform the single SVM models, which did not use an ICA as a preprocessor.     

Recognition of concurrent control chart patterns using wavelet transform decomposition and multiclass support vector machines

Statistical process control charts have been widely utilized for monitoring process variation in many applications. Nonrandom patterns exhibited by control charts imply certain potential assignable causes that may deteriorate the process performance. Though some effective approaches to recognition of control chart patterns (CCPs) have been developed, most of them only focus on recognition and analysis of single patterns. A hybrid approach by integrating wavelet transform and improved particle swarm optimization-based support vector machine (P-SVM) for on-line recognition of concurrent CCPs is developed in this paper. A statistical correlation coefficient is used to determine whether the input pattern is a single or concurrent CCP. Based on wavelet transform, a raw concurrent pattern signal is decomposed into two basic pattern signals, which can be recognized by multiclass SVMs. The performance of the hybrid approach is evaluated by simulation experiments, and numerical and graphical results are provided to demonstrate that the proposed approach can perform effectively and efficiently in on-line CCP recognition task.     

A hybrid approach for identification of concurrent control chart patterns

Control chart patterns (CCPs) are widely used to identify the potential process problems in modern manufacturing industries. The earliest statistical techniques, including chart and R chart, are respectively used for monitoring process mean and process variance. Recently, pattern recognition techniques based on artificial neural network (ANN) are very popular to be applied to recognize unnatural CCPs. However, most of them are limited to recognize simple CCPs arising from single type of unnatural variation. In other words, they are incapable to handle the problem of concurrent CCPs where two types of unnatural variation exist together within the manufacturing process. To facilitate the research gap, this paper presents a hybrid approach based on independent component analysis (ICA) and decision tree (DT) to identify concurrent CCPs. Without loss of generality, six types of concurrent CCPs are used to validate the proposed method. Experimental results show that the proposed approach is very successful to handle most of the concurrent CCPs. The proposed method has two limitations in real application: it needs at least two concurrent CCPs to reconstruct their source patterns and it may be incapable to handle the concurrent pattern incurred by two correlated process (“upward trend” and “upward shift”).     

A hybrid system for SPC concurrent pattern recognition

Any nonrandom patterns shown in Statistical Process Control (SPC) charts imply possible assignable causes that may deteriorate the process performance. Hence, timely detecting and recognizing Control Chart Patterns (CCPs) for nonrandomness is very important in the implementation of SPC. Due to the limitations of run-rule-based approaches, Artificial Neural Networks (ANNs) have been resorted for detecting CCPs. However, most of the reported ANN approaches are only limited to recognize single basic patterns. Different from these approaches, this paper presents a hybrid approach by integrating wavelet method with ANNs for on-line recognition of CCPs including concurrent patterns. The main advantage of this approach is its capability of recognizing coexisted or concurrent patterns without training by concurrent patterns. The test results using simulated data have demonstrated the improvements and the effectiveness of the methodology with a success rate up to 91.41% in concurrent CCP recognition.     

Effective recognition of control chart patterns in autocorrelated data using a support vector machine based approach

The effective recognition of unnatural control chart patterns (CCPs) is a critical issue in statistical process control, as unnatural CCPs can be associated with specific assignable causes adversely affecting the process. Machine learning techniques, such as artificial neural networks (ANNs), have been widely used in the research field of CCP recognition. However, ANN approaches can easily overfit the training data, producing models that can suffer from the difficulty of generalization. This causes a pattern misclassification problem when the training examples contain a high level of background noise (common cause variation). Support vector machines (SVMs) embody the structural risk minimization, which has been shown to be superior to the traditional empirical risk minimization principle employed by ANNs. This research presents a SVM-based CCP recognition model for the on-line real-time recognition of seven typical types of unnatural CCP, assuming that the process observations are AR(1) correlated over time. Empirical comparisons indicate that the proposed SVM-based model achieves better performance in both recognition accuracy and recognition speed than the model based on a learning vector quantization network. Furthermore, the proposed model is more robust toward background noise in the process data than the model based on a back propagation network. These results show the great potential of SVM methods for on-line CCP recognition.     

Recognition of control chart patterns using fuzzy SVM with a hybrid kernel function

Accurate control chart patterns recognition (CCPR) plays an essential role in the implementation of control charts. However, it is a challenging problem since nonrandom control chart patterns (CCPs) are normally distorted by “common process variations”. In this paper, a novel method of CCPR by integrating fuzzy support vector machine (SVM) with hybrid kernel function and genetic algorithm (GA) is proposed. Firstly, two shape features and two statistical features that do not depend on the distribution parameters and number of samples are presented to explicitly describe the characteristics of CCPs. Then, a novel multiclass method based on fuzzy SVM with a hybrid kernel function is proposed. In this method, the influence of outliers on classification accuracy of SVM-based classifiers is weakened by assigning a degree of membership for every training sample. Meanwhile, a hybrid kernel function combining Gaussian kernel and polynomial kernel is adopted to further enhance the generalization ability of the classifiers. To solve the issue of features selection and parameters optimization, GA is used to simultaneously optimize the input features subsets and parameters of fuzzy SVM-based classifier. Finally, several simulation experiments and a real example are addressed to validate the feasibility and effectiveness of the proposed methodology. And the results of simulation experiments demonstrate that it can achieve excellent performance for CCPR and outperforms other approaches, such as learning vector quantization network, multi-layer perceptron network, probability neural network, fuzzy clustering and SVM, in term of recognition accuracy. The results of the practical cases manifest that the proposed method has application potential for solving the problem of control chart interpretation in real-world.     

Intelligent forecasting of time series based on evolving distributed Neuro-Fuzzy network

An evolving methodology based on Neuro-Fuzzy Takagi-Sugeno network (NF-TS) for distributed forecasting of univariate time series, is proposed. First, the unobservable components, or hidden patterns, are extracted from experimental data of the time series. Then, a distributed forecasting is performed separately for each component, considering an evolving NF-TS associated with each extracted pattern. The evolving NF-TS uses components data to adapt and adjust its structure, as the number of fuzzy rules increases or decreases according the behavior of the unobservable components. A recursive version of singular spectral analysis (SSA) technique is formulated, as one of the main contributions of this article, and it is applied to extract the components. The efficiency of proposed methodology is illustrated from results of comparison to others state-of-the-art techniques for forecasting of various univariate time series.     

在变长模式识别中利用测地距离的非线性插值进行特征选取

讨论了变长模式识别中的特征选择问题。采用基于测地距离（Geodesic Distance）的非线性插值来进行特征选择．使得变长的模式映射为等长的模式,从而可以使用传统的等长模式的方法来解决变长模式识别问题。用非特定说话人的汉语孤立词识别来验证提出方法的性能,并采用支持向量机（Support Vector Machine,SVM）作为基本的分类方法。实验结果表明,提出的方法可以获得比传统方法诸如线性插值更好的性能,而计算量仅有很少增加。     

Recognition of control chart patterns using improved selection of features

Recognition of various control chart patterns (CCPs) can significantly reduce the diagnostic search process. Feature-based approaches can facilitate efficient pattern recognition. The full potentiality of feature-based approaches can be achieved by using the optimal set of features. In this paper, a set of seven most useful features is selected using a classification and regression tree (CART)-based systematic approach for feature selection. Based on these features, eight most commonly observed CCPs are recognized using heuristic and artificial neural network (ANN) techniques. Extensive performance evaluation of the two types of recognizers reveals that both these recognizers result in higher recognition accuracy than the earlier reported feature-based recognizers. In this work, various features are extracted from the control chart plot of actual process data in such a way that their values become independent of the process mean and standard deviation. Thus, the developed feature-based CCP recognizers can be applicable to any general process.     

Recognition of control chart patterns using an intelligent technique

Control chart patterns (CCPs) are important statistical process control tools for determining whether a process is run in its intended mode or in the presence of unnatural patterns. Automatic recognition of abnormal patterns in control charts has seen increasing demands nowadays in the manufacturing processes. This paper presents a novel hybrid intelligent method for recognition of common types of CCP. The proposed method includes three main modules: the feature extraction module, the classifier module and optimization module. In the feature extraction module, a proper set of the shape features and statistical features is proposed as the efficient characteristic of the patterns. In the classifier module multilayer perceptron neural network and support vector machine (SVM) are investigated. In support vector machine training, the hyper-parameters have very important roles for its recognition accuracy. Therefore, in the optimization module, improved bees algorithm is proposed for selecting of appropriate parameters of the classifier. Simulation results show that the proposed algorithm has very high recognition accuracy.     

基于融合特征与支持向量机的控制图模式识别

为了提高控制图模式识别的精度, 将控制图模式的原始特征与形状特征相融合得到分类特征, 并采用支持向量机进行模式分类的控制图模式识别。融合所得特征既保持了控制图模式的原始特征所蕴涵的模式全局特性信息, 又通过引入形状特征对部分易混淆模式的局部几何特性进行强化, 使不同模式间的区分度得到有效提高; 而以支持向量机作为模式分类器保证方法在高维度特征和小样本条件下也能获得较好的识别性能。仿真实验结果表明所提方法的识别精度相比其他几种基于形状特征的控制图模式识别方法有明显提高。     

Feature-based recognition of control chart patterns

Control charts primarily in the form of chart are widely used to identify the situations when control actions will be needed for manufacturing systems. Various types of patterns are observed in control charts. Identification of these control chart patterns (CCPs) can provide clues to potential quality problems in the manufacturing process. Each type of control chart pattern has its own geometric shape and various related features can represent this shape. Feature-based approaches can facilitate efficient pattern recognition since extracted shape features represent the main characteristics of the patterns in a condensed form. In this paper, a set of eight new features, extraction of which does not call for utilizing the experience and skill of the user in any form, is presented. Two feature-based approaches using heuristics and artificial neural network (ANN) are developed, which are capable of recognizing eight most commonly observed CCPs including stratification and systematic patterns. Relative performances of the feature-based heuristic and feature-based ANN recognizers are extensively studied using synthetic pattern data. The feature-based ANN recognizer results in better recognition performance and generalization compared to the feature-based heuristic recognizer.     

A flexible sequence alignment approach on pattern mining and matching for human activity recognition

This paper proposes a flexible sequence alignment approach for pattern mining and matching in the recognition of human activities. During pattern mining, the proposed sequence alignment algorithm is invoked to extract out the representative patterns which denote specific activities of a person from the training patterns. It features high performance and robustness on pattern diversity. Besides, the algorithm evaluates the appearance probability of each pattern as weight and allows adapting pattern length to various human activities. Both of them are able to improve the accuracy of activity recognition. In pattern matching, the proposed algorithm adopts a dynamic programming based strategy to evaluate the correlation degree between each representative activity pattern and the observed activity sequence. It can avoid the trouble on segmenting the observed sequence. Moreover, we are able to obtain recognition results continuously. Besides, the proposed matching algorithm favors recognition of concurrent human activities with parallel matching. The experimental result confirms the high accuracy of human activity recognition by the proposed approach.     

基于增量学习SVM的人脸识别

为了提高人脸识别率,本文提出了一种增量学习支持矢量机(SVM)人脸识别方法,有效地对SVM的参数进行更新。提出的方法采用高斯概率模型描述SVM的参数统计特征,在无需额外存储训练数据的前提下,采用增量学习SVM的方式实现参数的更新;并通过最小化分类误差准则最大化SVM两类输出值概率分布间的距离。详细的实验以及与现有方法的比较结果表明,提出的识别方法具有更好的识别性能。     

Simultaneous process mean and variance monitoring using artificial neural networks

Control chart patterns (CCPs) can be employed to determine the behavior of a process. Hence, CCP recognition is an important issue for an effective process-monitoring system. Artificial neural networks (ANNs) have been applied to CCP recognition tasks and promising results have been obtained. It is well known that mean and variance control charts are usually implemented together and that these two charts are not independent of each other, especially for the individual measurements and moving range (X–R_m) charts. CCPs on the mean and variance charts can be associated independently with different assignable causes when corresponding process knowledge is available. However, ANN-based CCP recognition models for process mean and variance have mostly been developed separately in the literature with the other parameter assumed to be under control. Little attention has been given to the use of ANNs for monitoring the process mean and variance simultaneously. This study presents a real-time ANN-based model for the simultaneous recognition of both mean and variance CCPs. Three most common CCP types, namely shift, trend, and cycle, for both mean and variance are addressed in this work. Both direct data and selected statistical features extracted from the process are employed as the inputs of ANNs. The numerical results obtained using extensive simulation indicate that the proposed model can effectively recognize not only single mean or variance CCPs but also mixed CCPs in which mean and variance CCPs exist concurrently. Empirical comparisons show that the proposed model performs better than existing approaches in detecting mean and variance shifts, while also providing the capability of CCP recognition that is very useful for bringing the process back to the in-control condition. A demonstrative example is provided.     

SVM-based hierarchical architectures for handwritten Bangla character recognition

We propose support vector machine (SVM) based hierarchical classification schemes for recognition of handwritten Bangla characters. A comparative study is made among multilayer perceptron, radial basis function network and SVM classifier for this 45 class recognition problem. SVM classifier is found to outperform the other classifiers. A fusion scheme using the three classifiers is proposed which is marginally better than SVM classifier. It is observed that there are groups of characters having similar shapes. These groups are determined in two different ways on the basis of the confusion matrix obtained from SVM classifier. In the former, the groups are disjoint while they are overlapped in the latter. Another grouping scheme is proposed based on the confusion matrix obtained from neural gas algorithm. Groups are disjoint here. Three different two-stage hierarchical learning architectures (HLAs) are proposed using the three grouping schemes. An unknown character image is classified into a group in the first stage. The second stage recognizes the class within this group. Performances of the HLA schemes are found to be better than single stage classification schemes. The HLA scheme with overlapped groups outperforms the other two HLA schemes.     

基于类内方差归一化和SVM的说话人识别方法

支持向量机（SVM）由于其强大的分类能力,引起人们广泛的重视,并且成功地应用于说话人识别。其中基于GLDS核的SVM系统性能比较优异。引入类内方差归一化（WCCN）方法来处理SVM的输入特征向量,并和GLDS核相结合,提出一种基于类内方差归一化和SVM的说话人识别方法。该方法利用WCCN方法对SVM的输入特征向量进行变换,增强特征向量的类间区分能力,再采用GLDS核函数进行SVM的训练,以提高SVM的分类效果。实验表明,新方法是有效的,其性能优于基于GLDS核的SVM系统。     

A weighted support vector machine method for control chart pattern recognition

Manual inspection and evaluation of quality control data is a tedious task that requires the undistracted attention of specialized personnel. On the other hand, automated monitoring of a production process is necessary, not only for real time product quality assessment, but also for potential machinery malfunction diagnosis. For this reason, control chart pattern recognition (CCPR) methods have received a lot of attention over the last two decades. Current state-of-the-art control monitoring methodology includes K charts which are based on support vector machines (SVM). Although K charts have some profound benefits, their performance deteriorate when the learning examples for the normal class greatly outnumbers the ones for the abnormal class. Such problems are termed imbalanced and represent the vast majority of the real life control pattern classification problems. Original SVM demonstrate poor performance when applied directly to these problems. In this paper, we propose the use of weighted support vector machines (WSVM) for automated process monitoring and early fault diagnosis. We show the benefits of WSVM over traditional SVM, compare them under various fault scenarios. We evaluate the proposed algorithm in binary and multi-class environments for the most popular abnormal quality control patterns as well as a real application from wafer manufacturing industry.     

融合小波与2DPCA的SVM人脸识别

近年来,人脸识别由于其诱人的应用前景再次成为模式识别领域的研究热点。分析了小波变换、2DPCA以及SVM 3种方法在人脸识别中各自的优势,提出了融合小波和2DPCA进行SVM人脸识别的方法。首先对原始图像采用小波分解提取低频信息,忽略高频分量;然后利用2DPCA进行特征提取;最后把降维后的数据输入SVM进行分类识别。该方法在ORL、Yale人脸库上的实验表明,与传统的方法相比,不但可以提高识别率,而且所用时间明显减少。