Principal components selection for dimensionality reduction using discriminant information applied to fault diagnosis

The Principal Component Analysis is one of most applied dimensionality reduction techniques for process monitoring and fault diagnosis in industrial process. This work proposes a procedure based on the discriminant information contained in the principal components to determine the most significant ones in fault separability. The Tennessee Eastman Process industrial benchmark is used to illustrate the effectiveness of the proposal. The use of statistical hypothesis tests as a separability measure between multiple failures is proposed for the selection of the principal components. The classifier profile concept has been introduced for comparison purposes. Results show an improvement in the classification process when compared with traditional techniques and the StepWise selection. This has resulted in a better classification for a fixed number of components, or a smaller number of required components to obtain a prefixed error rate. In addition, the computational advantage is demonstrated.     

Feature extraction and dimensionality reduction algorithms and their applications in vowel recognition

Feature extraction is an important component of a pattern recognition system. It performs two tasks: transforming input parameter vector into a feature vector and/or reducing its dimensionality. A well-defined feature extraction algorithm makes the classification process more effective and efficient. Two popular methods for feature extraction are linear discriminant analysis (LDA) and principal component analysis (PCA). In this paper, the minimum classification error (MCE) training algorithm (which was originally proposed for optimizing classifiers) is investigated for feature extraction. A generalized MCE (GMCE) training algorithm is proposed to mend the shortcomings of the MCE training algorithm. LDA, PCA, and MCE and GMCE algorithms extract features through linear transformation. Support vector machine (SVM) is a recently developed pattern classification algorithm, which uses non-linear kernel functions to achieve non-linear decision boundaries in the parametric space. In this paper, SVM is also investigated and compared to linear feature extraction algorithms.     

Multiresolution recognition of unconstrained handwritten numerals with wavelet transform and multilayer cluster neural network

In this paper, we propose a new scheme for multiresolution recognition of unconstrained handwritten numerals using wavelet transform and a simple multilayer cluster neural network. The proposed scheme consists of two stages: a feature extraction stage for extracting multiresolution features with wavelet transform, and a classification stage for classifying unconstrained handwritten numerals with a simple multilayer cluster neural network. In order to verify the performance of the proposed scheme, experiments with unconstrained handwritten numeral database of Concordia University of Canada, Electro-Technical Laboratory of Japan, and Electronics and Telecommunications Research Institute of Korea were performed. The error rates were 3.20%, 0.83%, and 0.75%, respectively. These results showed that the proposed scheme is very robust in terms of various writing styles and sizes.     

Fractional-order embedding canonical correlation analysis and its applications to multi-view dimensionality reduction and recognition

Due to the noise disturbance and limited number of training samples, within-set and between-set sample covariance matrices in canonical correlation analysis (CCA) usually deviate from the true ones. In this paper, we re-estimate within-set and between-set covariance matrices to reduce the negative effect of this deviation. Specifically, we use the idea of fractional order to respectively correct the eigenvalues and singular values in the corresponding sample covariance matrices, and then construct fractional-order within-set and between-set scatter matrices which can obviously alleviate the problem of the deviation. On this basis, a new approach is proposed to reduce the dimensionality of multi-view data for classification tasks, called fractional-order embedding canonical correlation analysis (FECCA). The proposed method is evaluated on various handwritten numeral, face and object recognition problems. Extensive experimental results on the CENPARMI, UCI, AT&T, AR, and COIL-20 databases show that FECCA is very effective and obviously outperforms the existing joint dimensionality reduction or feature extraction methods in terms of classification accuracy. Moreover, its improvements for recognition rates are statistically significant on most cases below the significance level 0.05.     

手写粉笔数字自动识别方法研究

手写粉笔字在生产生活中应用广泛,针对手写粉笔字易与背景混淆的特点,提出了手写粉笔数字自动识别方法,分别通过粉笔字普适性分割算法和基于SVM的手写数字分层分类自动识别技术解决了分割和识别两个关键问题,并结合钢材板坯号自动识别系统的应用进行方法的实验,达到了较高的识别率,具有重大的应用价值和推广意义。     

Verification—a method of enhancing the recognizers of isolated and touching handwritten numerals

This paper investigates verification schemes and their applications to the recognition of both isolated and touching handwritten numerals. Definitions and functionality analyses of the verifiers are given. The measurement of precision rate is used to assess the system reliability in a class-specific manner. Verification-enhanced systems are proposed with extensive experiments conducted on both isolated and touching numerals. Two databases for touching numerals are built/organized to serve as standard data sets. Experimental results indicate a substantial improvement in system precision rates by the verification scheme, which proves the effectiveness of the proposed systems and justifies the important role of verifiers in OCR systems.     

Recognition of handwritten Bengali characters: a novel multistage approach

A multistage scheme for the recognition of handwritten Bengali characters is introduced. An analysis of the Bengali character set has been carried out to isolate specific high-level features that can help in forming smaller sub-groups within the character set. This analysis demonstrates how detection of these various high-level features might help formulate successful multistage OCR design. A multiple expert decision combination hierarchy has been exploited to achieve higher performance from the proposed multi-stage framework.     

Feature extraction and dimensionality reduction by genetic programming based on the Fisher criterion

Abstract: Feature extraction helps to maximize the useful information within a feature vector, by reducing the dimensionality and making the classification effective and simple. In this paper, a novel feature extraction method is proposed: genetic programming (GP) is used to discover features, while the Fisher criterion is employed to assign fitness values. This produces non‐linear features for both two‐class and multiclass recognition, reflecting the discriminating information between classes. Compared with other GP‐based methods which need to generate c discriminant functions for solving c‐class (c>2) pattern recognition problems, only one single feature, obtained by a single GP run, appears to be highly satisfactory in this approach. The proposed method is experimentally compared with some non‐linear feature extraction methods, such as kernel generalized discriminant analysis and kernel principal component analysis. Results demonstrate the capability of the proposed approach to transform information from the high‐dimensional feature space into a single‐dimensional space by automatically discovering the relationships between data, producing improved performance.     

Bilinear Lanczos components for fast dimensionality reduction and feature extraction

Matrix-based methods such as generalized low rank approximations of matrices (GLRAM) have gained wide attention from researchers in pattern recognition and machine learning communities. In this paper, a novel concept of bilinear Lanczos components (BLC) is introduced to approximate the projection vectors obtained from eigen-based methods without explicit computing eigenvectors of the matrix. This new method sequentially reduces the reconstruction error for a Frobenius-norm based optimization criterion, and the resulting approximation performance is thus improved during successive iterations. In addition, a theoretical clue for selecting suitable dimensionality parameters without losing classification information is presented in this paper. The BLC approach realizes dimensionality reduction and feature extraction by using a small number of Lanczos components. Extensive experiments on face recognition and image classification are conducted to evaluate the efficiency and effectiveness of the proposed algorithm. Results show that the new approach is competitive with the state-of-the-art methods, while it has a much lower training cost.     

基于归一化背景方向特征的脱机手写汉字识别

对4方向背景方向特征进行了改进,提出了8方向背景特征描述方法。与4方向背景方向特征描述方法相比,改进后的特征描述方法可以从0°、45°、90°、135°、180°、225°、270°、315°共8个方向来对汉字图像进行考察,从而进一步提高描述的精度。此外,为了消除笔划粗细的影响,还对背景方向特征进行了归一化处理。实验结果表明改进后的归一化8方向背景方向特征具有更高的识别精度。     

手写体汉字扇形弹性网格特征提取的新方法

近年来，网格方向特征已广泛应用于许多手写体汉字识别系统中，并认为是目前较成熟的手写体汉字特征之一，网格技术和方向分解是网格方向特征的两个关键技术，该文提出了一种新的网格技术一扇形网格法，结合边缘方向分解技术^[1]，构造了一种新的手写体汉字特征提取方法一扇形网格边缘方向分解特征，实验结果验证了本方法的有效效。     

Eigen-deformations for elastic matching based handwritten character recognition

Deformations in handwritten characters have category-dependent tendencies. In this paper, the estimation and the utilization of such tendencies called eigen-deformations are investigated for the better performance of elastic matching based handwritten character recognition. The eigen-deformations are estimated by the principal component analysis of actual deformations automatically collected by the elastic matching. From experimental results it was shown that typical deformations of each category can be extracted as the eigen-deformations. It was also shown that the recognition performance can be improved significantly by using the eigen-deformations for the detection of overfitting, which is the main cause of the misrecognition in the elastic matching based recognition methods.     

Discriminative low-rank preserving projection for dimensionality reduction

As an effective image clustering tool, low-rank representation (LRR) can capture the intrinsic representation of the observed samples. However, firstly, the good representation does not mean good classification performance. Secondly, no projection matrix is obtained in the training stage, and it cannot deal with the new samples. By incorporating the discriminant analysis and the local neighborhood relationship of the original samples into the low-rank representation, a novel discriminative low-rank preserving projection (DLRPP) algorithm is presented for dimensionality reduction. In DLRPP, the global structure information can be captured by LRR, and the local geometricinformation is simultaneously preserved by the manifold regularization term. The constrained term is induced by the adaptive graph, which is obtained by low-rank representation coefficients. In addition, by introducing discriminant analysis constraint term, DLRPP can learn an optimal projection matrix for data dimensionality reduction. The numerous experiments on six public image datasets prove that the proposed DLRPP can obtain better recognition accuracy compared with the state-of-the-art feature extraction methods.     

A novel cascade ensemble classifier system with a high recognition performance on handwritten digits

This paper presents a novel cascade ensemble classifier system for the recognition of handwritten digits. This new system aims at attaining a very high recognition rate and a very high reliability at the same time, in other words, achieving an excellent recognition performance of handwritten digits. The trade-offs among recognition, error, and rejection rates of the new recognition system are analyzed. Three solutions are proposed: (i) extracting more discriminative features to attain a high recognition rate, (ii) using ensemble classifiers to suppress the error rate and (iii) employing a novel cascade system to enhance the recognition rate and to reduce the rejection rate. Based on these strategies, seven sets of discriminative features and three sets of random hybrid features are extracted and used in the different layers of the cascade recognition system. The novel gating networks (GNs) are used to congregate the confidence values of three parallel artificial neural networks (ANNs) classifiers. The weights of the GNs are trained by the genetic algorithms (GAs) to achieve the overall optimal performance. Experiments conducted on the MNIST handwritten numeral database are shown with encouraging results: a high reliability of 99.96% with minimal rejection, or a 99.59% correct recognition rate without rejection in the last cascade layer.     

An optimal orthonormal system for discriminant analysis

This paper proposes a new discriminant analysis with orthonormal coordinate axes of the feature space. In general, the number of coordinate axes of the feature space in the traditional discriminant analysis depends on the number of pattern classes. Therefore, the discriminatory capability of the feature space is limited considerably. The new discriminant analysis solves this problem completely. In addition, it is more powerful than the traditional one in so far as the discriminatory power and the mean error probability for coordinate axes are concerned. This is also shown by a numerical example.     

Feature extraction by learning Lorentzian metric tensor and its extensions

We develop a supervised dimensionality reduction method, called Lorentzian discriminant projection (LDP), for feature extraction and classification. Our method represents the structures of sample data by a manifold, which is furnished with a Lorentzian metric tensor. Different from classic discriminant analysis techniques, LDP uses distances from points to their within-class neighbors and global geometric centroid to model a new manifold to detect the intrinsic local and global geometric structures of data set. In this way, both the geometry of a group of classes and global data structures can be learnt from the Lorentzian metric tensor. Thus discriminant analysis in the original sample space reduces to metric learning on a Lorentzian manifold. We also establish the kernel, tensor and regularization extensions of LDP in this paper. The experimental results on benchmark databases demonstrate the effectiveness of our proposed method and the corresponding extensions.     

基于小波包分析的在线手写签名认证方法

针对在线手写签名难以提取有效特征的实际情况,提出用小波包分解和单支重构来构造能量特征向量的方法,直接利用各频段成分能量的变化来反映签名的动态特征。用该方法构造的特征向量能突出反映签名的动态特征,通过RBF神经网络进行签名识别。实验数据表明,采用此方法,识别的正确率可达96.75%,平均错误率ERR=3.34%,其性能是较满意的。     

Nonlinear dimensionality reduction using a temporal coherence principle

Temporal coherence principle is an attractive biologically inspired learning rule to extract slowly varying features from quickly varying input data. In this paper we develop a new Nonlinear Neighborhood Preserving (NNP) technique, by utilizing the temporal coherence principle to find an optimal low dimensional representation from the original high dimensional data. NNP is based on a nonlinear expansion of the original input data, such as polynomials of a given degree. It can be solved by the eigenvalue problem without using gradient descent and is guaranteed to find the global optimum. NNP can be viewed as a nonlinear dimensionality reduction framework which takes into consideration both time series and data sets without an obvious temporal structure. According to different situations, we introduce three algorithms of NNP, named NNP-1, NNP-2, and NNP-3. The objective function of NNP-1 is equal to Slow Feature Analysis (SFA), and it works well for time series such as image sequences. NNP-2 artificially constructs time series consisting of neighboring points for data sets without a clear temporal structure such as image data. NNP-3 is proposed for classification tasks, which can minimize the distances of neighboring points in the embedding space and ensure that the remaining points are as far apart as possible simultaneously. Furthermore, the kernel extension of NNP is also discussed in this paper. The proposed algorithms work very well on some image sequences and image data sets compared to other methods. Meanwhile, we perform the classification task on the MNIST handwritten digit database using the supervised NNP algorithms. The experimental results demonstrate that NNP is an effective technique for nonlinear dimensionality reduction tasks.     

支票大写金额的印刷体与手写体辨识

针对银行支票大写金额识别工作中的手写体和印刷体辨识，抽取了4组分类特征，较好地体现了印刷体字符和手写体字符的区别。并且在构造分类器时，介绍了求取各分类特征对识别结果影响大小的权值向量的方法。在银行提供的大量真实票据上进行了验证，并且应用于实际的银行自动支票处理系统，取得了很好的效果。     

一种组合特征抽取的新方法

该文提出了一种基于特征级融合的特征抽取新方法，首先，给出了一种合理的特征融合策略，即利用复向量给出组合特征的表示，将特征空间从实向量空间拓广到复向量空间，然后，发展了具有统计不相关性的鉴别分析的理论，并将其用于复向量空间内最优鉴别特征的抽取，最后，在Concordia大学的CENPARMI手写体阿拉伯数字数据库以及南京理工大学NUST603HW手写汉字库上的试验结果表明，所提出的组合特征抽取方法不仅具有很强的维数压缩能力，而且较大幅度地提高了识别率。