Feature extraction and dimensionality reduction algorithms and their applications in vowel recognition

Feature extraction is an important component of a pattern recognition system. It performs two tasks: transforming input parameter vector into a feature vector and/or reducing its dimensionality. A well-defined feature extraction algorithm makes the classification process more effective and efficient. Two popular methods for feature extraction are linear discriminant analysis (LDA) and principal component analysis (PCA). In this paper, the minimum classification error (MCE) training algorithm (which was originally proposed for optimizing classifiers) is investigated for feature extraction. A generalized MCE (GMCE) training algorithm is proposed to mend the shortcomings of the MCE training algorithm. LDA, PCA, and MCE and GMCE algorithms extract features through linear transformation. Support vector machine (SVM) is a recently developed pattern classification algorithm, which uses non-linear kernel functions to achieve non-linear decision boundaries in the parametric space. In this paper, SVM is also investigated and compared to linear feature extraction algorithms.     

Multiresolution recognition of unconstrained handwritten numerals with wavelet transform and multilayer cluster neural network

In this paper, we propose a new scheme for multiresolution recognition of unconstrained handwritten numerals using wavelet transform and a simple multilayer cluster neural network. The proposed scheme consists of two stages: a feature extraction stage for extracting multiresolution features with wavelet transform, and a classification stage for classifying unconstrained handwritten numerals with a simple multilayer cluster neural network. In order to verify the performance of the proposed scheme, experiments with unconstrained handwritten numeral database of Concordia University of Canada, Electro-Technical Laboratory of Japan, and Electronics and Telecommunications Research Institute of Korea were performed. The error rates were 3.20%, 0.83%, and 0.75%, respectively. These results showed that the proposed scheme is very robust in terms of various writing styles and sizes.     

Fractional-order embedding canonical correlation analysis and its applications to multi-view dimensionality reduction and recognition

Due to the noise disturbance and limited number of training samples, within-set and between-set sample covariance matrices in canonical correlation analysis (CCA) usually deviate from the true ones. In this paper, we re-estimate within-set and between-set covariance matrices to reduce the negative effect of this deviation. Specifically, we use the idea of fractional order to respectively correct the eigenvalues and singular values in the corresponding sample covariance matrices, and then construct fractional-order within-set and between-set scatter matrices which can obviously alleviate the problem of the deviation. On this basis, a new approach is proposed to reduce the dimensionality of multi-view data for classification tasks, called fractional-order embedding canonical correlation analysis (FECCA). The proposed method is evaluated on various handwritten numeral, face and object recognition problems. Extensive experimental results on the CENPARMI, UCI, AT&T, AR, and COIL-20 databases show that FECCA is very effective and obviously outperforms the existing joint dimensionality reduction or feature extraction methods in terms of classification accuracy. Moreover, its improvements for recognition rates are statistically significant on most cases below the significance level 0.05.     

Verification—a method of enhancing the recognizers of isolated and touching handwritten numerals

This paper investigates verification schemes and their applications to the recognition of both isolated and touching handwritten numerals. Definitions and functionality analyses of the verifiers are given. The measurement of precision rate is used to assess the system reliability in a class-specific manner. Verification-enhanced systems are proposed with extensive experiments conducted on both isolated and touching numerals. Two databases for touching numerals are built/organized to serve as standard data sets. Experimental results indicate a substantial improvement in system precision rates by the verification scheme, which proves the effectiveness of the proposed systems and justifies the important role of verifiers in OCR systems.     

Recognition of handwritten Bengali characters: a novel multistage approach

A multistage scheme for the recognition of handwritten Bengali characters is introduced. An analysis of the Bengali character set has been carried out to isolate specific high-level features that can help in forming smaller sub-groups within the character set. This analysis demonstrates how detection of these various high-level features might help formulate successful multistage OCR design. A multiple expert decision combination hierarchy has been exploited to achieve higher performance from the proposed multi-stage framework.     

Bilinear Lanczos components for fast dimensionality reduction and feature extraction

Matrix-based methods such as generalized low rank approximations of matrices (GLRAM) have gained wide attention from researchers in pattern recognition and machine learning communities. In this paper, a novel concept of bilinear Lanczos components (BLC) is introduced to approximate the projection vectors obtained from eigen-based methods without explicit computing eigenvectors of the matrix. This new method sequentially reduces the reconstruction error for a Frobenius-norm based optimization criterion, and the resulting approximation performance is thus improved during successive iterations. In addition, a theoretical clue for selecting suitable dimensionality parameters without losing classification information is presented in this paper. The BLC approach realizes dimensionality reduction and feature extraction by using a small number of Lanczos components. Extensive experiments on face recognition and image classification are conducted to evaluate the efficiency and effectiveness of the proposed algorithm. Results show that the new approach is competitive with the state-of-the-art methods, while it has a much lower training cost.     

基于归一化背景方向特征的脱机手写汉字识别

对4方向背景方向特征进行了改进,提出了8方向背景特征描述方法。与4方向背景方向特征描述方法相比,改进后的特征描述方法可以从0°、45°、90°、135°、180°、225°、270°、315°共8个方向来对汉字图像进行考察,从而进一步提高描述的精度。此外,为了消除笔划粗细的影响,还对背景方向特征进行了归一化处理。实验结果表明改进后的归一化8方向背景方向特征具有更高的识别精度。     

手写体汉字扇形弹性网格特征提取的新方法

近年来,网格方向特征已广泛应用于许多手写体汉字识别系统中,并认为是目前较成熟的手写体汉字特征之一,网格技术和方向分解是网格方向特征的两个关键技术,该文提出了一种新的网格技术一扇形网格法,结合边缘方向分解技术^[1],构造了一种新的手写体汉字特征提取方法一扇形网格边缘方向分解特征,实验结果验证了本方法的有效效。     

Eigen-deformations for elastic matching based handwritten character recognition

Deformations in handwritten characters have category-dependent tendencies. In this paper, the estimation and the utilization of such tendencies called eigen-deformations are investigated for the better performance of elastic matching based handwritten character recognition. The eigen-deformations are estimated by the principal component analysis of actual deformations automatically collected by the elastic matching. From experimental results it was shown that typical deformations of each category can be extracted as the eigen-deformations. It was also shown that the recognition performance can be improved significantly by using the eigen-deformations for the detection of overfitting, which is the main cause of the misrecognition in the elastic matching based recognition methods.     

A novel cascade ensemble classifier system with a high recognition performance on handwritten digits

This paper presents a novel cascade ensemble classifier system for the recognition of handwritten digits. This new system aims at attaining a very high recognition rate and a very high reliability at the same time, in other words, achieving an excellent recognition performance of handwritten digits. The trade-offs among recognition, error, and rejection rates of the new recognition system are analyzed. Three solutions are proposed: (i) extracting more discriminative features to attain a high recognition rate, (ii) using ensemble classifiers to suppress the error rate and (iii) employing a novel cascade system to enhance the recognition rate and to reduce the rejection rate. Based on these strategies, seven sets of discriminative features and three sets of random hybrid features are extracted and used in the different layers of the cascade recognition system. The novel gating networks (GNs) are used to congregate the confidence values of three parallel artificial neural networks (ANNs) classifiers. The weights of the GNs are trained by the genetic algorithms (GAs) to achieve the overall optimal performance. Experiments conducted on the MNIST handwritten numeral database are shown with encouraging results: a high reliability of 99.96% with minimal rejection, or a 99.59% correct recognition rate without rejection in the last cascade layer.     

An optimal orthonormal system for discriminant analysis

This paper proposes a new discriminant analysis with orthonormal coordinate axes of the feature space. In general, the number of coordinate axes of the feature space in the traditional discriminant analysis depends on the number of pattern classes. Therefore, the discriminatory capability of the feature space is limited considerably. The new discriminant analysis solves this problem completely. In addition, it is more powerful than the traditional one in so far as the discriminatory power and the mean error probability for coordinate axes are concerned. This is also shown by a numerical example.     

基于小波包分析的在线手写签名认证方法

针对在线手写签名难以提取有效特征的实际情况,提出用小波包分解和单支重构来构造能量特征向量的方法,直接利用各频段成分能量的变化来反映签名的动态特征。用该方法构造的特征向量能突出反映签名的动态特征,通过RBF神经网络进行签名识别。实验数据表明,采用此方法,识别的正确率可达96.75%,平均错误率ERR=3.34%,其性能是较满意的。     

Feature extraction by learning Lorentzian metric tensor and its extensions

We develop a supervised dimensionality reduction method, called Lorentzian discriminant projection (LDP), for feature extraction and classification. Our method represents the structures of sample data by a manifold, which is furnished with a Lorentzian metric tensor. Different from classic discriminant analysis techniques, LDP uses distances from points to their within-class neighbors and global geometric centroid to model a new manifold to detect the intrinsic local and global geometric structures of data set. In this way, both the geometry of a group of classes and global data structures can be learnt from the Lorentzian metric tensor. Thus discriminant analysis in the original sample space reduces to metric learning on a Lorentzian manifold. We also establish the kernel, tensor and regularization extensions of LDP in this paper. The experimental results on benchmark databases demonstrate the effectiveness of our proposed method and the corresponding extensions.     

Nonlinear dimensionality reduction using a temporal coherence principle

Temporal coherence principle is an attractive biologically inspired learning rule to extract slowly varying features from quickly varying input data. In this paper we develop a new Nonlinear Neighborhood Preserving (NNP) technique, by utilizing the temporal coherence principle to find an optimal low dimensional representation from the original high dimensional data. NNP is based on a nonlinear expansion of the original input data, such as polynomials of a given degree. It can be solved by the eigenvalue problem without using gradient descent and is guaranteed to find the global optimum. NNP can be viewed as a nonlinear dimensionality reduction framework which takes into consideration both time series and data sets without an obvious temporal structure. According to different situations, we introduce three algorithms of NNP, named NNP-1, NNP-2, and NNP-3. The objective function of NNP-1 is equal to Slow Feature Analysis (SFA), and it works well for time series such as image sequences. NNP-2 artificially constructs time series consisting of neighboring points for data sets without a clear temporal structure such as image data. NNP-3 is proposed for classification tasks, which can minimize the distances of neighboring points in the embedding space and ensure that the remaining points are as far apart as possible simultaneously. Furthermore, the kernel extension of NNP is also discussed in this paper. The proposed algorithms work very well on some image sequences and image data sets compared to other methods. Meanwhile, we perform the classification task on the MNIST handwritten digit database using the supervised NNP algorithms. The experimental results demonstrate that NNP is an effective technique for nonlinear dimensionality reduction tasks.     

支票大写金额的印刷体与手写体辨识

针对银行支票大写金额识别工作中的手写体和印刷体辨识,抽取了4组分类特征,较好地体现了印刷体字符和手写体字符的区别。并且在构造分类器时,介绍了求取各分类特征对识别结果影响大小的权值向量的方法。在银行提供的大量真实票据上进行了验证,并且应用于实际的银行自动支票处理系统,取得了很好的效果。     

一种组合特征抽取的新方法

该文提出了一种基于特征级融合的特征抽取新方法,首先,给出了一种合理的特征融合策略,即利用复向量给出组合特征的表示,将特征空间从实向量空间拓广到复向量空间,然后,发展了具有统计不相关性的鉴别分析的理论,并将其用于复向量空间内最优鉴别特征的抽取,最后,在Concordia大学的CENPARMI手写体阿拉伯数字数据库以及南京理工大学NUST603HW手写汉字库上的试验结果表明,所提出的组合特征抽取方法不仅具有很强的维数压缩能力,而且较大幅度地提高了识别率。     

Zoning methods for handwritten character recognition: A survey

This paper presents a survey on zoning methods for handwritten character recognition. Through the analysis of the relevant literature in the field, the most valuable zoning methods are presented in terms of both topologies and membership functions. Throughout the paper, diverse zoning topologies are presented based on both static and adaptive approaches. Concerning static approaches, uniform and non-uniform zoning strategies are discussed. When adaptive zonings are considered, manual and automatic strategies for optimal zoning design are illustrated as well as the most appropriate zoning representation techniques. In addition, the role of membership functions for zoning-based classification is highlighted and the diverse approaches to membership function selection are presented. Concerning global membership functions, the paper introduces order-based approaches as well as fuzzy approaches using border-based and ranked-based fuzzy membership values. Concerning local membership functions, the recent parameter-based approaches are described, in which the optimal membership-function is selected for each zone of the zoning method. Finally, a comparative analysis on the performance of zoning methods is presented and the most interesting approaches are focused on in terms of topology design and membership function selection. A list of selected references is provided as a useful tool for interested researchers working in the field.     

An empirical evaluation on dimensionality reduction schemes for dissimilarity-based classifications

This paper presents an empirical evaluation on the methods of reducing the dimensionality of dissimilarity spaces for optimizing dissimilarity-based classifications (DBCs). One problem of DBCs is the high dimensionality of the dissimilarity spaces. To address this problem, two kinds of solutions have been proposed in the literature: prototype selection (PS) based methods and dimension reduction (DR) based methods. Although PS-based and DR-based methods have been explored separately by many researchers, not much analysis has been done on the study of comparing the two. Therefore, this paper aims to find a suitable method for optimizing DBCs by a comparative study. Our empirical evaluation, obtained with the two approaches for an artificial and three real-life benchmark databases, demonstrates that DR-based methods, such as principal component analysis (PCA) and linear discriminant analysis (LDA) based methods, generally improve the classification accuracies more than PS-based methods. Especially, the experimental results demonstrate that PCA is more useful for the well-represented data sets, while LDA is more helpful for the small sample size problems.     

Corpus-based HIT-MW database for offline recognition of general-purpose Chinese handwritten text

A Chinese handwriting database named HIT-MW is presented to facilitate the offline Chinese handwritten text recognition. Both the writers and the texts for handcopying are carefully sampled with a systematic scheme. To collect naturally written handwriting, forms are distributed by postal mail or middleman instead of face to face. The current version of HIT-MW includes 853 forms and 186,444 characters that are produced under an unconstrained condition without preprinted character boxes. The statistics show that the database has an excellent representation of the real handwriting. Many new applications concerning real handwriting recognition can be supported by the database.     

From classifiers to discriminators: A nearest neighbor rule induced discriminant analysis

The current discriminant analysis method design is generally independent of classifiers, thus the connection between discriminant analysis methods and classifiers is loose. This paper provides a way to design discriminant analysis methods that are bound with classifiers. We begin with a local mean based nearest neighbor (LM-NN) classifier and use its decision rule to supervise the design of a discriminator. Therefore, the derived discriminator, called local mean based nearest neighbor discriminant analysis (LM-NNDA), matches the LM-NN classifier optimally in theory. In contrast to that LM-NNDA is a NN classifier induced discriminant analysis method, we further show that the classical Fisher linear discriminant analysis (FLDA) is a minimum distance classifier (i.e. nearest Class-mean classifier) induced discriminant analysis method. The proposed LM-NNDA method is evaluated using the CENPARMI handwritten numeral database, the NUST603 handwritten Chinese character database, the ETH80 object category database and the FERET face image database. The experimental results demonstrate the performance advantage of LM-NNDA over other feature extraction methods with respect to the LM-NN (or NN) classifier.