Eigen-deformations for elastic matching based handwritten character recognition

Deformations in handwritten characters have category-dependent tendencies. In this paper, the estimation and the utilization of such tendencies called eigen-deformations are investigated for the better performance of elastic matching based handwritten character recognition. The eigen-deformations are estimated by the principal component analysis of actual deformations automatically collected by the elastic matching. From experimental results it was shown that typical deformations of each category can be extracted as the eigen-deformations. It was also shown that the recognition performance can be improved significantly by using the eigen-deformations for the detection of overfitting, which is the main cause of the misrecognition in the elastic matching based recognition methods.     

An MLP-orthogonal Gaussian mixture model hybrid model for Chinese bank check printed numeral recognition

A hybrid model based on the combination of an orthogonal Gaussian mixture model (OGMM) and a multilayer perceptron (MLP) is proposed in this paper that is to be used for Chinese bank check machine printed numeral recognition. The combination of MLP with OGMM produces a hybrid model with high recognition accuracy as well as an excellent outlier rejection ability. Experimental results show that the proposed model can satisfy the requirements of Chinese bank check printed numeral recognition where high recognition accuracy, high processing speed, and high reliability are needed. Correspondence to: Hui Zhu     

Regularized discriminant analysis for face recognition

This paper studies regularized discriminant analysis (RDA) in the context of face recognition. We check RDA sensitivity to different photometric preprocessing methods and compare its performance to other classifiers. Our study shows that RDA is better able to extract the relevant discriminatory information from training data than the other classifiers tested, thus obtaining a lower error rate. Moreover, RDA is robust under various lighting conditions while the other classifiers perform badly when no photometric method is applied.     

Multiresolution recognition of unconstrained handwritten numerals with wavelet transform and multilayer cluster neural network

In this paper, we propose a new scheme for multiresolution recognition of unconstrained handwritten numerals using wavelet transform and a simple multilayer cluster neural network. The proposed scheme consists of two stages: a feature extraction stage for extracting multiresolution features with wavelet transform, and a classification stage for classifying unconstrained handwritten numerals with a simple multilayer cluster neural network. In order to verify the performance of the proposed scheme, experiments with unconstrained handwritten numeral database of Concordia University of Canada, Electro-Technical Laboratory of Japan, and Electronics and Telecommunications Research Institute of Korea were performed. The error rates were 3.20%, 0.83%, and 0.75%, respectively. These results showed that the proposed scheme is very robust in terms of various writing styles and sizes.     

Neural network language models for off-line handwriting recognition

Unconstrained off-line continuous handwritten text recognition is a very challenging task which has been recently addressed by different promising techniques. This work presents our latest contribution to this task, integrating neural network language models in the decoding process of three state-of-the-art systems: one based on bidirectional recurrent neural networks, another based on hybrid hidden Markov models and, finally, a combination of both. Experimental results obtained on the IAM off-line database demonstrate that consistent word error rate reductions can be achieved with neural network language models when compared with statistical N-gram language models on the three tested systems. The best word error rate, 16.1%, reported with ROVER combination of systems using neural network language models significantly outperforms current benchmark results for the IAM database.     

Performance evaluation of pattern classifiers for handwritten character recognition

This paper describes a performance evaluation study in which some efficient classifiers are tested in handwritten digit recognition. The evaluated classifiers include a statistical classifier (modified quadratic discriminant function, MQDF), three neural classifiers, and an LVQ (learning vector quantization) classifier. They are efficient in that high accuracies can be achieved at moderate memory space and computation cost. The performance is measured in terms of classification accuracy, sensitivity to training sample size, ambiguity rejection, and outlier resistance. The outlier resistance of neural classifiers is enhanced by training with synthesized outlier data. The classifiers are tested on a large data set extracted from NIST SD19. As results, the test accuracies of the evaluated classifiers are comparable to or higher than those of the nearest neighbor (1-NN) rule and regularized discriminant analysis (RDA). It is shown that neural classifiers are more susceptible to small sample size than MQDF, although they yield higher accuracies on large sample size. As a neural classifier, the polynomial classifier (PC) gives the highest accuracy and performs best in ambiguity rejection. On the other hand, MQDF is superior in outlier rejection even though it is not trained with outlier data. The results indicate that pattern classifiers have complementary advantages and they should be appropriately combined to achieve higher performance. Received: July 18, 2001 / Accepted: September 28, 2001     

(2D)LDA: An efficient approach for face recognition

Although 2DLDA algorithm obtains higher recognition accuracy, a vital unresolved problem of 2DLDA is that it needs huge feature matrix for the task of face recognition. To overcome this problem, this paper presents an efficient approach for face image feature extraction, namely, (2D)²LDA method. Experimental results on ORL and Yale database show that the proposed method obtains good recognition accuracy despite having less number of coefficients.     

Face recognition with one training image per person

At present there are many methods that could deal well with frontal view face recognition. However, most of them cannot work well when there is only one training image per person. In this paper, an extension of the eigenface technique, i.e. projection-combined principal component analysis, (PC)²A, is proposed. (PC)²A combines the original face image with its horizontal and vertical projections and then performs principal component analysis on the enriched version of the image. It requires less computational cost than the standard eigenface technique and experimental results show that on a gray-level frontal view face database where each person has only one training image, (PC)²A achieves 3–5% higher accuracy than the standard eigenface technique through using 10–15% fewer eigenfaces.     

基于SOC网络的商业银行信用风险识别模型

针对自组织竞争(SOC)神经网络在解决模式分类问题上的优势,结合主成分分析法来构建商业银行信用风险识别模型.首先构造一套用于描述贷款企业信用状况的指标体系,然后使用主成分分析法提取特征指标,再采用SOC神经网络进行非监督分类.通过选取陕西省2007年度在沪、深两市交易的26家上市公司作为样本进行实证分析,实证结果表明:模型对信用风险具有较强的识别能力,同时对商业银行还有较好的预测功能.     

Maximum mutual information training for an online neural predictive handwritten word recognition system

In this paper, we present a hybrid online handwriting recognition system based on hidden Markov models (HMMs). It is devoted to word recognition using large vocabularies. An adaptive segmentation of words into letters is integrated with recognition, and is at the heart of the training phase. A word-model is a left-right HMM in which each state is a predictive multilayer perceptron that performs local regression on the drawing (i.e., the written word) relying on a context of observations. A discriminative training paradigm related to maximum mutual information is used, and its potential is shown on a database of 9,781 words. Received June 19, 2000 / Revised October 16, 2000     

A modular clutter rejection technique for FLIR imagery using region-based principal component analysis

A modular clutter-rejection technique that uses region-based principal component analysis (PCA) is proposed. A major problem in FLIR ATR is the poorly centered targets generated by the preprocessing stage. Our modular clutter-rejection system usesstatic as well as dynamic region of interest (ROI) extraction to overcome the problem of poorly centered targets. In static ROI extraction, the center of the representative ROI coincides with the center of the potential target image. In dynamic ROI extraction, a representative ROI is moved in several directions with respect to the center of the potential target image to extract a number of ROIs. Each module in the proposed system applies region-based PCA to generate the feature vectors, which are subsequently used to make a decision about the identity of the potential target. Region-based PCA uses topological features of the targets to reject false alarms. In this technique, a potential target is divided into several regions and a PCA is performed on each region to extract regional feature vectors. We propose using regional feature vectors of arbitrary shapes and dimensions that are optimized for the topology of a target in a particular region. These regional feature vectors are then used by a two-class classifier based on the learning vector quantization to decide whether a potential target is a false alarm or a real target. We also present experimental results using real-life data to evaluate and compare the performance of the clutter-rejection systems with static and dynamic ROI extraction.     

Eigenspace updating for non-stationary process and its application to face recognition

In this paper, we introduce a novel approach to modeling non-stationary random processes. Given a set of training samples sequentially, we can iteratively update the eigenspace to manifest the current statistics provided by each new sample. The updated eigenspace is derived based more on recent samples and less on older samples, controlled by a number of decay parameters. Extensive study has been performed on how to choose these decay parameters. Other existing eigenspace updating algorithms can be regarded as special cases of our algorithm. We show the effectiveness of the proposed algorithm with both synthetic data and practical applications on face recognition. Significant improvements have been observed on face images with different variations, such as pose, expression and illumination variations. We expect the proposed algorithm to have other applications in active recognition and modeling as well.     

Neural network analysis of ophthalmic arterial doppler signals with Uveitis disease

Doppler ultrasound is known as a reliable technique, which demonstrates the flow characteristics and resistance of ophthalmic arteries. In this study, ophthalmic arterial Doppler signals were obtained from 95 subjects—that 45 of them had suffered from Uveitis disease and the rest of them had been healthy subjects. Multilayer perceptron neural network (MLPNN) employing quick propagation training algorithm was used to detect the presence of Uveitis disease. Spectral analysis of ophthalmic arterial Doppler signals was performed by autoregressive moving average (ARMA) method for determining the MLPNN inputs. The MLPNN was trained with training set, cross validated with cross validation set and tested with testing set. All these data sets were obtained from ophthalmic arteries of healthy subjects and subjects suffering from Uveitis disease. Performance indicators and statistical measures were used for evaluating the MLPNN. The correct classification rate was 95.83% for healthy subjects and 91.30% for subjects suffering from Uveitis disease. Based on the accuracy of the MLPNN detections, it can be mentioned that the classification of ophthalmic arterial Doppler signals with Uveitis disease is feasible by the MLPNN employing quick propagation training algorithm.     

Diagonal principal component analysis for face recognition

In this paper, a novel subspace method called diagonal principal component analysis (DiaPCA) is proposed for face recognition. In contrast to standard PCA, DiaPCA directly seeks the optimal projective vectors from diagonal face images without image-to-vector transformation. While in contrast to 2DPCA, DiaPCA reserves the correlations between variations of rows and those of columns of images. Experiments show that DiaPCA is much more accurate than both PCA and 2DPCA. Furthermore, it is shown that the accuracy can be further improved by combining DiaPCA with 2DPCA.     

Face recognition using the embedded HMM with second-order block-specific observations

The paper is concerned with face recognition using the embedded hidden Markov model (EHMM) with second-order block-specific observations. The proposed method partitions a face image into a 2-D lattice type, composed of many blocks. Each block is represented by the second-order block-specific observation that consists of a combination of first- and second-order feature vectors. The first-order (or second-order) feature vector is obtained by projecting the original (or residual) block image onto the first (or second) basis vector that is obtained block-specifically by applying the PCA to a set of original (or residual) block images. A sequence of feature vectors obtained from the top-to-bottom and the left-to-right scanned blocks are used as an observation sequence to train EHMM. The EHMM models the face image in a hierarchical manner as follows. Several super states are used to model the vertical facial features such as the forehead, eyes, nose, mouth, and chin, and several states in the super state are used to model the localized features in a vertical face feature. Recognition is performed by identifying the person of the model that provides the highest value of observation probability. Experimental results show that the proposed recognition method outperforms many existing methods, such as the second-order eigenface method, the EHMM with DCT observations, and the second-order eigenface method using a confidence factor in terms of average of the normalized modified retrieval rank and false identification rate.     

A globally convergent learning algorithm for PCA neural networks

Principal component analysis (PCA) by neural networks is one of the most frequently used feature extracting methods. To process huge data sets, many learning algorithms based on neural networks for PCA have been proposed. However, traditional algorithms are not globally convergent. In this paper, a new PCA learning algorithm based on cascade recursive least square (CRLS) neural network is proposed. This algorithm can guarantee the network weight vector converges to an eigenvector associated with the largest eigenvalue of the input covariance matrix globally. A rigorous mathematical proof is given. Simulation results show the effectiveness of the algorithm.     

Image covariance-based subspace method for face recognition

This paper proposes a new subspace method that is based on image covariance obtained from windowed features of images. A windowed input feature consists of a number of pixels, and the dimension of input space is determined by the number of windowed features. Each element of an image covariance matrix can be obtained from the inner product of two windowed features. The 2D-PCA and 2D-LDA methods are then obtained from principal component analysis and linear discriminant analysis, respectively, using the image covariance matrix. In the case of 2D-LDA, there is no need for PCA preprocessing and the dimension of subspace can be greater than the number of classes because the within-class and between-class image covariance matrices have full ranks. Comparative experiments are performed using the FERET, CMU, and ORL databases of facial images. The experimental results show that the proposed 2D-LDA provides the best recognition rate among several subspace methods in all of the tests.     

A novel hybrid approach based on sub-pattern technique and whitened PCA for face recognition

Recently, in a task of face recognition, some researchers presented that independent component analysis (ICA) Architecture I involves a vertically centered principal component analysis (PCA) process (PCA I) and ICA Architecture II involves a whitened horizontally centered PCA process (PCA II). They also concluded that the performance of ICA strongly depends on its involved PCA process. This means that the computationally expensive ICA projection is unnecessary for further process and involved PCA process of ICA, whether PCA I or II, can be used directly for face recognition. But these approaches only consider the global information of face images. Some local information may be ignored. Therefore, in this paper, the sub-pattern technique was combined with PCA I and PCA II, respectively, for face recognition. In other words, two new different sub-pattern based whitened PCA approaches (which are called Sp-PCA I and Sp-PCA II, respectively) were performed and compared with PCA I, PCA II, PCA, and sub-pattern based PCA (SpPCA). Then, we find that sub-pattern technique is useful to PCA I but not to PCA II and PCA. Simultaneously, we also discussed what causes this result in this paper. At last, by simultaneously considering global and local information of face images, we developed a novel hybrid approach which combines PCA II and Sp-PCA I for face recognition. The experimental results reveal that the proposed novel hybrid approach has better recognition performance than that obtained using other traditional methods.     

Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters

This paper describes an adaptive recognition system for isolated handwritten characters and the experiments carried out with it. The characters used in our experiments are alphanumeric characters, including both the upper- and lower-case versions of the Latin alphabets and three Scandinavian diacriticals. The writers are allowed to use their own natural style of writing. The recognition system is based on the k-nearest neighbor rule. The six character similarity measures applied by the system are all based on dynamic time warping. The aim of the first experiments is to choose the best combination of the simple preprocessing and normalization operations and the dissimilarity measure for a multi-writer system. However, the main focus of the work is on online adaptation. The purpose of the adaptations is to turn a writer-independent system into writer-dependent and increase recognition performance. The adaptation is carried out by modifying the prototype set of the classifier according to its recognition performance and the user's writing style. The ways of adaptation include: (1) adding new prototypes; (2) inactivating confusing prototypes; and (3) reshaping existing prototypes. The reshaping algorithm is based on the Learning Vector Quantization. Four different adaptation strategies, according to which the modifications of the prototype set are performed, have been studied both offline and online. Adaptation is carried out in a self-supervised fashion during normal use and thus remains unnoticed by the user. Received June 30, 1999 / Revised September 29, 2000     

Complete neighborhood preserving embedding for face recognition

Neighborhood preserving embedding (NPE) is a linear approximation to the locally linear embedding algorithm which can preserve the local neighborhood structure on the data manifold. However, in typical face recognition where the number of data samples is smaller than the dimension of data space, it is difficult to directly apply NPE to high dimensional matrices because of computational complexity. Moreover, in such case, NPE often suffers from the singularity problem of eigenmatrix, which makes the direct implementation of the NPE algorithm almost impossible. In practice, principal component analysis or singular value decomposition is applied as a preprocessing step to attack these problems. Nevertheless, this strategy may discard dimensions that contain important discriminative information and the eigensystem computation of NPE could be unstable. Towards a practical dimensionality reduction method for face data, we develop a new scheme in this paper, namely, the complete neighborhood preserving embedding (CNPE). CNPE transforms the singular generalized eigensystem computation of NPE into two eigenvalue decomposition problems. Moreover, a feasible and effective procedure is proposed to alleviate the computational burden of high dimensional matrix for typical face image data. Experimental results on the ORL face database and the Yale face database show that the proposed CNPE algorithm achieves better performance than other feature extraction methods, such as Eigenfaces, Fisherfaces and NPE, etc.