手写体数字有效鉴别特征的抽取与识别

文中提出了基于后验概率估计的多特征多分类器组合识别的估计法,并提出了基于具有统计不相关性的最佳鉴别变换与ＫＬ变换抽取手写体数字的有效鉴别特征的方法。实验采用Ｃｏｎｃｏｒｄｉａ Ｕｎｉｖｅｒｓｉｔｙ ＣＥＮＰＡＲＭＩ手写体数字数据库。用最近邻距离分类器与最近邻相关分类器这两个分类器,对手写体数字的１２个特征做多特征多分类器组合识别实验。实验结果表明：估计法优于常用的投票法与计分法,估计法的识别率高达     

独立分量分析在模式识别中的应用

模式识别中关键的两个环节是模式的特征提取及利用分类器分类识别。采用独立分量分析进行特征提取 ,并比较了最近邻分类器和cos分类器的分类识别性能。利用ORL人脸图像数据库进行实验 ,结果表明独立分量分析与cos分类器相结合可得到更好的识别结果。     

Sparse margin–based discriminant analysis for feature extraction

The existing margin-based discriminant analysis methods such as nonparametric discriminant analysis use K-nearest neighbor (K-NN) technique to characterize the margin. The manifold learning–based methods use K-NN technique to characterize the local structure. These methods encounter a common problem, that is, the nearest neighbor parameter K should be chosen in advance. How to choose an optimal K is a theoretically difficult problem. In this paper, we present a new margin characterization method named sparse margin–based discriminant analysis (SMDA) using the sparse representation. SMDA can successfully avoid the difficulty of parameter selection. Sparse representation can be considered as a generalization of K-NN technique. For a test sample, it can adaptively select the training samples that give the most compact representation. We characterize the margin by sparse representation. The proposed method is evaluated by using AR, Extended Yale B database, and the CENPARMI handwritten numeral database. Experimental results show the effectiveness of the proposed method; its performance is better than some other state-of-the-art feature extraction methods.     

Nearest neighbor classification from multiple feature subsets

Combining multiple classifiers is an effective technique for improving accuracy. There are many general combining algorithms, such as Bagging, Boosting, or Error Correcting Output Coding, that significantly improve classifiers like decision trees, rule learners, or neural networks. Unfortunately, these combining methods do not improve the nearest neighbor classifier. In this paper, we present MFS, a combining algorithm designed to improve the accuracy of the nearest neighbor (NN) classifier. MFS combines multiple NN classifiers each using only a random subset of features. The experimental results are encouraging: On 25 datasets from the UCI repository, MFS significantly outperformed several standard NN variants and was competitive with boosted decision trees. In additional experiments, we show that MFS is robust to irrelevant features, and is able to reduce both bias and variance components of error.     

Adaptive Graph Embedding Discriminant Projections

Graph embedding based learning method plays an increasingly significant role on dimensionality reduction (DR). However, the selection to neighbor parameters of graph is intractable. In this paper, we present a novel DR method called adaptive graph embedding discriminant projections (AGEDP). Compared with most existing DR methods based on graph embedding, such as marginal Fisher analysis which usually predefines the intraclass and interclass neighbor parameters, AGEDP applies all the homogeneous samples for constructing the intrinsic graph, and simultaneously selects heterogeneous samples within the neighborhood generated by the farthest homogeneous sample for constructing the penalty graph. Therefore, AGEDP not only greatly enhances the intraclass compactness and interclass separability, but also adaptively performs neighbor parameter selection which considers the fact that local manifold structure of each sample is generally different. Experiments on AR and COIL-20 datasets demonstrate the effectiveness of the proposed method for face recognition and object categorization, and especially under the interference of occlusion, noise and poses, it is superior to other graph embedding based methods with three different classifiers: nearest neighbor classifier, sparse representation classifier and linear regression classifier.     

WRITER IDENTIFICATION OF ARABIC TEXT USING STATISTICAL AND STRUCTURAL FEATURES

This article addresses writer identification of handwritten Arabic text. Several types of structural and statistical features were extracted from Arabic handwriting text. A novel approach was used to extract structural features that build on some of the main characteristics of the Arabic language. Connected component features for Arabic handwritten text as well as gradient distribution features, windowed gradient distribution features, contour chain code distribution features, and windowed contour chain code distribution features were extracted. A nearest neighbor (NN) classifier was used with the Euclidean distance measure. Data reduction algorithms (viz. principal component analysis [PCA], linear discriminant analysis [LDA], multiple discriminant analysis [MDA], multidimensional scaling [MDS], and forward/backward feature selection algorithm) were used. A database of 500 paragraphs handwritten in Arabic by 250 writers was used. The paragraphs used were randomly generated from a large corpus. NN provided the best accuracy in text-independent writer identification with top-1 result of 88.0%, top-5 result of 96.0%, and top-10 result of 98.5% for the first 100 writers. Extending the work to include all 250 writers and with the backward feature selection algorithm (using 54 out of 83 features), the system attained a top-1 result of 75.0%, top-5 result of 91.8%, and top-10 result of 95.4%.     

Performance evaluation of pattern classifiers for handwritten character recognition

This paper describes a performance evaluation study in which some efficient classifiers are tested in handwritten digit recognition. The evaluated classifiers include a statistical classifier (modified quadratic discriminant function, MQDF), three neural classifiers, and an LVQ (learning vector quantization) classifier. They are efficient in that high accuracies can be achieved at moderate memory space and computation cost. The performance is measured in terms of classification accuracy, sensitivity to training sample size, ambiguity rejection, and outlier resistance. The outlier resistance of neural classifiers is enhanced by training with synthesized outlier data. The classifiers are tested on a large data set extracted from NIST SD19. As results, the test accuracies of the evaluated classifiers are comparable to or higher than those of the nearest neighbor (1-NN) rule and regularized discriminant analysis (RDA). It is shown that neural classifiers are more susceptible to small sample size than MQDF, although they yield higher accuracies on large sample size. As a neural classifier, the polynomial classifier (PC) gives the highest accuracy and performs best in ambiguity rejection. On the other hand, MQDF is superior in outlier rejection even though it is not trained with outlier data. The results indicate that pattern classifiers have complementary advantages and they should be appropriately combined to achieve higher performance. Received: July 18, 2001 / Accepted: September 28, 2001     

BoostMap: an embedding method for efficient nearest neighbor retrieval

This paper describes BoostMap, a method for efficient nearest neighbor retrieval under computationally expensive distance measures. Database and query objects are embedded into a vector space, in which distances can be measured efficiently. Each embedding is treated as a classifier that predicts for any three objects X, A, B whether X is closer to A or to B. It is shown that a linear combination of such embeddingbased classifiers naturally corresponds to an embedding and a distance measure. Based on this property, the BoostMap method reduces the problem of embedding construction to the classical boosting problem of combining many weak classifiers into an optimized strong classifier. The classification accuracy of the resulting strong classifier is a direct measure of the amount of nearest neighbor structure preserved by the embedding. An important property of BoostMap is that the embedding optimization criterion is equally valid in both metric and non-metric spaces. Performance is evaluated in databases of hand images, handwritten digits, and time series. In all cases, BoostMap significantly improves retrieval efficiency with small losses in accuracy compared to brute-force search. Moreover, BoostMap significantly outperforms existing nearest neighbor retrieval methods, such as Lipschitz embeddings, FastMap, and VP-trees.     

Face recognition with adaptive local hyperplane algorithm

The paper introduces a novel adaptive local hyperplane (ALH) classifier and it shows its superior performance in the face recognition tasks. Four different feature extraction methods (2DPCA, (2D)²PCA, 2DLDA and (2D)²LDA) have been used in combination with five classifiers (K-nearest neighbor (KNN), support vector machine (SVM), nearest feature line (NFL), nearest neighbor line (NNL) and ALH). All the classifiers and feature extraction methods have been applied to the renown benchmarking face databases—the Cambridge ORL database and the Yale database and the ALH classifier with a LDA based extractor outperforms all the other methods on them. The ALH algorithm on these two databases is very promising but more study on larger databases need yet to be done to show all the advantages of the proposed algorithm.     

Discriminant waveletfaces and nearest feature classifiers for face recognition

Feature extraction, discriminant analysis, and classification rules are three crucial issues for face recognition. We present hybrid approaches to handle three issues together. For feature extraction, we apply the multiresolution wavelet transform to extract the waveletface. We also perform the linear discriminant analysis on waveletfaces to reinforce discriminant power. During classification, the nearest feature plane (NFP) and nearest feature space (NFS) classifiers are explored for robust decisions in presence of wide facial variations. Their relationships to conventional nearest neighbor and nearest feature line classifiers are demonstrated. In the experiments, the discriminant waveletface incorporated with the NFS classifier achieves the best face recognition performance.     

基于多分类器集成的维吾尔文联机手写字母识别

多分类器组合能够在一定程度上弥补单个分类器的缺陷,因此它在模式识别中得到了广泛应用。深入调研国内外联机手写识别技术的研究动态,结合维吾尔文字母的独特书写风格,研究了基于多分类器集成的维吾尔语联机手写字母识别。利用5种不同的特征提取方法构造了5个独立的维吾尔语字母分类识别器,采用了等权投票和不等权投票等两种策略将5种维吾尔语字母分类识别器进行了有效组合。其中,单分类器采用了基于动态时间弯折（DTW）匹配距离的最近邻分类方法。实验结果表明,提出的集成策略的识别率明显高于单分类器的识别率,而且为特征的综合集成提供了多种有效途径。     

一种基于预分类的高效最近邻分类器算法

本文的最近邻分类器算法是采用多分类器组合的方式对测试样本进行预分类，并根据预分类结果重新生成新的训练和测试样本集。对新的测试样本采用最近邻分类器进行分类识别，并将识别结果与预分类结果结合在一起进行正确率测试。在ORL人脸库上的实验结果说明，该算法对小样本数据的识别具有明显优势。     

Neural and statistical classifiers-taxonomy and two case studies

Pattern classification using neural networks and statistical methods is discussed. We give a tutorial overview in which popular classifiers are grouped into distinct categories according to their underlying mathematical principles; also, we assess what makes a classifier neural. The overview is complemented by two case studies using handwritten digit and phoneme data that test the performance of a number of most typical neural-network and statistical classifiers. Four methods of our own are included: reduced kernel discriminant analysis, the learning k-nearest neighbors classifier, the averaged learning subspace method, and a version of kernel discriminant analysis.     

Handwritten digit recognition: benchmarking of state-of-the-art techniques

This paper presents the results of handwritten digit recognition on well-known image databases using state-of-the-art feature extraction and classification techniques. The tested databases are CENPARMI, CEDAR, and MNIST. On the test data set of each database, 80 recognition accuracies are given by combining eight classifiers with ten feature vectors. The features include chaincode feature, gradient feature, profile structure feature, and peripheral direction contributivity. The gradient feature is extracted from either binary image or gray-scale image. The classifiers include the k-nearest neighbor classifier, three neural classifiers, a learning vector quantization classifier, a discriminative learning quadratic discriminant function (DLQDF) classifier, and two support vector classifiers (SVCs). All the classifiers and feature vectors give high recognition accuracies. Relatively, the chaincode feature and the gradient feature show advantage over other features, and the profile structure feature shows efficiency as a complementary feature. The SVC with RBF kernel (SVC-rbf) gives the highest accuracy in most cases but is extremely expensive in storage and computation. Among the non-SV classifiers, the polynomial classifier and DLQDF give the highest accuracies. The results of non-SV classifiers are competitive to the best ones previously reported on the same databases.     

The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier

We present the Nearest Subclass Classifier (NSC), which is a classification algorithm that unifies the flexibility of the nearest neighbor classifier with the robustness of the nearest mean classifier. The algorithm is based on the Maximum Variance Cluster algorithm and, as such, it belongs to the class of prototype-based classifiers. The variance constraint parameter of the cluster algorithm serves to regularize the classifier, that is, to prevent overfitting. With a low variance constraint value, the classifier turns into the nearest neighbor classifier and, with a high variance parameter, it becomes the nearest mean classifier with the respective properties. In other words, the number of prototypes ranges from the whole training set to only one per class. In the experiments, we compared the NSC with regard to its performance and data set compression ratio to several other prototype-based methods. On several data sets, the NSC performed similarly to the k-nearest neighbor classifier, which is a well-established classifier in many domains. Also concerning storage requirements and classification speed, the NSC has favorable properties, so it gives a good compromise between classification performance and efficiency.     

一种融合语义距离的最近邻图像标注方法

传统的基于最近邻的图像标注方法效果不佳,主要原因在于提取图像视觉特征时,损失了很多有价值的信息.提出了一种改进的最近邻分类模型.首先利用距离测度学习方法,引入图像的语义类别信息进行训练,生成新的语义距离;然后利用该距离对每一类图像进行聚类,生成多个类内的聚类中心;最后通过计算图像到各个聚类中心的语义距离来构建最近邻分类模型.在构建最近邻分类模型的整个过程中,都使用训练得到的语义距离来计算,这可以有效减少相同图像类内的变动和不同图像类之间的相似所造成的语义鸿沟.在ImageCLEF2012图像标注数据库上进行了实验,将本方法与传统分类模型和最新的方法进行了比较,验证了本方法的有效性.     

Overlap pattern synthesis with an efficient nearest neighbor classifier

Nearest neighbor (NN) classifier is the most popular non-parametric classifier. It is a simple classifier with no design phase and shows good performance. Important factors affecting the efficiency and performance of NN classifier are (i) memory required to store the training set, (ii) classification time required to search the nearest neighbor of a given test pattern, and (iii) due to the curse of dimensionality the number of training patterns needed by it to achieve a given classification accuracy becomes prohibitively large when the dimensionality of the data is high. In this paper, we propose novel techniques to improve the performance of NN classifier and at the same time to reduce its computational burden. These techniques are broadly based on: (i) overlap based pattern synthesis which can generate a larger number of artificial patterns than the number of input patterns and thus can reduce the curse of dimensionality effect, (ii) a compact representation of the given set of training patterns called overlap pattern graph (OLP-graph) which can be incrementally built by scanning the training set only once and (iii) an efficient NN classifier called OLP-NNC which directly works with OLP-graph and does implicit overlap based pattern synthesis. A comparison based on experimental results is given between some of the relevant classifiers. The proposed schemes are suitable for applications dealing with large and high dimensional datasets like those in data mining.     

基于数据挖掘的组合近邻模型算法

针对数据挖掘的组合模型问题，研究了组合模型的理论和技术，分析了组合理论在近邻法的应用现状，提出了一种通过随机属性子集组合近邻分类器的算法MNN，利用简单的投票方法，通过一个随机的属性子集来组合多重近邻分类器，对多重NN分类器的输出进行组合，MNN方法能有效地改进近邻法的分类精度。MNN方法与NN-E000相比，有两个主要的优点：(1) MNN是一个更简单的方法；(2) MNN不受多类问题的限制。     

A combination of CSP-based method with soft margin SVM classifier and generalized RBF kernel for imagery-based brain computer interface applications

Several methods utilizing common spatial pattern (CSP) algorithm have been presented for improving the identification of imagery movement patterns for brain computer interface applications. The present study focuses on improving a CSP-based algorithm for detecting the motor imagery movement patterns. A discriminative filter bank of CSP method using a discriminative sensitive learning vector quantization (DFBCSP-DSLVQ) system is implemented. Four algorithms are then combined to form three methods for improving the efficiency of the DFBCSP-DSLVQ method, namely the kernel linear discriminant analysis (KLDA), the kernel principal component analysis (KPCA), the soft margin support vector machine (SSVM) classifier and the generalized radial bases functions (GRBF) kernel. The GRBF is used as a kernel for the KLDA, the KPCA feature selection algorithms and the SSVM classifier. In addition, three types of classifiers, namely K-nearest neighbor (K-NN), neural network (NN) and traditional support vector machine (SVM), are employed to evaluate the efficiency of the classifiers. Results show that the best algorithm is the combination of the DFBCSP-DSLVQ method using the SSVM classifier with GRBF kernel (SSVM-GRBF), in which the best average accuracy, attained are 92.70% and 83.21%, respectively. Results of the Repeated Measures ANOVA shows the statistically significant dominance of this method at p <?0.05. The presented algorithms are then compared with the base algorithm of this study i.e. the DFBCSP-DSLVQ with the SVM-RBF classifier. It is concluded that the algorithms, which are based on the SSVM-GRBF classifier and the KLDA with the SSVM-GRBF classifiers give sufficient accuracy and reliable results.

     

A new k-harmonic nearest neighbor classifier based on the multi-local means

The k-nearest neighbor (KNN) rule is a classical and yet very effective nonparametric technique in pattern classification, but its classification performance severely relies on the outliers. The local mean-based k-nearest neighbor classifier (LMKNN) was firstly introduced to achieve robustness against outliers by computing the local mean vector of k nearest neighbors for each class. However, its performances suffer from the choice of the single value of k for each class and the uniform value of k for different classes. In this paper, we propose a new KNN-based classifier, called multi-local means-based k-harmonic nearest neighbor (MLM-KHNN) rule. In our method, the k nearest neighbors in each class are first found, and then used to compute k different local mean vectors, which are employed to compute their harmonic mean distance to the query sample. Finally, MLM-KHNN proceeds in classifying the query sample to the class with the minimum harmonic mean distance. The experimental results, based on twenty real-world datasets from UCI and KEEL repository, demonstrated that the proposed MLM-KHNN classifier achieves lower classification error rate and is less sensitive to the parameter k, when compared to nine related competitive KNN-based classifiers, especially in small training sample size situations.