一个联机识别自然手写汉字的多分类器集成系统

本文提出一种联机识别自然手写体汉字的多分类器集成模型。该模型中,我们把依照01、WB和SO特征码设计的不同分类器进行集成,综合模式多种全局和局部特征,从汉字的多个结构层进行识别。初步实验结果为,识别率98.6%。     

手写体汉字识别纯神经网络多分类器集成

多分类器集成是解决手写体汉字识别性能的重要方法之一,近年来受到了学术届的普遍关注。文章提出了一种基于单字单网的手写体汉字识别纯神经网络的多分类器集成方案,并通过实验证明该方案是行之有效的。     

基于笔划特征的单字符汉字字体识别

在文档电子化的文本自动分析、理解和识别过程中,除了有关文档内容的字符识别外,还必须解决字体识别问题.字体识别不仅是版面分析、理解和恢复的重要依据,还有助于实现高性能字符识别系统.有别于目前基于多个字符组成的文本块的字体识别方法,本文提出了一种基于单个汉字字符的字体识别方法.在单个汉字字符上提取两类特征:笔划属性特征和笔划分布特征,分别构成两个分类器对单个汉字字符进行字体识别,并集成两个分类器的结果得到最终的识别结果.我们使用的笔划属性特征分类器是文本无关的,而笔划分布特征分类器是文本相关的,集成的分类器属于文本相关的字体识别分类器.我们在包含7种字体的样本集上进行了测试,测试结果显示基于单字的字体识别率达到94.48%.     

一种用于手写体汉字识别的候选字加权多分类器集成方法

提出了一种基于候选字加权的多分类器集成方法,并将其应用于手写体汉字的识别研究中。利用４种不同的特征提取方法构造了４个独立的分类器;利用不同分类器各候选字加权处理得到的置信度函数来构造集成函数,从而将４个独立的分类器集成为一个多分类器系统。通过实验分析了几种分类器集成的方法,验证了具有一定互补性的多分类器集成对手写体汉字的识别率有较大的提高。实验结果表明、所提出的方法是行之有效的。     

一种用于手写体汉字识别的侯选字加权多分类器集成方法

提出了一种基于候选字加权的多分类器集成方法,并将其应用于手写体汉字的识别研究中。利用4种不同的特征提取方法构造了4个独立的分类器,利用不同分类器各候选字加权处理得到的置信度函数来构成集成函数,从而将4个独立的分类器集成为一个多分类器系统。通过实验分析了几种分类器集成的方法,验证了具有一定互补性的多分类器集成对手写体汉字的识别率有较大的提高。实验结果表明,所提出的方法是行之有效的。     

脱机手写汉字识别研究

脱机手写体汉字识别是当前OCR技术研究的热点之一.本文提出了一种用于手写体汉字识别的多特征多分类器集成的系统模型,并利用Matlab工具箱对50个汉字5000个样本进行了初步仿真实验,实验表明该模型是十分可行和有效的.     

一种手写体汉字识别的神经网络多分类器集成方案

提出了一种基于单字单网的手写体汉字识别纯神经网络的多分类器集成方案,并通过实验证明用该方案实现的神经网络集成系统性能均比任一个神经网络单分类器都好,对1 000种不同的手写体汉字的1 000×10个字进行测试,集成后的识别率最高达到95.22%,比单分类器的识别率高出5.0%-8.7%。     

一种识别手写汉字的多分类器集成方法

根据多信源信息处理与字符识别的经验知识,提出了一个识别手写汉字的多分类器 线性集成模型.这个模型不仅考虑到不同的分类器对不同字符识别能力的不同,而且还考虑 了不同的分类器得出的输入字符与参考模板之间相似度的实际大小对判决的影响,及不同分 类器提供的候选字符对判决的支持作用,更重要的是提供了一种通过监督学习,利用计算机 程序自动计算模型参数的方法,因而实现了一个较好的集成系统.同时,本文还提供了三个用 于集成的分类器,它们集成的结果充分显示了本方法的有效性.     

基于置信度的多分类器互补集成手写数字识别

离线手写数字识别是光学字符识别的一个重要分支,在银行票据识别、邮政编码识别等领域有着广泛的应用。由于单一分类器在识别率上很难达到要求,人们提出了各种集成分类器识别方案。通过对离线手写数字的特征提取,从特征互补的角度出发,采用了最小距离分类器、树分类器和BP网络分类器进行多分类器互补集成,提出了基于置信度的多分类器互补集成方法。通过实验对比,基于置信度的多分类器互补集成手写数字识别在识别率和识别速度上达到了满意的结果。     

一种简单有效的多分类器综合方法

针对小字符集脱机手写体汉字识别中的多分类器集成问题，提出了一种简单有效的综合方法，实验表明综合后系统的识别率明显高于单个分类器的识别率。     

Farsi/Arabic handwritten digit recognition based on ensemble of SVD classifiers and reliable multi-phase PSO combination rule

The problem of handwritten digit recognition has long been an open problem in the field of pattern classification and of great importance in industry. The heart of the problem lies within the ability to design an efficient algorithm that can recognize digits written and submitted by users via a tablet, scanner, and other digital devices. From an engineering point of view, it is desirable to achieve a good performance within limited resources. To this end, we have developed a new approach for handwritten digit recognition that uses a small number of patterns for training phase. To improve the overall performance achieved in classification task, the literature suggests combining the decision of multiple classifiers rather than using the output of the best classifier in the ensemble; so, in this new approach, an ensemble of classifiers is used for the recognition of handwritten digit. The classifiers used in proposed system are based on singular value decomposition (SVD) algorithm. The experimental results and the literature show that the SVD algorithm is suitable for solving sparse matrices such as handwritten digit. The decisions obtained by SVD classifiers are combined by a novel proposed combination rule which we named reliable multi-phase particle swarm optimization. We call the method “Reliable” because we have introduced a novel reliability parameter which is applied to tackle the problem of PSO being trapped in local minima. In comparison with previous methods, one of the significant advantages of the proposed method is that it is not sensitive to the size of training set. Unlike other methods, the proposed method uses just 15 % of the dataset as a training set, while other methods usually use (60–75) % of the whole dataset as the training set. To evaluate the proposed method, we tested our algorithm on Farsi/Arabic handwritten digit dataset. What makes the recognition of the handwritten Farsi/Arabic digits more challenging is that some of the digits can be legally written in different shapes. Therefore, 6000 hard samples (600 samples per class) are chosen by K-nearest neighbor algorithm from the HODA dataset which is a standard Farsi/Arabic digit dataset. Experimental results have shown that the proposed method is fast, accurate, and robust against the local minima of PSO. Finally, the proposed method is compared with state of the art methods and some ensemble classifier based on MLP, RBF, and ANFIS with various combination rules.     

Feature selection for ensembles applied to handwriting recognition

Feature selection for ensembles has shown to be an effective strategy for ensemble creation due to its ability of producing good subsets of features, which make the classifiers of the ensemble disagree on difficult cases. In this paper we present an ensemble feature selection approach based on a hierarchical multi-objective genetic algorithm. The underpinning paradigm is the “overproduce and choose”. The algorithm operates in two levels. Firstly, it performs feature selection in order to generate a set of classifiers and then it chooses the best team of classifiers. In order to show its robustness, the method is evaluated in two different contexts:supervised and unsupervised feature selection. In the former, we have considered the problem of handwritten digit recognition and used three different feature sets and multi-layer perceptron neural networks as classifiers. In the latter, we took into account the problem of handwritten month word recognition and used three different feature sets and hidden Markov models as classifiers. Experiments and comparisons with classical methods, such as Bagging and Boosting, demonstrated that the proposed methodology brings compelling improvements when classifiers have to work with very low error rates. Comparisons have been done by considering the recognition rates only.     

A novel cascade ensemble classifier system with a high recognition performance on handwritten digits

This paper presents a novel cascade ensemble classifier system for the recognition of handwritten digits. This new system aims at attaining a very high recognition rate and a very high reliability at the same time, in other words, achieving an excellent recognition performance of handwritten digits. The trade-offs among recognition, error, and rejection rates of the new recognition system are analyzed. Three solutions are proposed: (i) extracting more discriminative features to attain a high recognition rate, (ii) using ensemble classifiers to suppress the error rate and (iii) employing a novel cascade system to enhance the recognition rate and to reduce the rejection rate. Based on these strategies, seven sets of discriminative features and three sets of random hybrid features are extracted and used in the different layers of the cascade recognition system. The novel gating networks (GNs) are used to congregate the confidence values of three parallel artificial neural networks (ANNs) classifiers. The weights of the GNs are trained by the genetic algorithms (GAs) to achieve the overall optimal performance. Experiments conducted on the MNIST handwritten numeral database are shown with encouraging results: a high reliability of 99.96% with minimal rejection, or a 99.59% correct recognition rate without rejection in the last cascade layer.     

Dynamically building diversified classifier pruning ensembles via canonical correlation analysis

Empirical studies on ensemble learning that combines multiple classifiers have shown that, it is an effective technique to improve accuracy and stability of a single classifier. In this paper, we propose a novel method of dynamically building diversified sparse ensembles. We first apply a technique known as the canonical correlation to model the relationship between the input data variables and output base classifiers. The canonical (projected) output classifiers and input training data variables are encoded globally through a multi-linear projection of CCA, to decrease the impacts of noisy input data and incorrect classifiers to a minimum degree in such a global view. Secondly, based on the projection, a sparse regression method is used to prune representative classifiers by combining classifier diversity measurement. Based on the above methods, we evaluate the proposed approach by several datasets, such as UCI and handwritten digit recognition. Experimental results of the study show that, the proposed approach achieves better accuracy as compared to other ensemble methods such as QFWEC, Simple Vote Rule, Random Forest, Drep and Adaboost.

     

Architectures for detecting and solving conflicts: two-stage classification and support vector classifiers

In the majority of cases, a properly trained classifier or ensemble of classifiers may yield acceptable recognition results. However, in some cases, recognition will fail due to typical conflicts that are encountered, like the confusion between [A] and [H] or [U] and [V]. In this paper, two architectures for the recognition of handwritten text are described. The key issue for each of these systems is to detect the event of a possible conflict and subsequently attempt to solve that particular problem. Both systems exploit a two-stage classification method. In the event that the first-stage classifiers are not certain about the result, the second-stage system engages a set of support vector classifiers for refining the output hypothesis.Received: 31 October 2001, Accepted: 13 December 2002, Published online: 6 June 2003     

Troika - An improved stacking schema for classification tasks

Stacking is a general ensemble method in which a number of base classifiers are combined using one meta-classifier which learns their outputs. Such an approach provides certain advantages: simplicity; performance that is similar to the best classifier; and the capability of combining classifiers induced by different inducers. The disadvantage of stacking is that on multiclass problems, stacking seems to perform worse than other meta-learning approaches. In this paper we present Troika, a new stacking method for improving ensemble classifiers. The new scheme is built from three layers of combining classifiers. The new method was tested on various datasets and the results indicate the superiority of the proposed method to other legacy ensemble schemes, Stacking and StackingC, especially when the classification task consists of more than two classes.     

A novel SVM-kNN-PSO ensemble method for intrusion detection system

In machine learning, a combination of classifiers, known as an ensemble classifier, often outperforms individual ones. While many ensemble approaches exist, it remains, however, a difficult task to find a suitable ensemble configuration for a particular dataset. This paper proposes a novel ensemble construction method that uses PSO generated weights to create ensemble of classifiers with better accuracy for intrusion detection. Local unimodal sampling (LUS) method is used as a meta-optimizer to find better behavioral parameters for PSO. For our empirical study, we took five random subsets from the well-known KDD99 dataset. Ensemble classifiers are created using the new approaches as well as the weighted majority algorithm (WMA) approach. Our experimental results suggest that the new approach can generate ensembles that outperform WMA in terms of classification accuracy.     

Greedy optimization classifiers ensemble based on diversity

Decreasing the individual error and increasing the diversity among classifiers are two crucial factors for improving ensemble performances. Nevertheless, the “kappa-error” diagram shows that enhancing the diversity is at the expense of reducing individual accuracy. Hence, a new method named Matching Pursuit Optimization Ensemble Classifiers (MPOEC) is proposed in this paper in order to balance the diversity and the individual accuracy. MPOEC method adopts a greedy iterative algorithm of matching pursuit to search for an optimal combination of entire classifiers, and eliminates some similar or poor classifiers by giving zero coefficients. In MPOEC approach, the coefficient of every classifier is gained by minimizing the residual between the target function and the linear combination of the basis functions, especially, when the basis functions are similar, their coefficients will be close to zeros in one iteration of the optimization process, which indicates that obtained coefficients of classifiers are based on the diversity among ensemble individuals. Because some classifiers are given zero coefficients, MPOEC approach may be also considered as a selective classifiers ensemble method. Experimental results show that MPOEC improves the performance compared with other methods. Furthermore, the kappa-error diagrams indicate that the diversity is increased by the proposed method compared with standard ensemble strategies and evolutionary ensemble.     

A dynamic overproduce-and-choose strategy for the selection of classifier ensembles

The overproduce-and-choose strategy, which is divided into the overproduction and selection phases, has traditionally focused on finding the most accurate subset of classifiers at the selection phase, and using it to predict the class of all the samples in the test data set. It is therefore, a static classifier ensemble selection strategy. In this paper, we propose a dynamic overproduce-and-choose strategy which combines optimization and dynamic selection in a two-level selection phase to allow the selection of the most confident subset of classifiers to label each test sample individually. The optimization level is intended to generate a population of highly accurate candidate classifier ensembles, while the dynamic selection level applies measures of confidence to reveal the candidate ensemble with the highest degree of confidence in the current decision. Experimental results conducted to compare the proposed method to a static overproduce-and-choose strategy and a classical dynamic classifier selection approach demonstrate that our method outperforms both these selection-based methods, and is also more efficient in terms of performance than combining the decisions of all classifiers in the initial pool.     

Automatic feature generation for handwritten digit recognition

An automatic feature generation method for handwritten digit recognition is described. Two different evaluation measures, orthogonality and information, are used to guide the search for features. The features are used in a backpropagation trained neural network. Classification rates compare favorably with results published in a survey of high-performance handwritten digit recognition systems. This classifier is combined with several other high performance classifiers. Recognition rates of around 98% are obtained using two classifiers on a test set with 1000 digits per class