Robust vision-based features and classification schemes for off-line handwritten digit recognition

We use well-established results in biological vision to construct a model for handwritten digit recognition. We show empirically that the features extracted by our model are linearly separable over a large training set (MNIST). Using only a linear discriminant system on these features, our model is relatively simple yet outperforms other models on the same data set. In particular, the best result is obtained by applying triowise linear support vector machines with soft voting on vision-based features extracted from deslanted images.     

Latent classification models for binary data

One of the simplest, and yet most consistently well-performing set of classifiers is the naïve Bayes models (a special class of Bayesian network models). However, these models rely on the (naïve) assumption that all the attributes used to describe an instance are conditionally independent given the class of that instance. To relax this independence assumption, we have in previous work proposed a family of models, called latent classification models (LCMs). LCMs are defined for continuous domains and generalize the naïve Bayes model by using latent variables to model class-conditional dependencies between the attributes. In addition to providing good classification accuracy, the LCM has several appealing properties, including a relatively small parameter space making it less susceptible to over-fitting. In this paper we take a first step towards generalizing LCMs to hybrid domains, by proposing an LCM for domains with binary attributes. We present algorithms for learning the proposed model, and we describe a variational approximation-based inference procedure. Finally, we empirically compare the accuracy of the proposed model to the accuracy of other classifiers for a number of different domains, including the problem of recognizing symbols in black and white images.     

Arabic handwritten digit recognition

In this paper, we fill a gap in the literature by studying the problem of Arabic handwritten digit recognition. The performances of different classification and feature extraction techniques on recognizing Arabic digits are going to be reported to serve as a benchmark for future work on the problem. The performance of well known classifiers and feature extraction techniques will be reported in addition to a novel feature extraction technique we present in this paper that gives a high accuracy and competes with the state-of-the-art techniques. A total of 54 different classifier/features combinations will be evaluated on Arabic digits in terms of accuracy and classification time. The results are analyzed and the problem of the digit ‘0’ is identified with a proposed method to solve it. Moreover, we propose a strategy to select and design an optimal two-stage system out of our study and, hence, we suggest a fast two-stage classification system for Arabic digits which achieves as high accuracy as the highest classifier/features combination but with much less recognition time.     

Automatic feature generation for handwritten digit recognition

An automatic feature generation method for handwritten digit recognition is described. Two different evaluation measures, orthogonality and information, are used to guide the search for features. The features are used in a backpropagation trained neural network. Classification rates compare favorably with results published in a survey of high-performance handwritten digit recognition systems. This classifier is combined with several other high performance classifiers. Recognition rates of around 98% are obtained using two classifiers on a test set with 1000 digits per class     

Cloud-based efficient scheme for handwritten digit recognition

Multimedia Tools and Applications - Handwritten character recognition has been acknowledged and achieved more prominent attention in pattern recognition research community due to enormous...     

A trainable feature extractor for handwritten digit recognition

This article focuses on the problems of feature extraction and the recognition of handwritten digits. A trainable feature extractor based on the LeNet5 convolutional neural network architecture is introduced to solve the first problem in a black box scheme without prior knowledge on the data. The classification task is performed by support vector machines to enhance the generalization ability of LeNet5. In order to increase the recognition rate, new training samples are generated by affine transformations and elastic distortions. Experiments are performed on the well-known MNIST database to validate the method and the results show that the system can outperform both SVMs and LeNet5 while providing performances comparable to the best performance on this database. Moreover, an analysis of the errors is conducted to discuss possible means of enhancement and their limitations.     

脱机手写数字识别方法

脱机手写体数字识别有着重大的使用价值,特征提取占据了重要的位置.提出了一种通过拓扑特征构造的特征提取新方法,利于了9种特征对数字进行特征提取,然后利用分类树的方法将数字进行分类.最后,在本科学生手写数字图像样本库上的试验结果表明,提出的特征提取方法不仅具有很快的运算能力,而且较大幅度地提高了识别率.     

手写数字识别的研究

为了提高手写数字识别的性能,研究了利用BP神经网络作为分类器在设计上的几个关键问题,给出每个关键环节的可行方案并进行有效总结.同时对脱机手写数字的图像预处理及特征提取部分的关键技术做了详细阐述.在此基础上给出分类器设计与训练的详细实验,实验结果表明,合理解决设计BP神经网络分类器的关键问题能够确保其对手写数字的高分类性能.     

Segmentation and recognition system for unknown-length handwritten digit strings

The segmentation of handwritten digit strings into isolated digits remains a challenging task. The difficulty for recognizing handwritten digit strings is related to several factors such as sloping, overlapping, connecting and unknown length of the digit string. Hence, this paper aims to propose a segmentation and recognition system for unknown-length handwritten digit strings by combining several explicit segmentation methods depending on the configuration link between digits. Three segmentation methods are combined based on histogram of the vertical projection, the contour analysis and the sliding window Radon transform. A recognition and verification module based on support vector machine classifiers allows analyzing and deciding the rejection or acceptance each segmented digit image. Moreover, various submodules are included leading to enhance the robustness of the proposed system. Experimental results conducted on the benchmark dataset show that the proposed system is effective for segmenting handwritten digit strings without prior knowledge of their length comparatively to the state of the art.     

An MLP-SVM combination architecture for offline handwritten digit recognition

This paper presents an original hybrid MLP-SVM method for unconstrained handwritten digits recognition. Specialized Support Vector Machines (SVMs) are introduced to improve significantly the multilayer perceptron (MLP) performance in local areas around the separating surfaces between each pair of digit classes, in the input pattern space. This hybrid architecture is based on the idea that the correct digit class almost systematically belongs to the two maximum MLP outputs and that some pairs of digit classes constitute the majority of MLP substitutions (errors). Specialized local SVMs are introduced to detect the correct class among these two classification hypotheses. The hybrid MLP-SVM recognizer achieves a recognition rate of 98.01%98.01\% , for real mail zipcode digits recognition task. By introducing a rejection mechanism based on the distances provided by the local SVMs, the error/reject trade-off performance of our recognition system is better than several classifiers reported in recent research.     

Data augmentation for handwritten digit recognition using generative adversarial networks

Multimedia Tools and Applications - Supervised learning techniques require labeled examples that can be time consuming to obtain. In particular, deep learning approaches, where all the feature...     

CUDA-enabled implementation of a neural network algorithm for handwritten digit recognition

Using a convolutional neural network as an example, we discuss specific aspects of implementing a learning algorithm of pattern recognition on the GPU graphics card using NVIDIA CUDA architecture. The training time of the neural network on a video-adapter is decreased by a factor of 5.96 and the recognition time of a test set is decreased by a factor of 8.76 when compared with the implementation of an optimized algorithm on a central processing unit (CPU). We show that the implementation of the neural network algorithms on graphics processors holds promise.     

Distance-based classification of handwritten symbols

We study online classification of isolated handwritten symbols using distance measures on spaces of curves. We compare three distance-based measures on a vector space representation of curves to elastic matching and ensembles of SVM. We consider the Euclidean and Manhattan distances and the distance to the convex hull of nearest neighbors. We show experimentally that of all these methods the distance to the convex hull of nearest neighbors yields the best classification accuracy of about 97.5%. Any of the above distance measures can be used to find the nearest neighbors and prune totally irrelevant classes, but the Manhattan distance is preferable for this because it admits a very efficient implementation. We use the first few Legendre-Sobolev coefficients of the coordinate functions to represent the symbol curves in a finite-dimensional vector space and choose the optimal dimension and number of bits per coefficient by cross-validation. We discuss an implementation of the proposed classification scheme that will allow classification of a sample among hundreds of classes in a setting with strict time and storage limitations.     

Fast sequence segmentation using log-linear models

Sequence segmentation is a well-studied problem, where given a sequence of elements, an integer K, and some measure of homogeneity, the task is to split the sequence into K contiguous segments that are maximally homogeneous. A classic approach to find the optimal solution is by using a dynamic program. Unfortunately, the execution time of this program is quadratic with respect to the length of the input sequence. This makes the algorithm slow for a sequence of non-trivial length. In this paper we study segmentations whose measure of goodness is based on log-linear models, a rich family that contains many of the standard distributions. We present a theoretical result allowing us to prune many suboptimal segmentations. Using this result, we modify the standard dynamic program for 1D log-linear models, and by doing so reduce the computational time. We demonstrate empirically, that this approach can significantly reduce the computational burden of finding the optimal segmentation.     

Postprocessing statistical language models for handwritten Chinesecharacter recognizer

Two statistical language models have been investigated on their effectiveness in upgrading the accuracy of a Chinese character recognizer. The baseline model is one of lexical analytic nature which segments a sequence of character images according to the maximum matching of words with consideration of word binding forces. A model of bigram statistics of word-classes is then investigated and compared against the baseline model in terms of recognition rate improvement on the image recognizer. On the average, the baseline language model improves the recognition rate by about 7% while the bigram statistics model upgrades it by about 10%     

A modal learning adaptive function neural network applied to handwritten digit recognition

A novel combination of the adaptive function neural network (ADFUNN) and on-line snap-drift learning is presented in this paper and applied to optical and pen-based recognition of handwritten digits [E. Alpaydin, F. Alimoglu for Optical Recognition of Handwritten Digits and E. Alpaydin, C. Kaynak for Pen-Based Recognition of Handwritten Digits http://www.ics.uci.edu/~mlearn/databases/optdigits/http://www.ics.uci.edu/~mlearn/databases/pendigits/]. Snap-drift [S.W. Lee, D. Palmer-Brown, C.M. Roadknight, Performance-guided neural network for rapidly self-organising active network management (Invited Paper), Journal of Neurocomputing, 61C, 2004, pp. 5-20] employs the complementary concepts of common (intersection) feature learning (called snap) and LVQ (drift towards the input patterns) learning, and is a fast, unsupervised method suitable for on-line learning and non-stationary environments where new patterns are continually introduced. ADFUNN [M. Kang, D. Palmer-Brown, An adaptive function neural network (ADFUNN) for phrase recognition, in: The International Joint Conference on Neural Networks (IJCNN05), Montréal, Canada, 2005, D. Palmer-Brown, M. Kang, ADFUNN: An adaptive function neural network, in: The 7th International Conference on Adaptive and Natural Computing Algorithms (ICANNGA05), Coimbra, Portugal, 2005] is based on a linear piecewise neuron activation function that is modified by a novel gradient descent supervised learning algorithm. It has recently been applied to the Iris dataset, and a natural language phrase recognition problem, exhibiting impressive generalisation classification ability with no hidden neurons. The unsupervised single layer snap-drift is effective in extracting distinct features from the complex cursive-letter datasets, and the supervised single layer ADFUNN is capable of solving linearly inseparable problems rapidly. In combination within one network (SADFUNN), these two methods are more powerful and yet simpler than MLPs, at least on this problem domain. We experiment on SADFUNN with two handwritten digits datasets problems from the UCI Machine Learning repository. The problems are learned rapidly and higher generalisation results are achieved than with a MLP.