An architecture for handwritten text recognition systems

This paper presents an end-to-end system for reading handwritten page images. Five functional modules included in the system are introduced in this paper: (i) pre-processing, which concerns introducing an image representation for easy manipulation of large page images and image handling procedures using the image representation; (ii) line separation, concerning text line detection and extracting images of lines of text from a page image; (iii) word segmentation, which concerns locating word gaps and isolating words from a line of text image obtained efficiently and in an intelligent manner; (iv) word recognition, concerning handwritten word recognition algorithms; and (v) linguistic post-pro- cessing, which concerns the use of linguistic constraints to intelligently parse and recognize text. Key ideas employed in each functional module, which have been developed for dealing with the diversity of handwriting in its various aspects with a goal of system reliability and robustness, are described in this paper. Preliminary experiments show promising results in terms of speed and accuracy. Received October 30, 1998 / Revised January 15, 1999     

Rejection strategies for offline handwritten text line recognition

This paper investigates rejection strategies for unconstrained offline handwritten text line recognition. The rejection strategies depend on various confidence measures that are based on alternative word sequences. The alternative word sequences are derived from specific integration of a statistical language model in the hidden Markov model based recognition system. Extensive experiments on the IAM database validate the proposed schemes and show that the novel confidence measures clearly outperform two baseline systems which use normalised likelihoods and local n-best lists, respectively.     

A Bayesian-based method of unconstrained handwritten offline Chinese text line recognition

This paper presents a new Bayesian-based method of unconstrained handwritten offline Chinese text line recognition. In this method, a sample of a real character or non-character in realistic handwritten text lines is jointly recognized by a traditional isolated character recognizer and a character verifier, which requires just a moderate number of handwritten text lines for training. To improve its ability to distinguish between real characters and non-characters, the isolated character recognizer is negatively trained using a linear discriminant analysis (LDA)-based strategy, which employs the outputs of a traditional MQDF classifier and the LDA transform to re-compute the posterior probability of isolated character recognition. In tests with 383 text lines in HIT-MW database, the proposed method achieved the character-level recognition rates of 71.37% without any language model, and 80.15% with a bi-gram language model, respectively. These promising results have shown the effectiveness of the proposed method for unconstrained handwritten offline Chinese text line recognition.     

Biometric recognition using online uppercase handwritten text

This paper presents a handwritten text biometric recognition system suitable to be applied to short sequences of text (words). Strokes are considered the structural units of handwriting with words being regarded as two separate sequences: one of pen-down and one of pen-up strokes. Unsupervised categorization by means of a self-organized map allows mapping strokes to integers and the efficient comparison of the sequences by means of dynamic time warping. Measures obtained from each sequence are combined in a later step. This separation gives us the opportunity to show that pen-up strokes possess a surprisingly high discriminative power, while the performance of the combination suggests they may carry non-redundant information with respect to pen-down strokes.A writer identification rate of 92.38% and a minimum of detection cost function of 0.046 (4.6%) is achieved with 370 users and just one word. Results are improved up to 96.46% and 0.033 (3.3%) when combining two words.     

A robust model for on-line handwritten japanese text recognition

This paper describes a robust context integration model for on-line handwritten Japanese text recognition. Based on string class probability approximation, the proposed method evaluates the likelihood of candidate segmentation–recognition paths by combining the scores of character recognition, unary and binary geometric features, as well as linguistic context. The path evaluation criterion can flexibly combine the scores of various contexts and is insensitive to the variability in path length, and so, the optimal segmentation path with its string class can be effectively found by Viterbi search. Moreover, the model parameters are estimated by the genetic algorithm so as to optimize the holistic string recognition performance. In experiments on horizontal text lines extracted from the TUAT Kondate database, the proposed method achieves the segmentation rate of 0.9934 that corresponds to a f-measure and the character recognition rate of 92.80%.     

Unsupervised writer adaptation applied to handwritten text recognition

This paper deals with the problem of off-line handwritten text recognition. It presents a system of text recognition that exploits an original principle of adaptation to the handwriting to be recognized. The adaptation principle is based on the automatic learning, during the recognition, of the graphical characteristics of the handwriting. This on-line adaptation of the recognition system relies on the iteration of two steps: a word recognition step that allows to label the writer's representations (allographs) on the whole text and a re-evaluation step of character models. Tests carried out on a sample of 15 writers, all unknown by the system, show the interest of the proposed adaptation scheme since we obtain during iterations an improvement of recognition rates both at the letter and the word levels.     

小波包变换在手写体金融汉字识别中的应用

针对小波包变换的特点,提出了一种基于小波包变换的手写体金融汉字识别算法。该算法首先对汉字图像进行二维小波包分解,利用基于子图像能量方差的准则选择适当的部分分解树;然后将得到的子图像划分成多个局部窗口,计算局部窗口的能量值组成特征向量;再通过主成分分析(PCA)选择分类能力最强的一组特征,降低特征空间的维数;最后,将特征向量送入支持向量机进行分类。实验结果表明,该算法取得了较好的识别效果。     

Hidden Markov model-based ensemble methods for offline handwritten text line recognition

This paper investigates various ensemble methods for offline handwritten text line recognition. To obtain ensembles of recognisers, we implement bagging, random feature subspace, and language model variation methods. For the combination, the word sequences returned by the individual ensemble members are first aligned. Then a confidence-based voting strategy determines the final word sequence. A number of confidence measures based on normalised likelihoods and alternative candidates are evaluated. Experiments show that the proposed ensemble methods can improve the recognition accuracy over an optimised single reference recogniser.     

Combining diverse on-line and off-line systems for handwritten text line recognition

In this paper we present a multiple classifier system (MCS) for on-line handwriting recognition. The MCS combines several individual recognition systems based on hidden Markov models (HMMs) and bidirectional long short-term memory networks (BLSTM). Beside using two different recognition architectures (HMM and BLSTM), we use various feature sets based on on-line and off-line features to obtain diverse recognizers. Furthermore, we generate a number of different neural network recognizers by changing the initialization parameters. To combine the word sequences output by the recognizers, we incrementally align these sequences using the recognizer output voting error reduction framework (ROVER). For deriving the final decision, different voting strategies are applied. The best combination ensemble has a recognition rate of 84.13%, which is significantly higher than the 83.64% achieved if only one recognition architecture (HMM or BLSTM) is used for the combination, and even remarkably higher than the 81.26% achieved by the best individual classifier. To demonstrate the high performance of the classification system, the results are compared with two widely used commercial recognizers from Microsoft and Vision Objects.     

Region growing based segmentation algorithm for typewritten and handwritten text recognition

This paper presents a new technique of high accuracy to recognize both typewritten and handwritten English and Arabic texts without thinning. After segmenting the text into lines (horizontal segmentation) and the lines into words, it separates the word into its letters. Separating a text line (row) into words and a word into letters is performed by using the region growing technique (implicit segmentation) on the basis of three essential lines in a text row. This saves time as there is no need to skeletonize or to physically isolate letters from the tested word whilst the input data involves only the basic information—the scanned text. The baseline is detected, the word contour is defined and the word is implicitly segmented into its letters according to a novel algorithm described in the paper. The extracted letter with its dots is used as one unit in the system of recognition. It is resized into a 9 × 9 matrix following bilinear interpolation after applying a lowpass filter to reduce aliasing. Then the elements are scaled to the interval [0,1]. The resulting array is considered as the input to the designed neural network. For typewritten texts, three types of Arabic letter fonts are used—Arial, Arabic Transparent and Simplified Arabic. The results showed an average recognition success rate of 93% for Arabic typewriting. This segmentation approach has also found its application in handwritten text where words are classified with a relatively high recognition rate for both Arabic and English languages. The experiments were performed in MATLAB and have shown promising results that can be a good base for further analysis and considerations of Arabic and other cursive language text recognition as well as English handwritten texts. For English handwritten classification, a success rate of about 80% in average was achieved while for Arabic handwritten text, the algorithm performance was successful in about 90%. The recent results have shown increasing success for both Arabic and English texts.     

Minimum-risk training for semi-Markov conditional random fields with application to handwritten Chinese/Japanese text recognition

Semi-Markov conditional random fields (semi-CRFs) are usually trained with maximum a posteriori (MAP) criterion which adopts the 0/1 cost for measuring the loss of misclassification. In this paper, based on our previous work on handwritten Chinese/Japanese text recognition (HCTR) using semi-CRFs, we propose an alternative parameter learning method by minimizing the risk on the training set, which has unequal misclassification costs depending on the hypothesis and the ground-truth. Based on this framework, three non-uniform cost functions are compared with the conventional 0/1 cost, and training data selection is incorporated to reduce the computational complexity. In experiments of online handwriting recognition on databases CASIA-OLHWDB and TUAT Kondate, we compared the performances of the proposed method with several widely used learning criteria, including conditional log-likelihood (CLL), softmax-margin (SMM), minimum classification error (MCE), large-margin MCE (LM-MCE) and max-margin (MM). On the test set (online handwritten texts) of ICDAR 2011 Chinese handwriting recognition competition, the proposed method outperforms the best system in competition.     

Boosting modern and historical handwritten text recognition with deformable convolutions

International Journal on Document Analysis and Recognition (IJDAR) - Handwritten Text Recognition (HTR) in free-layout pages is a challenging image understanding task that can provide a relevant...     

Writer recognition enhancement by means of synthetically generated handwritten text

This paper presents a new method to generate synthetic executions of on-line words from real samples. The proposed generation method takes advantage of the characteristics of a writer recognition system developed by the authors and can be seamlessly integrated into it. Both the generation method and the recognition system consider strokes as the structural units of handwriting with words being regarded as two sequences, one of pen-up and one of pen-down strokes. Given two samples from the same word and writer, a new sample is produced by aligning their sequences of strokes and then averaging the matching pairs. Thanks to a self-organising map used to categorise strokes, the alignment and comparison of sequences of strokes are performed in a straightforward and computationally efficient way.The synthetically generated words not only retain much of the discriminative power (i.e. the capability to discriminate among writers) of the words from which they are generated, but in some cases exhibit an increased recognition performance. Also, the newly generated words allow enlarging the number of available samples in the enrolment sets that are used to build writers' models. In most cases, this enlargement has the effect to improve the performance of the recognition system. Experimenting with 320 writers and enrolment sets containing 3 real samples and 6 synthetically generated ones, verification is improved for 15 of the 16 words in the BiosecurID database, with the minimum of the detection cost function being reduced by up to a 26.5%.     

Fast self-generation voting for handwritten Chinese character recognition

In this paper, a fast self-generation voting method is proposed for further improving the performance in handwritten Chinese character recognition. In this method, firstly, a set of samples are generated by the proposed fast self-generation method, and then these samples are classified by the baseline classifier, and the final recognition result is determined by voting from these classification results. Two methods that are normalization-cooperated feature extraction strategy and an approximated line density are used for speeding up the self-generation method. We evaluate the proposed method on the CASIA and CASIA-HWDB1.1 databases. High recognition rate of 98.84 % on the CASIA database and 91.17 % on the CASIA-HWDB1.1 database are obtained. These results demonstrate that the proposed method outperforms the state-of-the-art methods and is useful for practical applications.     

基于特征反馈的手写体汉字识别系统研究

针对传统基于开环的汉字识别系统不完全符合人类识字过程的问题,构建了一种具有反馈结构的仿人智能识别系统。该系统根据待识别汉字的多模态定性识别结果来选择最佳的首轮识别方案,在完成识别之后,提取广义识字误差对候选字进行可信度判断和反馈校正。设计了3种广义识字误差,通过对这3种广义识字误差的类型和数值进行定性与定量相结合的分析,建立了识别结果的可信度评价指标体系和反馈校正决策机制。仿真实验结果验证了方法的可行性。     

A novel offline handwritten text recognition technique to convert ruled-line text into digital text through deep neural networks

Multimedia Tools and Applications - Offline Handwritten Text Recognition (HTR) has been an active area of research due to its wide range of applications and challenges. Recently, many offline HTR...     

Thinning and stroke segmentation for handwritten Chinese character recognition

Firstly, a thinning technique by means of stroke tracking is proposed. The method is considered to preserve the straightness of strokes and the length, which is useful for the stroke segmentation procedure on the recognition of handwritten Chinese characters.Secondly, a method for stroke segmentation, i.c. a way of breaking down a character to a set of consecutive partial strokes, is proposed, which works well owing to the favourable properties of the proposed thinning technique. The method consists of five procedures: extraction of feature pixels, calculation of stroke directions, piecewise linear representation of strokes, unification of intersections and extraction of the consecutive partial strokes.     

希尔伯特-黄变换手写体汉字识别方法

针对脱机手写体汉字的非平稳特性,提出了一种基于希尔伯特-黄变换的手写体汉字识别方法.该方法根据汉字图像的特点将一维HHT扩展为二维HHT,用以获得汉字的瞬时频率特征,并将局部的瞬时频率特征和全局的Zernike矩特征组合作为最终的识别特征.实验结果表明,该方法是有效的.