An expert system for general symbol recognition

An expert system for analysis and recognition of general symbols is introduced. The system uses the structural pattern recognition technique for modeling symbols by a set of straight lines referred to as segments. The system rotates, scales and thins the symbol, then extracts the symbol strokes. Each stroke is transferred into segments (straight lines). The system is shown to be able to map similar styles of the symbol to the same representation. When the system had some stored models for each symbol (an average of 97 models/symbol), the rejection rate was 16.1% and the recognition rate was 83.9% of which 95% was recognized correctly. The system is tested by 5726 handwritten characters from the Center of Excellence for Document Analysis and Recognition (CEDAR) database. The system is capable of learning new symbols by simply adding their models to the system knowledge base.     

语义制导的建筑结构图的全局识别方法

建筑结构图CSD的语法识别方法存在对噪声和差错的敏感性强以及通用性较差等局限,本文对此提出了两点改进方法。一方面,CSD中图形符号之间的关系得到了充分的重视。相互关联的符号分为核心符号和导出符号两类,彼此间关系紧密的所有核心符号被结构化为一个全局符号加以识别。导出符号的识别以核心符号为基础,另一方面,用图形符号在建筑结构领域背景下的语义控制和指导核心符号与导出符号的识别。     

Recognizing handwritten Arabic words using grapheme segmentation and recurrent neural networks

The Arabic alphabet is used in around 27 languages, including Arabic, Persian, Kurdish, Urdu, and Jawi. Many researchers have developed systems for recognizing cursive handwritten Arabic words, using both holistic and segmentation-based approaches. This paper introduces a system that achieves high accuracy using efficient segmentation, feature extraction, and recurrent neural network (RNN). We describe a robust rule-based segmentation algorithm that uses special feature points identified in the word skeleton to segment the cursive words into graphemes. We show that careful selection from a wide range of features extracted during and after the segmentation stage produces a feature set that significantly reduces the label error. We demonstrate that using same RNN recognition engine, the segmentation approach with efficient feature extraction gives better results than a holistic approach that extracts features from raw pixels. We evaluated this segmentation approach against an improved version of the holistic system MDLSTM that won the ICDAR 2009 Arabic handwritten word recognition competition. On the IfN/ENIT database of handwritten Arabic words, the segmentation approach reduces the average label error by 18.5 %, the sequence error by 22.3 %, and the execution time by 31 %, relative to MDLSTM. This approach also has the best published accuracies on two IfN/ENIT test sets.     

Computer recognition of printed Tamil characters

Computer recognition of machine-printed letters of the Tamil alphabet is described. Each character is represented as a binary matrix and encoded into a string using two different methods. The encoded strings form a dictionary. A given text is presented symbol by symbol and information from each symbol is extracted in the form of a string and compared with the strings in the dictionary. When there is agreement the letters are recognized and printed out in Roman letters following a special method of transliteration. The lengthening of vowels and hardening of consonants are indicated by numerals printed above each letter.     

Confidence- and margin-based MMI/MPE discriminative training for off-line handwriting recognition

We present a novel confidence- and margin-based discriminative training approach for model adaptation of a hidden Markov model (HMM)-based handwriting recognition system to handle different handwriting styles and their variations. Most current approaches are maximum-likelihood (ML) trained HMM systems and try to adapt their models to different writing styles using writer adaptive training, unsupervised clustering, or additional writer-specific data. Here, discriminative training based on the maximum mutual information (MMI) and minimum phone error (MPE) criteria are used to train writer-independent handwriting models. For model adaptation during decoding, an unsupervised confidence-based discriminative training on a word and frame level within a two-pass decoding process is proposed. The proposed methods are evaluated for closed-vocabulary isolated handwritten word recognition on the IFN/ENIT Arabic handwriting database, where the word error rate is decreased by 33% relative compared to a ML trained baseline system. On the large-vocabulary line recognition task of the IAM English handwriting database, the word error rate is decreased by 25% relative.     

Fast self-generation voting for handwritten Chinese character recognition

In this paper, a fast self-generation voting method is proposed for further improving the performance in handwritten Chinese character recognition. In this method, firstly, a set of samples are generated by the proposed fast self-generation method, and then these samples are classified by the baseline classifier, and the final recognition result is determined by voting from these classification results. Two methods that are normalization-cooperated feature extraction strategy and an approximated line density are used for speeding up the self-generation method. We evaluate the proposed method on the CASIA and CASIA-HWDB1.1 databases. High recognition rate of 98.84 % on the CASIA database and 91.17 % on the CASIA-HWDB1.1 database are obtained. These results demonstrate that the proposed method outperforms the state-of-the-art methods and is useful for practical applications.     

Finding consensus in speech recognition: word error minimization and other applications of confusion networks

We describe a new framework for distilling information from word lattices to improve the accuracy of the speech recognition output and obtain a more perspicuous representation of a set of alternative hypotheses. In the standard MAP decoding approach the recognizer outputs the string of words corresponding to the path with the highest posterior probability given the acoustics and a language model. However, even given optimal models, the MAP decoder does not necessarily minimize the commonly used performance metric, word error rate (WER). We describe a method for explicitly minimizing WER by extracting word hypotheses with the highest posterior probabilities from word lattices. We change the standard problem formulation by replacing global search over a large set of sentence hypotheses with local search over a small set of word candidates. In addition to improving the accuracy of the recognizer, our method produces a new representation of a set of candidate hypotheses that specifies the sequence of word-level confusions in a compact lattice format. We study the properties of confusion networks and examine their use for other tasks, such as lattice compression, word spotting, confidence annotation, and reevaluation of recognition hypotheses using higher-level knowledge sources.     

Arrow R-CNN for handwritten diagram recognition

We address the problem of offline handwritten diagram recognition. Recently, it has been shown that diagram symbols can be directly recognized with deep learning object detectors. However, object detectors are not able to recognize the diagram structure. We propose Arrow R-CNN, the first deep learning system for joint symbol and structure recognition in handwritten diagrams. Arrow R-CNN extends the Faster R-CNN object detector with an arrow head and tail keypoint predictor and a diagram-aware postprocessing method. We propose a network architecture and data augmentation methods targeted at small diagram datasets. Our diagram-aware postprocessing method addresses the insufficiencies of standard Faster R-CNN postprocessing. It reconstructs a diagram from a set of symbol detections and arrow keypoints. Arrow R-CNN improves state-of-the-art substantially: on a scanned flowchart dataset, we increase the rate of recognized diagrams from 37.7 to 78.6%.

     

多模版匹配的特殊符号识别定位算法研究

针对试卷智能批阅场景模式,由于Tesseract-OCR缺少特殊符号包,直接定位符号存在较多漏检等问题,提出具有覆盖保留机制的多模板匹配方法。通过OCR定位空白试卷中的符号分别建立多类型元素的方块、圆圈、括号模板集;而对于试卷中的直线,通过筛选查找轮廓的方法建立多类型元素的直线模板集,综合多模板匹配技术提高试卷中符号的识别性能及定位准确率。经实际试卷测试结果表明:该算法符号定位准确率、精确度和召回率均高于94%;查找轮廓法定位直线准确率达96%,模板匹配直线定位准确率、精确度和召回率高于87%;将空白试卷符号坐标应用于学生作答试卷,能较完美地定位手写答案。     

Detection of a polymorphic Mesoamerican symbol using a rule-based approach

This paper describes a technique to recognize a Mesoamerican symbol whose shape is extremely variable. We extract symbols from a drawing and we encode them as discrete curves. We perform the recognition using a set of rules that define the correct symbol. One of the rules is the presence of a single symmetry axis. We describe a comparison metric between curves in order to search for symmetries. The other rules used for the recognition concern the morphology of the symbol. The proposed technique proves to be fast and efficient. We present recognition results obtained on various pre-hispanic images. The rule-based approach proposed and implemented here, appears suitable to detect polymorphic signs, a common feature of Mesoamerican symbols. To the best of our knowledge, this is the first study of pattern recognition into the field of Mesoamerican iconography.     

基于改进YOLOv5s的离线手写数学符号识别

离线数学符号识别是离线数学表达式识别的前提。针对现有离线符号识别方法只是单纯的对符号进行识别,对离线表达式识别的其他环节未有任何帮助,反而会限制表达式识别,提出一种改进YOLOv5s的离线符号识别方法。首先,根据符号图像小的特点,用生成对抗网络(GAN)进行数据增强;其次,从符号类别的角度分析,在YOLOv5s模型中引入空间注意力机制,利用全局最大值和全局平均值池化,扩大类别间的差异特征;最后,从符号自身角度分析,引入双向长短期记忆网络(BiLSTM)对符号特征矩阵进行处理,使符号特征具有上下相关联的信息。实验结果表明：改进后的YOLOv5s取得较好离线符号识别效果,有92.47%的识别率,与其他方法进行对比,证明了其有效性和稳健性。同时,能有效避免离线数学表达式识别中错误累积的问题,且能为表达式的结构分析提供有效依据。     

基于模糊理论的数学公式上下标关系判别

上下标关系数学公式中出现频繁又难于解决的特殊结构,容易与其它关系混淆.提出了基于模糊理论的数学公式上下标关系判别.运用模糊理论对数学公式中符号的空间区域关系进行划分,然后应用模糊识别的方法对上下标关系进行判别.实验结果表明,运用该方法能明显提高符号空间关系判别的识别率,尤其是能很好地判别手写数学公式中的空间关系,识别的正确率可达到96.4%.     

Improving discrimination ability of convolutional neural networks by hybrid learning

The discrimination of similar patterns is important because they are the major sources of the classification error. This paper proposes a novel method to improve the discrimination ability of convolutional neural networks (CNNs) by hybrid learning. The proposed method embeds a collection of discriminators as well as a recognizer in a shared CNN. By visualizing contrastive class saliency, we show that learning with embedded discriminators leads the shared CNN to detect and catch the differences among similar classes. Also proposed is a hybrid learning algorithm that learns recognition and discrimination together. The proposed method learns recognition focusing on the differences among similar classes, and thereby improves the discrimination ability of the CNN. Unlike conventional discrimination methods, the proposed method does not require predefined sets of similar classes or additional step to integrate its result with that of the recognizer. In experiments on two handwritten Hangul databases SERI95a and PE92, the proposed method reduced classification error from 2.56 to 2.33, and from 4.04 to 3.66 % respectively. These improvement lead to relative error reduction rates of 8.97 % on SERI95a, and 9.42 % on PE92. Our best results update the state-of-the-art performance which were 4.04 % on SERI95a and 7.08 % on PE92.     

声学模型区分性训练中的动态加权数据选取方法

提出了一种基于动态加权的数据选取方法, 并应用到连续语音识别的声学模型区分性训练中. 该方法联合后验概率和音素准确率选取数据, 首先, 采用后验概率的Beam算法裁剪词图, 在此基础上依据候选词所在候选路径的错误率, 基于后验概率动态的赋予候选词不同的权值; 其次, 通过统计音素对之间的混淆程度, 给易混淆音素对动态地加以不同的惩罚权重, 计算音素准确率; 最后, 在估计得到弧段期望准确率分布的基础上, 采用高斯函数形式对所有竞争弧段的期望音素准确率软加权.实验结果表明, 与最小音素错误准则相比, 该动态加权方法识别准确率提高了0.61%, 可有效减少训练时间.     

基于游程特征的选票符号识别方法

选票符号识别是基于图像理解的计票系统的关键技术,为提高选票符号识别的正确率,提出了一种基于游程特征的选票符号识别方法。首先给出了选票符号游程特征的定义,构建了选票符号的游程判定模型;然后利用三叉树结构描述了游程区域之间的相对位置关系;此外,通过游程区域的合并实现了噪声环境下主游程区域的提取,并对歧义符号的处理方法进行了研究;最后,实验结果表明,游程特征能够准确描述选票符号的几何特征,所给出的算法细分能力强,识别正确率高,比基于模板匹配的算法的正确率提高了6.07%。     

Use of multilayer networks for the recognition of phonetic features and phonemes

Artificial neural networks capable of doing hard learning offer a new way to undertake automatic speech recognition. The Boltzmann machine algorithm and the error back-propagation algorithm have been used to perform speaker normalization. Spectral segments are represented by spectral lines. Speaker-independent recognition of place of articulation for vowels is performed on lines. Performance of the networks is shown to depend on the coding of the input data. Samples were extracted from continuous speech of 38 speakers. The error rate obtained (4.2% error on test set of 72 samples with the Boltzmann machine and 6.9% error with error back-propagation) is better than that of previous experiments, using the same data, with continuous Hidden Markov Models (7.3% error on test set and 3% error on training set). These experiments are part of an attempt to construct a data-driven speech recognition system with multiple neural networks specialized to different tasks. Results are also reported on the recognition performance of other trained networks, such as one trained on the E-set consonants.