Effects of classifier structures and training regimes on integrated segmentation and recognition of handwritten numeral strings

In integrated segmentation and recognition of character strings, the underlying classifier is trained to be resistant to noncharacters. We evaluate the performance of state-of-the-art pattern classifiers of this kind. First, we build a baseline numeral string recognition system with simple but effective presegmentation. The classification scores of the candidate patterns generated by presegmentation are combined to evaluate the segmentation paths and the optimal path is found using the beam search strategy. Three neural classifiers, two discriminative density models, and two support vector classifiers are evaluated. Each classifier has some variations depending on the training strategy: maximum likelihood, discriminative learning both with and without noncharacter samples. The string recognition performances are evaluated on the numeral string images of the NIST special database 19 and the zipcode images of the CEDAR CDROM-1. The results show that noncharacter training is crucial for neural classifiers and support vector classifiers, whereas, for the discriminative density models, the regularization of parameters is important. The string recognition results compare favorably to the best ones reported in the literature though we totally ignored the geometric context. The best results were obtained using a support vector classifier, but the neural classifiers and discriminative density models show better trade-off between accuracy and computational overhead.     

联机手写维吾尔文字母识别方法

通过分析维吾尔文字母自身的结构和书写特点,提出一种联机手写维吾尔文字母识别方案,并选择在手写汉字识别技术中所提出来的归一化、特征提取及常用的分类方法,从中找出最佳的技术选择。在实验对比中,采用8种不同的归一化预处理方法,基于坐标归一化的特征提取 (NCFE) 方法,以及改进的二次分类函数(MQDF)、判别学习型二次判别函数(DLQDF)、学习矢量量化(LVQ)、支持向量机(SVM)4种分类器。同时,再考虑字符在文档中的空间几何特征,进一步提高识别性能。在128个维吾尔文字母类别、38 400个测试样本的实验中,正确识别率最高达89。08%,为进一步研究面向维吾尔文字母特性的识别技术奠定重要基础。     

Performance evaluation of pattern classifiers for handwritten character recognition

This paper describes a performance evaluation study in which some efficient classifiers are tested in handwritten digit recognition. The evaluated classifiers include a statistical classifier (modified quadratic discriminant function, MQDF), three neural classifiers, and an LVQ (learning vector quantization) classifier. They are efficient in that high accuracies can be achieved at moderate memory space and computation cost. The performance is measured in terms of classification accuracy, sensitivity to training sample size, ambiguity rejection, and outlier resistance. The outlier resistance of neural classifiers is enhanced by training with synthesized outlier data. The classifiers are tested on a large data set extracted from NIST SD19. As results, the test accuracies of the evaluated classifiers are comparable to or higher than those of the nearest neighbor (1-NN) rule and regularized discriminant analysis (RDA). It is shown that neural classifiers are more susceptible to small sample size than MQDF, although they yield higher accuracies on large sample size. As a neural classifier, the polynomial classifier (PC) gives the highest accuracy and performs best in ambiguity rejection. On the other hand, MQDF is superior in outlier rejection even though it is not trained with outlier data. The results indicate that pattern classifiers have complementary advantages and they should be appropriately combined to achieve higher performance. Received: July 18, 2001 / Accepted: September 28, 2001     

Handwritten digit recognition: benchmarking of state-of-the-art techniques

This paper presents the results of handwritten digit recognition on well-known image databases using state-of-the-art feature extraction and classification techniques. The tested databases are CENPARMI, CEDAR, and MNIST. On the test data set of each database, 80 recognition accuracies are given by combining eight classifiers with ten feature vectors. The features include chaincode feature, gradient feature, profile structure feature, and peripheral direction contributivity. The gradient feature is extracted from either binary image or gray-scale image. The classifiers include the k-nearest neighbor classifier, three neural classifiers, a learning vector quantization classifier, a discriminative learning quadratic discriminant function (DLQDF) classifier, and two support vector classifiers (SVCs). All the classifiers and feature vectors give high recognition accuracies. Relatively, the chaincode feature and the gradient feature show advantage over other features, and the profile structure feature shows efficiency as a complementary feature. The SVC with RBF kernel (SVC-rbf) gives the highest accuracy in most cases but is extremely expensive in storage and computation. Among the non-SV classifiers, the polynomial classifier and DLQDF give the highest accuracies. The results of non-SV classifiers are competitive to the best ones previously reported on the same databases.     

Building compact MQDF classifier for large character set recognition by subspace distribution sharing

Quadratic classifier with modified quadratic discriminant function (MQDF) has been successfully applied to recognition of handwritten characters to achieve very good performance. However, for large category classification problem such as Chinese character recognition, the storage of the parameters for the MQDF classifier is usually too large to make it practical to be embedded in the memory limited hand-held devices. In this paper, we aim at building a compact and high accuracy MQDF classifier for these embedded systems. A method by combining linear discriminant analysis and subspace distribution sharing is proposed to greatly compress the storage of the MQDF classifier from 76.4 to 2.06 MB, while the recognition accuracy still remains above 97%, with only 0.88% accuracy loss. Furthermore, a two-level minimum distance classifier is employed to accelerate the recognition process. Fast recognition speed and compact dictionary size make the high accuracy quadratic classifier become practical for hand-held devices.     

基于MQDF-DBM模型的脱机手写体汉字识别

针对脱机手写体汉字识别准确率较低的问题,提出一种基于修正的二次判别函数（Modified Quadratic Discriminant Function,MQDF）与深度玻尔兹曼机（Deep Boltzmann Machine,DBM）的分类器级联模型。该模型的主要思想是MQDF和DBM在特征提取和分类机制上可以相辅相成。先用MQDF进行识别并得出结果,同时计算该结果的一个广义置信度。若置信度满足要求,则将识别结果作为最终结果输出,否则结合DBM进行二次识别,得到最终识别结果。实验结果表明,使用MQDF-DBM模型可以获得比单独使用MQDF和DBM模型更高的识别准确率,且识别速度比DBM更快。     

Approximate Test Risk Bound Minimization Through Soft Margin Estimation

Inspired by the great success of margin-based classifiers, there is a trend to incorporate the margin concept into hidden Markov modeling for speech recognition. Several attempts based on margin maximization were proposed recently. In this paper, a new discriminative learning framework, called soft margin estimation (SME), is proposed for estimating the parameters of continuous-density hidden Markov models. The proposed method makes direct use of the successful ideas of soft margin in support vector machines to improve generalization capability and decision feedback learning in minimum classification error training to enhance model separation in classifier design. SME is illustrated from a perspective of statistical learning theory. By including a margin in formulating the SME objective function, SME is capable of directly minimizing an approximate test risk bound. Frame selection, utterance selection, and discriminative separation are unified into a single objective function that can be optimized using the generalized probabilistic descent algorithm. Tested on the TIDIGITS connected digit recognition task, the proposed SME approach achieves a string accuracy of 99.43%. On the 5 k-word Wall Street Journal task, SME obtains relative word error rate reductions of about 10% over our best baseline results in different experimental configurations. We believe this is the first attempt to show the effectiveness of margin-based acoustic modeling for large vocabulary continuous speech recognition in a hidden Markov model framework. Further improvements are expected because the approximate test risk bound minimization principle offers a flexible and rigorous framework to facilitate incorporation of new margin-based optimization criteria into hidden Markov model training.     

多字体印刷藏文字符识别

藏文字符识别系统是中文多文种信息处理系统的重要组成部分,但至今国内外的研究基本处于空白。本文提出了一种基于统计模式识别的多字体印刷藏文字符识别方法:从字符轮廓中抽取方向线素特征,利用线性鉴别分析(LDA)压缩降维后得到紧凑的字符特征向量。采用基于置信度分析的两级分类策略,设计了带偏差欧氏距离分类器(EDD)完成高效的粗分类,细分类采用修正二次鉴别函数(MQDF)。通过实验选取恰当的分类器参数后,在容量为177,600字符(300样本/字符类)的测试集上的识别率达到99.79%,证明了该方法的有效性。     

基于堆叠模型的司法短文本多标签分类

司法文书短文本的语义多样性和特征稀疏性等特点,对短文本多标签分类精度提出了很大的挑战,传统单一模型的分类算法已无法满足业务需求。为此,提出一种融合深度学习与堆叠模型的多标签分类方法。该方法将分类器划分成两个层次,第一层使用BERT、卷积神经网络、门限循环单元等深度学习方法作为基础分类器,每个基础分类器模型通过K折交叉验证得到所有数据的多标签分类概率值,将此概率值数据进行融合形成元数据;第二层使用自定义的深度神经网络作为混合器,以第一层的元数据为输入,通过训练多标签概率矩阵获取模型参数。该方法将强分类器关联在一起,获得比单个分类器更加强大的性能。实验结果表明,深度学习堆叠模型实现了87%左右的短文本分类F1分数,优于BERT、卷积神经网络、循环神经网络及其他单个模型的性能。     

Speech recognition using hybrid hidden markov model and NN classifier

This paper discusses the use of an integrated HMM/NN classifier for speech recognition. The proposed classifier combines the time normalization property of the HMM classifier with the superior discriminative ability of the neural net (NN) classifier. Speech signals display a strong time varying characteristic. Although the neural net has been successful in many classification problems, its success (compared to HMM) is secondary to HMM in the field of speech recognition. The main reason is the lack of time normalization characteristics of most neural net structures (time-delay neural net is one notable exception but its structure is very complex). In the proposed integrated hybrid HMM/NN classifier, a left-to-right HMM module is used first to segment the observation sequence of every exemplar into a fixed number of states. Subsequently, all the frames belonging to the same state are replaced by one average frame. Thus, every exemplar, irrespective of its time scale variation, is transformed into a fixed number of frames, i.e., a static pattern. The multilayer perceptron (MLP) neural net is then used as the classifier for these time normalized exemplars. Some experimental results using telephone speech databases are presented to demonstrate the potential of this hybrid integrated classifier.     

基于卷积神经网络与集成学习的交通标志识别

为提高复杂情况（如遮挡、透视畸变等）下交通标志识别的精度,提出一种有效的基于卷积神经网络(Convolutional Neural Network, CNN)与集成学习的交通标志识别方法。首先通过融合颜色分割、形态学处理、形状检测等多种方法分割出交通标志,然后利用卷积神经网络对其特征进行提取并分别采用支持向量机(Support Vector Machine, SVM)和Softmax多类分类器对其进行识别,最后将2种分类结果进行集成作为最终的识别结果。实验结果表明,本文算法可有效提高复杂情况下交通标志识别精度,整体上具有较高的性能。     

基于条件对数似然函数导数的贝叶斯网络分类器优化算法

通常基于鉴别式学习策略训练的贝叶斯网络分类器有较高的精度,但在具有冗余边的网络结构之上鉴别式参数学习算法的性能受到一定的限制.为了在实际应用中进一步提高贝叶斯网络分类器的分类精度,该文定量描述了网络结构与真实数据变量分布之间的关系,提出了一种不存在冗余边的森林型贝叶斯网络分类器及其相应的FAN学习算法(Forest-Augmented Naive Bayes Algorithm),FAN算法能够利用对数条件似然函数的偏导数来优化网络结构学习.实验结果表明常用的限制性贝叶斯网络分类器通常存在一些冗余边,其往往会降低鉴别式参数学习算法的性能;森林型贝叶斯网络分类器减少了结构中的冗余边,更加适合于采用鉴别式学习策略训练参数;应用条件对数似然函数偏导数的FAN算法在大多数实验数据集合上提高了分类精度.     

High accuracy handwritten Chinese character recognition using LDA-based compound distances

To improve the accuracy of handwritten Chinese character recognition (HCCR), we propose linear discriminant analysis (LDA)-based compound distances for discriminating similar characters. The LDA-based method is an extension of previous compound Mahalanobis function (CMF), which calculates a complementary distance on a one-dimensional subspace (discriminant vector) for discriminating two classes and combines this complementary distance with a baseline quadratic classifier. We use LDA to estimate the discriminant vector for better discriminability and show that under restrictive assumptions, the CMF is a special case of our LDA-based method. Further improvements can be obtained when the discriminant vector is estimated from higher-dimensional feature spaces. We evaluated the methods in experiments on the ETL9B and CASIA databases using the modified quadratic discriminant function (MQDF) as baseline classifier. The results demonstrate the superiority of LDA-based method over the CMF and the superiority of discriminant vector learning from high-dimensional feature spaces. Compared to the MQDF, the proposed method reduces the error rates by factors of over 26%.     

贝叶斯分类器的判别式参数学习

为了提高贝叶斯分类器的分类性能,针对贝叶斯网络分类器的构成特征,提出一种基于参数集成的贝叶斯分类器判别式参数学习算法PEBNC。该算法将贝叶斯分类器的参数学习视为回归问题,将加法回归模型应用于贝叶斯网络分类器的参数学习,实现贝叶斯分类器的判别式参数学习。实验结果表明,在大多数实验数据上,PEBNC能够明显提高贝叶斯分类器的分类准确率。此外,与一般的贝叶斯集成分类器相比,PEBNC不必存储成员分类器的参数,空间复杂度大大降低。     

Chinese calligraphic style representation for recognition

Chinese calligraphy draws a lot of attention for its beauty and elegance. The various styles of calligraphic characters make calligraphy even more charming. But it is not always easy to recognize the calligraphic style correctly, especially for beginners. In this paper, an automatic character styles representation for recognition method is proposed. Three kinds of features are extracted to represent the calligraphic characters. Two of them are typical hand-designed features: the global feature, GIST and the local feature, scale invariant feature transform. The left one is deep feature which is extracted by a deep convolutional neural network (CNN). The state-of-the-art classifier modified quadratic discriminant function was employed to perform recognition. We evaluated our method on two calligraphic character datasets, the unconstraint real-world calligraphic character dataset (CCD) and SCL (the standard calligraphic character library). And we also compare MQDF with other two classifiers, support vector machine and neural network, to perform recognition. In our experiments, all three kinds of feature are evaluated with all three classifiers, respectively, finding that deep feature is the best feature for calligraphic style recognition. We also fine-tune the deep CNN (alex-net) in Krizhevsky et al. (Advances in Neural Information Processing Systems, pp. 1097–1105, 2012) to perform calligraphic style recognition. It turns out our method achieves about equal accuracy comparing with the fine-tuned alex-net but with much less training time. Furthermore, the algorithm style discrimination evaluation is developed to evaluate the discriminative style quantitatively.     

Aerial scene classification via an ensemble extreme learning machine classifier based on discriminative hybrid convolutional neural networks features

Identifying a discriminative feature can effectively improve the classification performance of aerial scene classification. Deep convolutional neural networks (DCNN) have been widely used in aerial scene classification for its learning discriminative feature ability. The DCNN feature can be more discriminative by optimizing the training loss function and using transfer learning methods. To enhance the discriminative power of a DCNN feature, the improved loss functions of pretraining models are combined with a softmax loss function and a centre loss function. To further improve performance, in this article, we propose hybrid DCNN features for aerial scene classification. First, we use DCNN models with joint loss functions and transfer learning from pretrained deep DCNN models. Second, the dense DCNN features are extracted, and the discriminative hybrid features are created using linear connection. Finally, an ensemble extreme learning machine (EELM) classifier is adopted for classification due to its general superiority and low computational cost. Experimental results based on the three public benchmark data sets demonstrate that the hybrid features obtained using the proposed approach and classified by the EELM classifier can result in remarkable performance.     

产生式与判别式线性混合分类器

产生式方法和判别式方法是解决分类问题的两种不同框架,具有各自的优势。为利用两种方法各自的优势,文中提出一种产生式与判别式线性混合分类模型,并设计一种基于遗传算法的产生式与判别式线性混合分类模型的学习算法。该算法将线性混合分类器混合参数的学习看作一个最优化问题,以两个基分类器对每个训练数据的后验概率值为数据依据,用遗传算法找出线性混合分类器混合参数的最优值。实验结果表明,在大多数数据集上,产生式与判别式线性混合分类器的分类准确率优于或近似于它的两个基分类器中的优者。     

基于模糊高斯基函数神经网络的遥感图像分类

针对遥感图像分类的特点,提出了一种基于模糊高斯基函数神经网络的遥感图像分类器。该分类器将模糊技术与神经网络相结合,采用神经网络来实现模糊推理,利用神经网络的学习能力来达到调整模糊隶属函数和模型规则的目的,从而使系统具备了自适应的特性,实验结果表明,这种基于模糊高斯基孙数神经网络的分类器经过训练后,可应用于遥感图像的分类,其分类精度明显高于传统的最大似然分类法。     

触觉手势情感识别的超限学习方法

为了解决声音和图像情感识别的不足,提出一种新的情感识别方式:触觉情感识别。对CoST（corpus of social touch）数据集进行了一系列触觉情感识别研究,对CoST数据集进行数据预处理,提出一些关于触觉情感识别的特征。利用极限学习机分类器探究不同手势下的情感识别,对14种手势下的3种情感（温柔、正常、暴躁）进行识别,准确度较高,且识别速度快识别时间短。结果表明,手势的不同会影响情感识别的准确率,其中手势“stroke”的识别效果在不同分类器下的分类精度均为最高,且有较好的分类精度,达到72.07%;极限学习机作为触觉情感识别的分类器,具有较好的分类效果,识别速度快;有的手势本身对应着某种情感,从而影响分类结果。     

Class-specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition

The polynomial classifier (PC) that takes the binomial terms of reduced subspace features as inputs has shown superior performance to multilayer neural networks in pattern classification. In this paper, we propose a class-specific feature polynomial classifier (CFPC) that extracts class-specific features from class-specific subspaces, unlike the ordinary PC that uses a class-independent subspace. The CFPC can be viewed as a hybrid of ordinary PC and projection distance method. The class-specific features better separate one class from the others, and the incorporation of class-specific projection distance further improves the separability. The connecting weights of CFPC are efficiently learned class-by-class to minimize the mean square error on training samples. To justify the promise of CFPC, we have conducted experiments of handwritten digit recognition and numeral string recognition on the NIST Special Database 19 (SD19). The digit recognition task was also benchmarked on two standard databases USPS and MNIST. The results show that the performance of CFPC is superior to that of ordinary PC, and is competitive with support vector classifiers (SVCs).