一种识别手写汉字的多分类器集成方法

根据多信源信息处理与字符识别的经验知识，提出了一个识别手写汉字的多分类器线性集成模型．这个模型不仅考虑到不同的分类器对不同字符识别能力的不同，而且还考虑了不同的分类器得出的输入字符与参考模板之间相似度的实际大小对判决的影响，及不同分类器提供的候选字符对判决的支持作用，更重要的是提供了一种通过监督学习，利用计算机程序自动计算模型参数的方法，因而实现了一个较好的集成系统．同时，本文还提供了三个用于集成的分类器，它们集成的结果充分显示了本方法的有效性。     

一种识别手写汉字的多分类器集成方法

根据多信源信息处理与字符识别的经验知识,提出了一个识别手写汉字的多分类器 线性集成模型.这个模型不仅考虑到不同的分类器对不同字符识别能力的不同,而且还考虑 了不同的分类器得出的输入字符与参考模板之间相似度的实际大小对判决的影响,及不同分 类器提供的候选字符对判决的支持作用,更重要的是提供了一种通过监督学习,利用计算机 程序自动计算模型参数的方法,因而实现了一个较好的集成系统.同时,本文还提供了三个用 于集成的分类器,它们集成的结果充分显示了本方法的有效性.     

一种新的基于多样性赋权证据推理的集成学习方法

在集成学习中使用平均法、投票法作为结合策略无法充分利用基分类器的有效信息,且根据波动性设置基分类器的权重不精确、不恰当。以上问题会降低集成学习的效果,为了进一步提高集成学习的性能,提出将证据推理(evidence reasoning, ER)规则作为结合策略,并使用多样性赋权法设置基分类器的权重。首先,由多个深度学习模型作为基分类器、ER规则作为结合策略,构建集成学习的基本结构;然后,通过多样性度量方法计算每个基分类器相对于其他基分类器的差异性;最后,将差异性归一化实现基分类器的权重设置。通过多个图像数据集的分类实验,结果表明提出的方法较实验选取的其他方法准确率更高且更稳定,证明了该方法可以充分利用基分类器的有效信息,且多样性赋权法更精确。     

基于选择性集成分类器的面部表情识别研究*

为了提高面部表情的分类识别性能,基于集成学习理论,提出了一种二次优化选择性(Quadratic Optimization Choice, QOC)集成分类模型。首先,对于9个基分类器,依据性能进行排序,选择前30%的基分类器作为集成模型的候选基分类器。其次,依据组合规则产生集成模型簇。最后,对集成模型簇进行二次优化选择,选择具有最小泛化误差的集成分类器的子集,从而确定最优集成分类模型。为了验证QOC集成分类模型的性能,选择采用最大值、最小值和均值规则的集成模型作为对比模型,实验结果表明：相对基分类器,QOC集成分类模型取得了较好的分类效果,尤其是对于识别率较差的悲伤表情类,平均识别率提升了21.11%。相对于非选择性集成模型,QOC集成分类模型识别性能也有显著提高。     

基于全信息矩阵的多分类器集成方法

自动文本分类是提高信息利用效率和质量的有效方法,而多分类器的有效组合能够得到更高的分类准确率.给出了样本集在多分类器下的全信息矩阵概念,并提出一种权重自适应调整的多分类器集成方法.该方法能够自适应地选择分类器组合及确定分类器权重,并利用分类统计信息指导分类结果的集成判决.通过在标准文本集Reuters-21578上的实验表明:该方法能从查准率和查全率两方面提高文本分类的整体性能,同时表明了该方法的有效性.     

一种新的启发式分类器选择方法

分类器选择是一种设计多分类器系统的有效方法,从给定候选分类器集中挑选出一个子集,使得该子集集成性能最佳。现有的分类器选择方法大多采用基于集成精度的随机搜索方法,但巨大的搜索复杂度限制了它们在更大系统中的应用。该文提出一种新的选择标准——IWCECR及一种基于IWCECR的启发式搜索算法,在手写体数字识别的实验中,从20个候选分类器中挑选子集,结果表明,该方法具有较高的搜索效率,在子集集成性能方面仅次于穷举法。     

基于免疫克隆的核匹配追踪集成图像识别算法

为了从分类器集成系统中选择一组较优个体子集,从而改善集成学习系统的性能,提出一种基于免疫克隆选择的核匹配追踪集成图像识别算法.该算法充分利用免疫克隆算法的快速收敛于全局最优解的特性,对训练得到的多个子核匹配追踪分类器进行免疫克降选择,得到一个具有更好推广性能的集成系统.对Brodatz纹理图像库以及SAR图像进行目标识别.仿真实验结果表明,相比传统核匹配追踪分类器集成和基于遗传算法的选择集成方法,本文方法有更好的集成性能.     

快速多分类器集成算法研究

研究快速多分类器集成算法。对多分类器集成需选定一定数量的弱分类器,再为每个弱分类器分配一定权重。在选择弱分类器时,通过计算每个弱分类器在全部训练样本集上的分类错误率,对其进行排序,挑选出分类效果最好的若干弱分类器。在多分类器权重分配策略上,提出2种权重分配方法：Biased AdaBoost算法与基于差分演化的多分类器集成算法。在人脸数据库上的实验结果表明,与经典AdaBoost算法相比,该算法能有效降低训练时间,提高识别准确率。     

基于重采样和集成选择的适用于实体识别的多分类器系统

实体识别常利用分类器根据记录对的字段相似度向量将记录对分为匹配、不匹配和可能匹配,因此分类器的准确性与实体识别的准确性直接相关。为提高分类准确性,本文基于重采样和集成选择技术构建一个多分类器系统。充分利用实体识别的特点,在分类之前发现分类困难的样本,并使重采样比率在一个区间内变化,生成一组重采样样本;然后用重采样后的样本训练分类器构建一个并行多分类器系统,强调分类器之间的差异度和稀疏度,从该多分类器系统中选择最优分类器子集,即最优的重采样比率组合,分别用非线性规划和极值方法求解该集成选择模型。实验结果表明,本方法与现有的多分类器系统相比具有更高的准确性。     

基于联合训练的分类器的乳腺癌图像分类

利用机器学习的乳腺癌组织病理图像诊断节省了大量的人力物力,因此提高乳腺癌组织病理图像识别准确率有很好的现实意义;针对单一分类器和集成学习分类器模型观测域有限容易陷入局部最优的问题,提出一种基于联合训练的分类器模型;通过单一分类器相互影响扩大观测感知域来寻找损失最小的估计点,根据估计点来迭代优化超参数进而联合训练出拟合性能最好的分类器,这样既汲取不同分类器模型的可取之处来增强泛化能力,又加大了模型观测域在可以更快的得到全局最优的同时提升了识别准确率;实验表明,提出的联合训练的分类器能够提升乳腺癌组织病理学图像的分类性能,在不同放大倍数40×、100×、200×、400×下图像良恶性分类准确率分别为99.67%、98.08%、99.01%、96.34%。     

Optimal linear combination of neural networks for improvingclassification performance

This paper presents a new method for linearly combining multiple neural network classifiers based on the statistical pattern recognition theory. In our approach, several neural networks are first selected based on which works best for each class in terms of minimizing classification errors. Then, they are linearly combined to form an ideal classifier that exploits the strengths of the individual classifiers. In this approach, the minimum classification error criterion is utilized to estimate the optimal linear weights. In this formulation, because the classification decision rule is incorporated into the cost function, a more suitable better combination of weights for the classification objective could be obtained. Experimental results using artificial and real data sets show that the proposed method can construct a better combined classifier that outperforms the best single classifier in terms of overall classification errors for test data     

Classifier combination based on confidence transformation

This paper investigates the effects of confidence transformation in combining multiple classifiers using various combination rules. The combination methods were tested in handwritten digit recognition by combining varying classifier sets. The classifier outputs are transformed to confidence measures by combining three scaling functions (global normalization, Gaussian density modeling, and logistic regression) and three confidence types (linear, sigmoid, and evidence). The combination rules include fixed rules (sum-rule, product-rule, median-rule, etc.) and trained rules (linear discriminants and weighted combination with various parameter estimation techniques). The experimental results justify that confidence transformation benefits the combination performance of either fixed rules or trained rules. Trained rules mostly outperform fixed rules, especially when the classifier set contains weak classifiers. Among the trained rules, the support vector machine with linear kernel (linear SVM) performs best while the weighted combination with optimized weights performs comparably well. I have also attempted the joint optimization of confidence parameters and combination weights but its performance was inferior to that of cascaded confidence transformation-combination. This justifies that the cascaded strategy is a right way of multiple classifier combination.     

Automatic recognition of serial numbers in bank notes

This paper presents a new topic of automatic recognition of bank note serial numbers, which will not only facilitate the prevention of forgery crimes, but also have a positive impact on the economy. Among all the different currencies, we focus on the study of RMB (renminbi bank note, the paper currency used in China) serial numbers. For evaluation, a new database NUST-RMB2013 has been collected from scanned RMB images, which contains the serial numbers of 35 categories with 17,262 training samples and 7000 testing samples in total. We comprehensively implement and compare two classic and one newly merged feature extraction methods (namely gradient direction feature, Gabor feature, and CNN trainable feature), four different types of well-known classifiers (SVM, LDF, MQDF, and CNN), and five multiple classifier combination strategies (including a specially designed novel cascade method). To further improve the recognition accuracy, the enhancements of three different kinds of distortions have been tested. Since high reliability is more important than accuracy in financial applications, we introduce three rejection schemes of first rank measurement (FRM), first two ranks measurement (FTRM) and linear discriminant analysis based measurement (LDAM). All the classifiers and classifier combination schemes are combined with different rejection criteria. A novel cascade rejection measurement achieves 100% reliability with less rejection rate compared with the existing methods. Experimental results show that MQDF reaches the accuracy of 99.59% using the gradient direction feature trained with gray level normalized data; the cascade classifier combination achieves the best performance of 99.67%. The distortions have been proved to be very helpful because the performances of CNNs boost at least 0.5% by training with transformed samples. With the cascade rejection method, 100% reliability has been obtained by rejecting 1.01% test samples.     

Local Classifier Weighting by Quadratic Programming

It has been widely accepted that the classification accuracy can be improved by combining outputs of multiple classifiers. However, how to combine multiple classifiers with various (potentially conflicting) decisions is still an open problem. A rich collection of classifier combination procedures-many of which are heuristic in nature-have been developed for this goal. In this brief, we describe a dynamic approach to combine classifiers that have expertise in different regions of the input space. To this end, we use local classifier accuracy estimates to weight classifier outputs. Specifically, we estimate local recognition accuracies of classifiers near a query sample by utilizing its nearest neighbors, and then use these estimates to find the best weights of classifiers to label the query. The problem is formulated as a convex quadratic optimization problem, which returns optimal nonnegative classifier weights with respect to the chosen objective function, and the weights ensure that locally most accurate classifiers are weighted more heavily for labeling the query sample. Experimental results on several data sets indicate that the proposed weighting scheme outperforms other popular classifier combination schemes, particularly on problems with complex decision boundaries. Hence, the results indicate that local classification-accuracy-based combination techniques are well suited for decision making when the classifiers are trained by focusing on different regions of the input space.     

Linear combination of component results in information retrieval

In information retrieval, data fusion (also known as meta-search) has been investigated by many researchers. Previous investigation and experimentation demonstrate that the linear combination method is an effective data fusion method for combining multiple information retrieval results. One advantage is its flexibility, since different weights can be assigned to different component systems so as to obtain better fusion results. The key issue is how to assign good weights to all the component retrieval systems involved. Surprisingly, research in this field is limited and it is still an open question. In this paper, we use the multiple linear regression technique with estimated relevance scores and judged scores to obtain suitable weights. Although the multiple linear regression technique is not new, the way of using it in this paper has never been attempted before for the data fusion problem in information retrieval. Our experiments with five groups of runs submitted to TREC show that the linear combination method with such a weighting strategy steadily outperforms the best component system and other data fusion methods including CombSum, CombMNZ, PosFuse, MAPFuse, SegFuse, and the linear combination method with performance level/performance square weighting schemes by large margins.     

Alternative mixed integer linear programming models for identifying the most efficient decision making unit in data envelopment analysis

A mixed integer linear model for selecting the best decision making unit (DMU) in data envelopment analysis (DEA) has recently been proposed by Foroughi [Foroughi, A. A. (2011a). A new mixed integer linear model for selecting the best decision making units in data envelopment analysis. Computers and Industrial Engineering, 60(4), 550–554], which involves many unnecessary constraints and requires specifying an assurance region (AR) for input weights and output weights, respectively. Its selection of the best DMU is easy to be affected by outliers and may sometimes be incorrect. To avoid these drawbacks, this paper proposes three alternative mixed integer linear programming (MILP) models for identifying the most efficient DMU under different returns to scales, which contain only essential constraints and decision variables and are much simpler and more succinct than Foroughi’s. The proposed alternative MILP models can make full use of input and output information without the need of specifying any assurance regions for input and output weights to avoid zero weights, can make correct selections without being affected by outliers, and are of significant importance to the decision makers whose concerns are not DMU ranking, but the correct selection of the most efficient DMU. The potential applications of the proposed alternative MILP models and their effectiveness are illustrated with four numerical examples.     

Classifier hierarchy learning by means of genetic algorithms

Classifier combination falls in the so called data mining area. Its aim is to combine some paradigms from the supervised classification – sometimes with a previous non-supervised data division phase – in order to improve the individual accuracy of the component classifiers. Formation of classifier hierarchies is an alternative among the several methods of classifier combination. In this paper we present a novel method to find good hierarchies of classifiers for given databases. In this new proposal, a search is performed by means of genetic algorithms, returning the best individual according to the classification accuracy over the dataset, estimated through 10-fold cross-validation. Experiments have been carried out over 14 databases from the UCI repository, showing an improvement in the performance compared to the single classifiers. Moreover, similar or better results than other approaches, such as decision tree bagging and boosting, have been obtained.     

A group decision classifier with particle swarm optimization and decision tree for analyzing achievements in mathematics and science

Group decision making is a multi-criteria decision-making method applied in many fields. However, the use of group decision-making techniques in multi-class classification problems and rule generation is not explored widely. This investigation developed a group decision classifier with particle swarm optimization (PSO) and decision tree (GDCPSODT) for analyzing students’ mathematic and scientific achievements, which is a multi-class classification problem involving rule generation. The PSO technique is employed to determine weights of condition attributes; the decision tree is used to generate rules. To demonstrate the performance of the developed GDCPSODT model, other classifiers such as the Bayesian classifier, the k-nearest neighbor (KNN) classifier, the back propagation neural networks classifier with particle swarm optimization (BPNNPSO) and the radial basis function neural networks classifier with PSO (RBFNNPSO) are used to cope with the same data. Experimental results indicated the testing accuracy of GDCPSODT is higher than the other four classifiers. Furthermore, rules and some improvement directions of academic achievements are provided by the GDCPSODT model. Therefore, the GDCPSODT model is a feasible and promising alternative for analyzing student-related mathematic and scientific achievement data.     

基于异构网络特征与梯度提升决策树的协同药物预测

组合药物在复杂疾病特别是癌症的治疗中发挥越来越重要的作用。以组合药物靶标为初始节点在药物-蛋白质异构网络上执行重启型随机游走,将收敛后的概率分布作为药物组合的特征向量,训练梯度提升决策树模型来预测新的药物组合。在标准药物组合数据集的性能评估表明,该方法比其他七种典型分类器和传统的提升算法具有更好的性能,且基于异构网络的特征显著提升了各分类器的性能,AUC值从0.528提升至0.909。     

基于差分进化的ELM加权集成分类

集成分类通过将若干个弱分类器依据某种规则进行组合,能有效改善分类性能。在组合过程中,各个弱分类器对分类结果的重要程度往往不一样。极限学习机是最近提出的一个新的训练单隐层前馈神经网络的学习算法。以极限学习机为基分类器,提出了一个基于差分进化的极限学习机加权集成方法。提出的方法通过差分进化算法来优化集成方法中各个基分类器的权值。实验结果表明,该方法与基于简单投票集成方法和基于Adaboost集成方法相比,具有较高的分类准确性和较好的泛化能力。