On the design of an ECOC-Compliant Genetic Algorithm

Genetic Algorithms (GA) have been previously applied to Error-Correcting Output Codes (ECOC) in state-of-the-art works in order to find a suitable coding matrix. Nevertheless, none of the presented techniques directly take into account the properties of the ECOC matrix. As a result the considered search space is unnecessarily large. In this paper, a novel Genetic strategy to optimize the ECOC coding step is presented. This novel strategy redefines the usual crossover and mutation operators in order to take into account the theoretical properties of the ECOC framework. Thus, it reduces the search space and lets the algorithm to converge faster. In addition, a novel operator that is able to enlarge the code in a smart way is introduced. The novel methodology is tested on several UCI datasets and four challenging computer vision problems. Furthermore, the analysis of the results done in terms of performance, code length and number of Support Vectors shows that the optimization process is able to find very efficient codes, in terms of the trade-off between classification performance and the number of classifiers. Finally, classification performance per dichotomizer results shows that the novel proposal is able to obtain similar or even better results while defining a more compact number of dichotomies and SVs compared to state-of-the-art approaches.     

偏标记学习研究综述

在弱监督信息条件下进行学习已成为机器学习领域的热点研究课题。偏标记学习作为一类重要的弱监督机器学习框架,适于多种实际应用问题的学习建模。在该框架下,每个对象在输入空间由单个示例(属性向量)进行刻画,而在输出空间与一组候选标记相关联,其中仅有一个为其真实标记。本文将对偏标记学习的研究现状进行综述,首先给出该学习框架的定义以及与相关学习框架的区别与联系,然后重点介绍几种典型的偏标记学习算法以及作者在该方面的初步工作,最后对偏标记学习进一步的研究方向进行简要讨论。     

Thinned-ECOC ensemble based on sequential code shrinking

Error-correcting output coding (ECOC) is a strategy to create classifier ensembles which reduces a multi-class problem into some binary sub-problems. A key issue in designing any ECOC classifier refers to defining optimal codematrix having maximum discrimination power and minimum number of columns. This paper proposes a heuristic method for application-dependent design of optimal ECOC matrix based on a thinning algorithm. The main idea of the proposed Thinned-ECOC method is to successively remove some redundant and unnecessary columns of any initial codematrix based on a metric defined for each column. As a result, computational cost of the ensemble is reduced while preserving its accuracy. Proposed method has been validated using the UCI machine learning database and further applied to a couple of real-world pattern recognition problems (the face recognition and gene expression based cancer classification). Experimental results emphasize the robustness of Thinned-ECOC in comparison with existing state-of-the-art code generation methods.     

Soft computing methods applied to combination of one-class classifiers

The paper shows the possibilities of generalizing the two-class classification into multi-class classification by means of a fuzzy inference system. Fuzzy combiner harnesses the support values from classifiers to provide final response having no other restrictions on their structure. We compare proposed combination methods with ECOC and two variations of decision templates, based on Euclidean and symmetric distance. The effectiveness of the proposed combination method based on the fuzzy logic theory is also evaluated via computer experiments carried out on benchmark datasets.     

适用于非平衡数据的多关系多分类模型

针对多关系多分类的非平衡数据,提出一种分类模型。在预处理阶段,建立目标类纠错输出编码(ECOC)、目标关系与背景关系间的虚拟连接并完成属性聚集处理,进而划分训练集和验证集。在训练阶段,依据一对多划分思想,结合CrossMine算法构造多个子分类器,采用AUC法评估验证各子分类器。在验证阶段,比较目标类ECOC与各子分类器分类结果连接字的海明距离,选择最小海明距离的目标类为最终分类。经合成和真实数据的实验,验证了模型有效性及分类效果。     

基于免疫克隆选择的最优ECOC编码输出

纠错输出编码（Error Correcting Output Codes,ECOC）是解决模式识别领域多类分类问题的有效工具。在寻找最优编码输出的问题上,现有方法忽略了样本类别之间的相关性,导致学习效率和分类效果低下。为构造数据感知的编码矩阵,提出基于免疫克隆选择（Immune Clonal Selection Algorithm,ICSA）的最优纠错输出编码方法,将矩阵构造的多约束NP（Non-deterministic Polynomial,NP）难问题转换为优化搜索问题.首先基于分类精度和编码长度定义亲合度函数,然后结合样本知识改进变异交叉算子,根据约束性条件对矩阵进行搜索,从而快速有效地构建最优ECOC编码.实验表明该方法能够在提升多类分类精度的同时加快算法效率,而且输出的编码矩阵更加紧凑.     

Generic object recognition with regional statistical models and layer joint boosting

This paper presents novel regional statistical models for extracting object features, and an improved discriminative learning method, called as layer joint boosting, for generic multi-class object detection and categorization in cluttered scenes. Regional statistical properties on intensities are used to find sharing degrees among features in order to recognize generic objects efficiently. Based on boosting for multi-classification, the layer characteristic and two typical weights in sharing-code maps are taken into account to keep the maximum Hamming distance in categories, and heuristic search strategies are provided in the recognition process. Experimental results reveal that, compared with interest point detectors in representation and multi-boost in learning, joint layer boosting with statistical feature extraction can enhance the recognition rate consistently, with a similar detection rate.     

基于AdaBoost.ECOC的合成孔径雷达图像目标识别研究

为了提高合成孔径雷达图像目标识别系统的性能,提出了一种合成孔径雷达图像目标识别的新方法,结合纠错输出码对基本AdaBoost算法进行多类别推广,并将推广后的算法(AdaBoost.ECOC)应用于合成孔径雷达图像目标识别.用运动和静止目标获取与识别数据库中的三类地面军事目标进行识别实验,并将识别结果与其他识别方法进行比较.实验结果表明,提出的基于AdaBoost.ECOC的识别算法可以有效地应用于合成孔径雷达目标识别,并能显著提高目标识别系统的识别性能.     

一种搜索编码法及其在监督分类中的应用

纠错输出码作为监督分类领域中的一个新的研究方向,是提高分类器泛化能力的一种有效方法,但目前还没有通用的确定性编码方法.分析了现有纠错输出码的性质,提出一种搜索编码法,该方法通过对整数空间的顺序搜索,获得满足任意类别数目与最小汉明距离要求的输出码;然后探讨了基于搜索编码的监督分类技术.对简单贝叶斯与BP神经网络算法进行实验,结果表明,搜索编码法可作为一种通用的编码方法用于提高监督分类器的泛化能力.     

Effectiveness of error correcting output coding methods in ensemble and monolithic learning machines

Abstract Error Correcting Output Coding (ECOC) methods for multiclass classification present several open problems ranging from the trade-off between their error recovering capabilities and the learnability of the induced dichotomies to the selection of proper base learners and to the design of well-separated codes for a given multiclass problem. We experimentally analyse some of the main factors affecting the effectiveness of ECOC methods. We show that the architecture of ECOC learning machines influences the accuracy of the ECOC classifier, highlighting that ensembles of parallel and independent dichotomic Multi-Layer Perceptrons are well-suited to implement ECOC methods. We quantitatively evaluate the dependence among codeword bit errors using mutual information based measures, experimentally showing that a low dependence enhances the generalisation capabilities of ECOC. Moreover we show that the proper selection of the base learner and the decoding function of the reconstruction stage significantly affects the performance of the ECOC ensemble. The analysis of the relationships between the error recovering power, the accuracy of the base learners, and the dependence among codeword bits show that all these factors concur to the effectiveness of ECOC methods in a not straightforward way, very likely dependent on the distribution and complexity of the data.An erratum to this article can be found at