采用二重扰动机制的支持向量机的集成训练算法

为了有效提升支持向量机的泛化性能,提出两种集成算法对其进行训练.首先分析了扰动输入特征空间和扰动模型参数两种方式对于增大成员分类器之间差异性的作用;然后提出两种基于二重扰动机制的集成训练算法.其共同特点是,同时扰动输入特征空间和模型参数以产生成员分类器,并利用多数投票法对它们进行组合.实验结果表明,因为同时缩减了误差的偏差部分和方差部分,所以两种算法均能显著提升支持向量机的泛化性能.

 

     

OCSVM集成模型解决支持向量机的大规模训练问题

该文针对集成方法实现支持向量机大规模训练的相关问题进行了深入研究,提出了一种称为"DD-Boosting"的成员分类器产生算法,能够在大规模数据集情况下利用类似Boosting技术产生稳定、高泛化性能的成员分类器。在此基础上,推导出基于OCSVM的分类器集成模型,实验仿真表明,该集成模型能够获得比主投票方法更好的泛化性能,且通过调整正则参数避免了训练过拟合问题。     

基于双重扰动的选择性支持向量机集成

为了进一步提升支持向量机泛化性能,提出一种基于双重扰动的选择性支持向量机集成算法。利用Boosting方法对训练集进行扰动基础上,采用基于相对核的粗糙集相对约简与重采样技术相结合的动态约简算法进行特征扰动以生成个体成员,然后基于负相关学习理论构造遗传个体适应度函数,利用加速遗传算法选择权重大于阈值的最优个体进行加权集成。实验结果表明,该算法具有较高的泛化性能和较低的时、空复杂性,是一种高效的集成方法。     

基于动态粗糙集约简的选择性支持向量机集成

支持向量机集成是提高支持向量机泛化性能的有效手段,个体支持向量机的泛化能力及其之间的差异性是影响集成性能的关键因素。为了进一步提升支持向量机整体泛化性能,提出利用动态粗糙集的选择性支持向量机集成算法。首先在利用Boosting算法对样本进行扰动基础上,采用遗传算法改进的粗糙集与重采样技术相结合的动态约简算法进行特征扰动,获得稳定、泛化能力较强的属性约简集,继而生成差异性较大的个体学习器;然后利用模糊核聚类根据个体学习器在验证集上的泛化误差来选择最优个体;并用支持向量机算法对最优个体进行非线性集成。通过在UCI数据集进行仿真,结果表明算法能明显提高支持向量机的泛化性能,具有较低的时、空复杂性,是一种高效、稳定的集成方法。     

基于约束投影的支持向量机选择性集成

提出两种基于约束投影的支持向量机选择性集成算法。首先利用随机选取的must-link和cannot-link成对约束集确定投影矩阵,将原始训练样本投影到不同的低维空间训练一组基分类器;然后,分别采用遗传优化和最小化偏离度误差两种选择性集成技术对基分类器进行组合。基于UCI数据的实验表明,提出的两种集成算法均能有效提高支持向量机的泛化性能,显著优于Bagging,Boosting,特征Bagging及LoBag等集成算法。     

基于粗集理论的选择性支持向量机集成

集成分类器的性能很大程度决定于各成员分类器的构造和对各成员分类器的组合方法。提出一种基于粗集理论的选择性支持向量机集成算法,该算法首先利用粗集技术产生一个属性约简集合,然后以各约简集为样本属性空间构造各成员分类器,其次通过对各成员分类器精度与差异度的计算,选择既满足个体的精度要求,又满足个体差异性要求的成员分类器进行集成。最后通过对UCI上一组实验数据的测试,证实该方法能够有效提高支持向量机的推广性能。     

基于训练集平行分割的集成学习算法研究

针对大规模数据分类中训练集分解导致的分类器泛化能力下降问题,提出基于训练集平行分割的集成学习算法.它采用多簇平行超平面对训练集实施多次划分,在各次划分的训练集上采用一种模块化支持向量机网络算法训练基分类器.测试时采用多数投票法对各个基分类器的输出进行集成.在3个大规模问题上的实验表明:在不增加训练时间和测试时间的条件下,集成学习在保持分类器偏置基本不变的同时有效减少了分类器的方差,从而有效降低了由于训练集分割导致的分类器泛化能力下降.     

基于改进PSO的SVM文本分类研究

提出基于改进PSO优化支持向量机的文本分类方法,首先采用向量空间模型对文本特征进行提取,使用互信息对文本特征进行降维,然后提出改进PSO算法,该算法可实现对SVM参数的精确、稳定、快速优化选择,对支持向量机进行训练,使用训练后的分类器对新的文本进行分类,实验结果表明该方法具有良好的分类性能。     

选择性支持向量机集成算法

为有效提升支持向量机泛化性能,提出了基于差分进化算法和负相关学习的选择性支持向量机集成。通过bootstrap技术产生并训练得到多个独立子SVM,基于负相关学习理论构造适应度函数,既提高子SVM的泛化性能,又增大其之间差异度。利用差分进化算法计算各子SVM在加权平均中的最优权重,选择权值大于一定阈值的部分SVM进行加权集成。实验结果表明,该算法是一种有效的集成方法,能进一步提高SVM的泛化性能。     

基于遗传算法的SVM带权特征和模型参数优化

建立在统计学习理论和结构风险最小原则上的支持向量机(SVM)在理论上保证了模型的最大泛化能力,因此将支持向量机理论应用于入侵检测领域可以获得很好的效果.但是在应用中也存在如何对网络数据进行特征编码和选择适当的支持向量机模型参数的问题.在分析了特征编码和模型参数对分类器识别精度的影响基础上,提出用遗传算法建立支持向量机带权特征和分类器模型参数的自适应优化算法,并在网络入侵检测中成功的运用算法.最后,使用KDD CUP 1999数据进行的仿真实验表明了算法的正确有效性.     

Input Decimated Ensemble based on Neighborhood Preserving Embedding for spectrogram classification

In this work a novel technique for building ensembles of classifiers for spectrogram classification is presented. We propose a simple approach for classifying signals from a large database of plant echoes, these echoes are highly complex stochastic signals, anyway their spectrograms contain enough information for extracting a good set of features for training the proposed ensemble of classifiers.The proposed ensemble of classifiers is a novel modified version of a recent feature transform based ensemble method: the Input Decimated Ensemble. In the proposed variant different subsets of randomly extracted training patterns are used to create a set of different Neighborhood Preserving Embedding subspace projections. These feature transformations are applied to the whole dataset and a set of decision trees are trained using these transformed spaces. Finally, the scores of this set of classifiers are combined by sum rule.Experiments carried out on a yet proposed dataset show the superiority of this method with respect to other approaches. The proposed approach outperforms the yet proposed, for the tested dataset, combination of principal component analysis and support vector machine (SVM). Moreover, we show that the fusion between the proposed ensemble and the system based on SVM outperforms both the stand-alone methods.     

基于SVM动态集成的高光谱遥感图像分类

在Bagging支持向量机(SVM)的基础上,将动态分类器集选择技术用于SVM的集成学习,研究了SVM动态集成在高光谱遥感图像分类中的应用。结合高光谱数据特性,通过随机选取特征子空间和反馈学习改进了Bagging SVM方法;通过引进加性复合距离改善了K近邻局部空间的计算方法;通过将错分的训练样本添加到验证集增强了验证集样本的代表性。实验结果表明,与单个优化的SVM和其他常见的SVM集成方法相比,改进后的SVM动态集成分类精度最高,能有效地提高高光谱遥感图像的分类精度。     

多模式扰动模型动态加权SVM集成研究

针对集成学习中bootstrap方法不能产生具有较大差异性的成员分类器,提出基于多模式扰动模型动态加权SVM集成方法。该方法在训练样本中使用bootstrap采样产生扰动,在输入特征中使用PCA特征滤波子空间法产生扰动,用自动模型选择法来动态扰动每个成员分类器的参数,用分类精度对成员分类器加权集成扰动输出。实验结果表明该方法比常用的bootstrap集成方法具有更好的集成效果。     

基于单核苷酸统计和支持向量机集成的人类基因启动子识别

为高效地判别人类基因启动子,提出了一种基于单核苷酸统计和支持向量机集成的人类基因启动子识别算法。首先通过基因单核苷酸统计,从而将一个基因数据集分为C偏好和G偏好两个子集;然后分别对这两个子集提取DNA刚性特征、词频统计特征和CpG岛特征;最后采用多个支持向量机(SVM)集成的方式来学习这三种特征,并讨论了三种集成方式,包括单层SVM集成、双层SVM集成和级联SVM集成。实验结果表明所提算法能够提高人类基因启动子识别的敏感性和特异性,其中双层SVM集成的敏感性达到79.51%,且级联SVM集成的特异性高达84.58%。     

优化的支持向量机集成分类器在非平衡数据集分类中的应用

传统的分类算法大都建立在平衡数据集的基础上,当样本数据不平衡时,这些学习算法的性能往往会明显下降.对于非平衡数据分类问题,提出了一种优化的支持向量机(SVM)集成分类器模型,采用KSMOTE和Bootstrap对非平衡数据进行预处理,生成相应的SVM模型并用复合形算法优化模型参数,最后利用优化的参数并行生成SVM集成分类器模型,采用投票机制得到分类结果.对5组UCI标准数据集进行实验,结果表明采用优化的SVM集成分类器模型较SVM模型、优化的SVM模型等分类精度有了明显的提升,同时验证了不同的bootNum取值对分类器性能效果的影响.     

Ensemble classification based on generalized additive models

Generalized additive models (GAMs) are a generalization of generalized linear models (GLMs) and constitute a powerful technique which has successfully proven its ability to capture nonlinear relationships between explanatory variables and a response variable in many domains. In this paper, GAMs are proposed as base classifiers for ensemble learning. Three alternative ensemble strategies for binary classification using GAMs as base classifiers are proposed: (i) GAMbag based on Bagging, (ii) GAMrsm based on the Random Subspace Method (RSM), and (iii) GAMens as a combination of both. In an experimental validation performed on 12 data sets from the UCI repository, the proposed algorithms are benchmarked to a single GAM and to decision tree based ensemble classifiers (i.e. RSM, Bagging, Random Forest, and the recently proposed Rotation Forest). From the results a number of conclusions can be drawn. Firstly, the use of an ensemble of GAMs instead of a single GAM always leads to improved prediction performance. Secondly, GAMrsm and GAMens perform comparably, while both versions outperform GAMbag. Finally, the value of using GAMs as base classifiers in an ensemble instead of standard decision trees is demonstrated. GAMbag demonstrates performance comparable to ordinary Bagging. Moreover, GAMrsm and GAMens outperform RSM and Bagging, while these two GAM ensemble variations perform comparably to Random Forest and Rotation Forest. Sensitivity analyses are included for the number of member classifiers in the ensemble, the number of variables included in a random feature subspace and the number of degrees of freedom for GAM spline estimation.     

Financial distress prediction using support vector machines: Ensemble vs. individual

Financial distress prediction (FDP) is of great importance to both inner and outside parts of companies. Though lots of literatures have given comprehensive analysis on single classifier FDP method, ensemble method for FDP just emerged in recent years and needs to be further studied. Support vector machine (SVM) shows promising performance in FDP when compared with other single classifier methods. The contribution of this paper is to propose a new FDP method based on SVM ensemble, whose candidate single classifiers are trained by SVM algorithms with different kernel functions on different feature subsets of one initial dataset. SVM kernels such as linear, polynomial, RBF and sigmoid, and the filter feature selection/extraction methods of stepwise multi discriminant analysis (MDA), stepwise logistic regression (logit), and principal component analysis (PCA) are applied. The algorithm for selecting SVM ensemble's base classifiers from candidate ones is designed by considering both individual performance and diversity analysis. Weighted majority voting based on base classifiers’ cross validation accuracy on training dataset is used as the combination mechanism. Experimental results indicate that SVM ensemble is significantly superior to individual SVM classifier when the number of base classifiers in SVM ensemble is properly set. Besides, it also shows that RBF SVM based on features selected by stepwise MDA is a good choice for FDP when individual SVM classifier is applied.     

一种基于局部随机子空间的分类集成算法

分类器集成学习是当前机器学习研究领域的热点之一。然而,经典的采用完全随机的方法,对高维数据而言,难以保证子分类器的性能。 为此,文中提出一种基于局部随机子空间的分类集成算法,该算法首先采用特征选择方法得到一个有效的特征序列,进而将特征序列划分为几个区段并依据在各区段的采样比例进行随机采样,以此来改进子分类器性能和子分类器的多样性。在5个UCI数据集和5个基因数据集上进行实验,实验结果表明,文中方法优于单个分类器的分类性能,且在多数情况下优于经典的分类集成方法。     

On Taxonomy and Evaluation of Feature Selection‐Based Learning Classifier System Ensemble Approaches for Data Mining Problems

Ensemble methods aim at combining multiple learning machines to improve the efficacy in a learning task in terms of prediction accuracy, scalability, and other measures. These methods have been applied to evolutionary machine learning techniques including learning classifier systems (LCSs). In this article, we first propose a conceptual framework that allows us to appropriately categorize ensemble‐based methods for fair comparison and highlights the gaps in the corresponding literature. The framework is generic and consists of three sequential stages: a pre‐gate stage concerned with data preparation; the member stage to account for the types of learning machines used to build the ensemble; and a post‐gate stage concerned with the methods to combine ensemble output. A taxonomy of LCSs‐based ensembles is then presented using this framework. The article then focuses on comparing LCS ensembles that use feature selection in the pre‐gate stage. An evaluation methodology is proposed to systematically analyze the performance of these methods. Specifically, random feature sampling and rough set feature selection‐based LCS ensemble methods are compared. Experimental results show that the rough set‐based approach performs significantly better than the random subspace method in terms of classification accuracy in problems with high numbers of irrelevant features. The performance of the two approaches are comparable in problems with high numbers of redundant features.