一种支持向量机集成的核选择方法

核选择问题是支持向量机（Support Vector Machine,SVM）建模中的一个关键问题,虽然支持向量机具有良好的泛化性能,但其性能受核函数的影响比较明显,而对于一个给定问题,选择合适的核函数及参数通常很困难。提出一种基于SVM集成的核选择方法,利用不同的核函数构造子SVM学习器,然后对子学习器的预测结果集成。提出的核选择方法将SVM集成学习与核选择同时进行,不仅避免了单个SVM的核选择对泛化能力的影响,而且可以获得良好的泛化能力。在UCI标准数据集上的结果说明了提出的方法的有效性。     

选择性支持向量机集成算法

为有效提升支持向量机泛化性能,提出了基于差分进化算法和负相关学习的选择性支持向量机集成。通过bootstrap技术产生并训练得到多个独立子SVM,基于负相关学习理论构造适应度函数,既提高子SVM的泛化性能,又增大其之间差异度。利用差分进化算法计算各子SVM在加权平均中的最优权重,选择权值大于一定阈值的部分SVM进行加权集成。实验结果表明,该算法是一种有效的集成方法,能进一步提高SVM的泛化性能。     

基于加速遗传算法的选择性支持向量机集成*

为有效提升支持向量机的泛化性能,提出基于加速遗传算法的选择性支持向量机集成。通过Bootstrap技术产生并训练得到多个独立子SVM,基于负相关学习理论构造适应度函数,提高子SVM的泛化性能,并增大其之间差异度。利用加速遗传算法计算各子SVM在加权平均中的最优权重,然后选择权值大于一定阈值的部分SVM进行加权集成。实验结果表明,该算法是一种有效的集成方法,能进一步提高SVM的集成效率和泛化性能。     

一种多层次选择性集成学习算法

选择性集成学习是为解决同一个问题而训练多个基分类器,并依据某种规则选取部分基分类器的结果进行整合的学习算法。通过选择性集成可以获得比单个学习器和全部集成学习更好的学习效果,可以显著地提高学习系统的泛化性能。提出了一种多层次选择性集成学习算法Ada_ens。试验结果表明,Ada_ens具有更好的学习效果和泛化性能。     

集成SVM的回归预测及其遥感应用

为了提高单支持向量机(SvM)回归模型的泛化能力,引入遗传算法(GA)用以搜索SVM的"低偏差区域",给出了一种基于GA的SVM异构集成方法.用此方法对十个典型的数据集进行回归预测,并与单SVM回归结果和Bagging集成回归结果进行了比较,表明这种异构集成方法有较好的泛化能力.将这种方法应用于感兴趣的四个渭河陕西段水质参数的遥感反演,取得了更为精确的预测结果.实验表明,对小样本情况,基于GA的SVM异构集成方法能在付出合理时间花销的条件下,使单SVM的泛化能力得到有效提升.     

基于半监督回归的选择性集成算法

为了提高小数据量的有标记样本问题中学习器的性能,结合半监督学习和选择性集成学习,提出了基于半监督回归的选择性集成算法SSRES。算法基于半监督学习的基本思想,同时使用有标记样本和未标记样本训练学习器从而减少对有标记样本的需求,使用选择性集成算法GRES对不同学习器进行适当的选择,并将选择的结果结合提高学习器的泛化能力。实验结果表明,在小数据量的有标记样本问题中,该算法能够有效地提高学习器的性能。     

基于FP-Tree 的快速选择性集成算法

选择性集成通过选择部分基分类器参与集成,从而提高集成分类器的泛化能力,降低预测开销.但已有的选择性集成算法普遍耗时较长,将数据挖掘的技术应用于选择性集成,提出一种基于FP-Tree(frequent pattern tree)的快速选择性集成算法:CPM-EP(coverage based pattern mining for ensemble pruning).该算法将基分类器对校验样本集的分类结果组织成一个事务数据库,从而使选择性集成问题可转化为对事务数据集的处理问题.针对所有可能的集成分类器大小,CPM-EP算法首先得到一个精简的事务数据库,并创建一棵FP-Tree树保存其内容;然后,基于该FP-Tree获得相应大小的集成分类器.在获得的所有集成分类器中,对校验样本集预测精度最高的集成分类器即为算法的输出.实验结果表明,CPM-EP算法以很低的计算开销获得优越的泛化能力,其分类器选择时间约为GASEN的1/19以及Forward-Selection的1/8,其泛化能力显著优于参与比较的其他方法,而且产生的集成分类器具有较少的基分类器.     

一种基于Bagging-SVM的智能传感器集成学习方法

集成多个传感器的智能片上系统( SoC)在物联网得到了广泛的应用.在融合多个传感器数据的分类算法方面,传统的支持向量机( SVM)单分类器不能直接对传感器数据流进行小样本增量学习.针对上述问题,提出一种基于Bagging-SVM的集成增量算法,该算法通过在增量数据中采用Bootstrap方式抽取训练集,构造能够反映新信息变化的集成分类器,然后将新老分类器集成,实现集成增量学习.实验结果表明:该算法相比SVM单分类器能够有效降低分类误差,提高分类准确率,且具有较好的泛化能力,可以满足当下智能传感器系统基于小样本数据流的在线学习需求.     

选择性集成学习算法的研究

通过选择性集成可以获得比单个学习器和全部集成学习更好的学习效果,可以显著地提高学习系统的泛化性能。文中提出一种多层次选择性集成学习算法,即在基分类器中通过多次按权重进行部分选择,形成多个集成分类器,对形成的集成分类器进行再集成,最后通过对个集成分类器多数投票的方式决定算法的输出。针对决策树与神经网络模型在20个标准数据集对集成学习算法Ada—ens进行了实验研究,试验证明基于数据的集成学习算法的性能优于基于特征集的集成学习算法的性能,有更好的分类准确率和泛化性能。     

选择性集成学习算法的研究

通过选择性集成可以获得比单个学习器和全部集成学习更好的学习效果,可以显著地提高学习系统的泛化性能。文中提出一种多层次选择性集成学习算法,即在基分类器中通过多次按权重进行部分选择,形成多个集成分类器,对形成的集成分类器进行再集成,最后通过对个集成分类器多数投票的方式决定算法的输出。针对决策树与神经网络模型在20个标准数据集对集成学习算法Ada—ens进行了实验研究,试验证明基于数据的集成学习算法的性能优于基于特征集的集成学习算法的性能,有更好的分类准确率和泛化性能。     

基于离散化方法的支持向量机集成研究

基于数据离散化方法,提出一种新的支持向量机集成算法,该算法采用粗糙集和布尔推理离散化方法构造有差异的基分类器.并引入一致度指标控制离散化过程,可进一步提高集成学习的分类性能.实验结果表明,该算法不仅具有明显优于单一支持向量机的分类性能.而且能取得比传统集成学习算法Bagging和Adaboost更高的分类正确率.     

基于证据理论的多类分类支持向量机集成

针对多类分类问题,研究支持向量机集成中的分类器组合架构与方法.分析已有的多类级和两类级支持向量机集成架构的不足后,提出两层的集成架构.在此基础上,研究基于证据理论的支持向量机度量层输出信息融合方法,针对一对多与一对一两种多类扩展策略,分别定义基本概率分配函数,并根据证据冲突程度采用不同的证据组合规则.在一对多策略下,采用经典的Dempster规则;在一对一策略下则提出一条新的规则,以组合冲突严重的证据.实验表明,两层架构优于多类级架构,证据理论方法能有效地利用两类支持向量机的度量层输出信息,取得了满意的结果.     

Improving the prediction of petroleum reservoir characterization with a stacked generalization ensemble model of support vector machines

The ensemble learning paradigm has proved to be relevant to solving most challenging industrial problems. Despite its successful application especially in the Bioinformatics, the petroleum industry has not benefited enough from the promises of this machine learning technology. The petroleum industry, with its persistent quest for high-performance predictive models, is in great need of this new learning methodology. A marginal improvement in the prediction indices of petroleum reservoir properties could have huge positive impact on the success of exploration, drilling and the overall reservoir management portfolio. Support vector machines (SVM) is one of the promising machine learning tools that have performed excellently well in most prediction problems. However, its performance is a function of the prudent choice of its tuning parameters most especially the regularization parameter, C. Reports have shown that this parameter has significant impact on the performance of SVM. Understandably, no specific value has been recommended for it. This paper proposes a stacked generalization ensemble model of SVM that incorporates different expert opinions on the optimal values of this parameter in the prediction of porosity and permeability of petroleum reservoirs using datasets from diverse geological formations. The performance of the proposed SVM ensemble was compared to that of conventional SVM technique, another SVM implemented with the bagging method, and Random Forest technique. The results showed that the proposed ensemble model, in most cases, outperformed the others with the highest correlation coefficient, and the lowest mean and absolute errors. The study indicated that there is a great potential for ensemble learning in petroleum reservoir characterization to improve the accuracy of reservoir properties predictions for more successful explorations and increased production of petroleum resources. The results also confirmed that ensemble models perform better than the conventional SVM implementation.     

Feature-subspace aggregating: ensembles for stable and unstable learners

This paper introduces a new ensemble approach, Feature-Subspace Aggregating (Feating), which builds local models instead of global models. Feating is a generic ensemble approach that can enhance the predictive performance of both stable and unstable learners. In contrast, most existing ensemble approaches can improve the predictive performance of unstable learners only. Our analysis shows that the new approach reduces the execution time to generate a model in an ensemble through an increased level of localisation in Feating. Our empirical evaluation shows that Feating performs significantly better than Boosting, Random Subspace and Bagging in terms of predictive accuracy, when a stable learner SVM is used as the base learner. The speed up achieved by Feating makes feasible SVM ensembles that would otherwise be infeasible for large data sets. When SVM is the preferred base learner, we show that Feating SVM performs better than Boosting decision trees and Random Forests. We further demonstrate that Feating also substantially reduces the error of another stable learner, k-nearest neighbour, and an unstable learner, decision tree.     

Developing an SVM-based ensemble learning system for customer risk identification collaborating with customer relationship management

In this study, we propose a support vector machine (SVM)-based ensemble learning system for customer relationship management (CRM) to help enterprise managers effectively manage customer risks from the risk aversion perspective. This system differs from the classical CRM for retaining and targeting profitable customers; the main focus of the proposed SVM-based ensemble learning system is to identify high-risk customers in CRM for avoiding possible loss. To build an effective SVM-based ensemble learning system, the effects of ensemble members’ diversity, ensemble member selection and different ensemble strategies on the performance of the proposed SVM-based ensemble learning system are each investigated in a practical CRM case. Through experimental analysis, we find that the Bayesian-based SVM ensemble learning system with diverse components and choose from space selection strategy show the best performance over various testing samples.     

基于支持向量机集成的故障诊断

为提高故障诊断的准确性,提出了一种基于遗传算法的支持向量机集成学习方法,定义了相应的遗传操作算子,并探讨了集成下的分类器的构造策略。对汽轮机转子不平衡故障诊断的仿真实验结果表明,集成学习方法的性能通常优于单个支持向量机,而所提方法性能则优于Bagging与Boosting等传统集成学习方法,获得的集成所包括的分类器数目更少,而且结合多种分类器构造策略可提高分类器的多样性。该方法能容易地推广到神经网络、决策树等其他学习算法。     

几种机器学习方法在人脸识别中的性能比较

BP神经网络、RBF神经网络、支持向量机（SVM）和集成学习是目前应用最为广泛的四种机器学习方法。将这四种常用的机器学习方法分别应用于人脸识别,并利用ORL人脸图像库对各学习方法性能进行了测试和评估。测试结果表明SVM和集成学习在实验中取得了较好的性能,最适合用于人脸识别中特征分类器。     

基于单核苷酸统计和支持向量机集成的人类基因启动子识别

为高效地判别人类基因启动子,提出了一种基于单核苷酸统计和支持向量机集成的人类基因启动子识别算法。首先通过基因单核苷酸统计,从而将一个基因数据集分为C偏好和G偏好两个子集;然后分别对这两个子集提取DNA刚性特征、词频统计特征和CpG岛特征;最后采用多个支持向量机(SVM)集成的方式来学习这三种特征,并讨论了三种集成方式,包括单层SVM集成、双层SVM集成和级联SVM集成。实验结果表明所提算法能够提高人类基因启动子识别的敏感性和特异性,其中双层SVM集成的敏感性达到79.51%,且级联SVM集成的特异性高达84.58%。     

集成学习和主动学习相结合的个性化垃圾邮件过滤

本文提出了一种个性化垃圾邮件过滤方法,它能够根据用户反馈自动学习出用户兴趣,并随时间的推移自动适应用户兴趣的变化。该方法首先抽取邮件的语言特征和行为特征构建多个基于规则的单独过滤器,然后采用SVM集成学习方法组合这些单独过滤器的结果。为了提高学习速度、减少用户提供反馈的数量,本文采用了主动学习方法挑选更加富含知识的邮件请求用户给出反馈。实验结果表明:集成学习和主动学习相结合的个性化过滤方法在个性化程度、分类准确率、过滤速度以及自动学习能力等方面具有更好的性能。     

Aggregative model-based classifier ensemble for improving land-use/cover classification of Landsat TM Images

This article proposes a new approach to improve the classification performance of remotely sensed images with an aggregative model based on classifier ensemble (AMCE). AMCE is a multi-classifier system with two procedures, namely ensemble learning and predictions combination. Two ensemble algorithms (Bagging and AdaBoost.M1) were used in the ensemble learning process to stabilize and improve the performance of single classifiers (i.e. maximum likelihood classifier, minimum distance classifier, back propagation neural network, classification and regression tree, and support vector machine (SVM)). Prediction results from single classifiers were integrated according to a diversity measurement with an averaged double-fault indicator and different combination strategies (i.e. weighted vote, Bayesian product, logarithmic consensus, and behaviour knowledge space). The suitability of the AMCE model was examined using a Landsat Thematic Mapper (TM) image of Dongguan city (Guangdong, China), acquired on 2 January 2009. Experimental results show that the proposed model was significantly better than the most accurate single classification (i.e. SVM) in terms of classification accuracy (i.e. from 88.83% to 92.45%) and kappa coefficient (i.e. from 0.8624 to 0.9088). A stepwise comparison illustrates that both ensemble learning and predictions combination with the AMCE model improved classification.