集成学习在脑机接口分类算法中的研究

提出了一种基于独立分量分析的支持向量机集成学习算法,用于脑机接口中P300字符识别.首先由P300信号分解出独立分量,基于Bagging算法送入支持向量机基分类器进行集成学习,通过平均的方法获得对应类别概率进行分类决策.数据来源于P300字符拼写实验,不同导联和不同序列的分类结果表明,该分类算法学习效率和分类精度高,全...     

Corrective classification: Learning from data imperfections with aggressive and diverse classifier ensembling

Learning from imperfect (noisy) information sources is a challenging and reality issue for many data mining applications. Common practices include data quality enhancement by applying data preprocessing techniques or employing robust learning algorithms to avoid developing overly complicated structures that overfit the noise. The essential goal is to reduce noise impact and eventually enhance the learners built from noise-corrupted data. In this paper, we propose a novel corrective classification (C2) design, which incorporates data cleansing, error correction, Bootstrap sampling and classifier ensembling for effective learning from noisy data sources. C2 differs from existing classifier ensembling or robust learning algorithms in two aspects. On one hand, a set of diverse base learners of C2 constituting the ensemble are constructed via a Bootstrap sampling process; on the other hand, C2 further improves each base learner by unifying error detection, correction and data cleansing to reduce noise impact. Being corrective, the classifier ensemble is built from data preprocessed/corrected by the data cleansing and correcting modules. Experimental comparisons demonstrate that C2 is not only more accurate than the learner built from original noisy sources, but also more reliable than Bagging [4] or aggressive classifier ensemble (ACE) [56], which are two degenerated components/variants of C2. The comparisons also indicate that C2 is more stable than Boosting and DECORATE, which are two state-of-the-art ensembling methods. For real-world imperfect information sources (i.e. noisy training and/or test data), C2 is able to deliver more accurate and reliable prediction models than its other peers can offer.     

Predicting adequacy of vancomycin regimens: A learning-based classification approach to improving clinical decision making

Clinicians' drug regimen decision making is critical, particularly when involving high-alert medications. In this study, we use decision-tree induction C4.5 and a backpropagation neural network to construct decision support systems for predicting the regimen adequacy of vancomycin, a glycopeptide antimicrobial antibiotic effective for Gram-positive bacterial infections. We comparatively evaluate the respective systems using a total of 987 clinical vancomycin cases collected from a major tertiary medical center in southern Taiwan. We supplement each system using Bagging and then examine the predictive power of the extended system. Overall, our evaluation results show the overall accuracy of the decision support system based on C4.5 or the neural network to be significantly higher than that of the benchmark one-compartment pharmacokinetic model. Use of Bagging can considerably improve the effectiveness of each system across different performance measures, particularly for cases of decision classes in which the base systems (i.e., without Bagging) perform modestly. Taken together, our evaluation results seem to favor the use of Bagging to enhance the performance of decision support systems constructed using decision-tree induction C4.5.     

Trimmed bagging

Bagging has been found to be successful in increasing the predictive performance of unstable classifiers. Bagging draws bootstrap samples from the training sample, applies the classifier to each bootstrap sample, and then averages over all obtained classification rules. The idea of trimmed bagging is to exclude the bootstrapped classification rules that yield the highest error rates, as estimated by the out-of-bag error rate, and to aggregate over the remaining ones. In this note we explore the potential benefits of trimmed bagging. On the basis of numerical experiments, we conclude that trimmed bagging performs comparably to standard bagging when applied to unstable classifiers as decision trees, but yields better results when applied to more stable base classifiers, like support vector machines.     

集成分类对比:Bagging NB & Boosting NB

Bagging和Boosting是两种重要的投票分类算法,前者并行生成多个分类器,后者通过调整样本权重,串行生成多个分类器.将Bagging与Boosting算法与朴素贝叶斯算法相集成,构建了Bagging NB和AdaBoosting NB算法.以UCI数据集为基础,进行实验对比,结果表明,Bagging NB算法较为稳定,可以产生优于NB算法的分类结果,而Boosting算法受到数据分布中的奇异值影响较大,部分数据集上与NB算法的基础效果较差.     

基于Bagging支持向量机集成的入侵检测研究

对大数据集来说，支持向量机的时空耗费非常大，本文采用bagging技术对支持向量机进行集成。首先用bootstrap技术对训练样本集进行可重复采样，使所得到的新子样本集有较大差异，然后用多个支持向量机对各子样本集进行学习，并将学习后的结果用多数投票法集成最终的结论。实验表明，支持向量机集成对入侵检测数据有比单个支持向量机更好的分类性能。     

一种基于投票法融合的ASTER遥感影像水体提取方法

遥感影像在水资源调查和洪涝灾害监测中发挥着重要作用,但从遥感影像中提取水体通常面临着阴影和狭小水体漏提等难题。针对单一方法在水体提取中的局限性,引入分类器集成的思想,提出一种基于投票法融合的水体提取方法,首先利用Bagging、Random Forests和神经网络(NN)分类器对遥感影像进行分类,然后采用多数投票法从决策层融合3个分类结果,得到研究区水体提取结果。试验结果表明,该方法能够有效去除阴影且能较好地识别狭小水体,具有良好的应用效果。     

Switching class labels to generate classification ensembles

Ensembles that combine the decisions of classifiers generated by using perturbed versions of the training set where the classes of the training examples are randomly switched can produce a significant error reduction, provided that large numbers of units and high class switching rates are used. The classifiers generated by this procedure have statistically uncorrelated errors in the training set. Hence, the ensembles they form exhibit a similar dependence of the training error on ensemble size, independently of the classification problem. In particular, for binary classification problems, the classification performance of the ensemble on the training data can be analysed in terms of a Bernoulli process. Experiments on several UCI datasets demonstrate the improvements in classification accuracy that can be obtained using these class-switching ensembles.