Accurate ensemble pruning with PL-bagging

Ensemble pruning deals with the selection of base learners prior to combination in order to improve prediction accuracy and efficiency. In the ensemble literature, it has been pointed out that in order for an ensemble classifier to achieve higher prediction accuracy, it is critical for the ensemble classifier to consist of accurate classifiers which at the same time diverse as much as possible. In this paper, a novel ensemble pruning method, called PL-bagging, is proposed. In order to attain the balance between diversity and accuracy of base learners, PL-bagging employs positive Lasso to assign weights to base learners in the combination step. Simulation studies and theoretical investigation showed that PL-bagging filters out redundant base learners while it assigns higher weights to more accurate base learners. Such improved weighting scheme of PL-bagging further results in higher classification accuracy and the improvement becomes even more significant as the ensemble size increases. The performance of PL-bagging was compared with state-of-the-art ensemble pruning methods for aggregation of bootstrapped base learners using 22 real and 4 synthetic datasets. The results indicate that PL-bagging significantly outperforms state-of-the-art ensemble pruning methods such as Boosting-based pruning and Trimmed bagging.     

Optimal ensemble construction via meta-evolutionary ensembles

In this paper, we propose a meta-evolutionary approach to improve on the performance of individual classifiers. In the proposed system, individual classifiers evolve, competing to correctly classify test points, and are given extra rewards for getting difficult points right. Ensembles consisting of multiple classifiers also compete for member classifiers, and are rewarded based on their predictive performance. In this way we aim to build small-sized optimal ensembles rather than form large-sized ensembles of individually-optimized classifiers. Experimental results on 15 data sets suggest that our algorithms can generate ensembles that are more effective than single classifiers and traditional ensemble methods.     

Remarks on different reviews of Chinese character recognition

This paper gives an introduction and remarks on two review papers for Chinese character recognition. One review is made by Chinese authors, another is from American scientists. They investigate Chinese character from different language environments; they do the research from different points of view. Thus, a more comprehensive view on Chinese character recognition, which is an important branch of pattern recognition, can be provided to the readers. Meantime, one article pays attention to online process, and other paper deals with offline recognition, which complement each other. The author is the Associate Editor-in-Chief of Frontiers of Computer Science in China     

Vote counting measures for ensemble classifiers

Various measures, such as Margin and Bias/Variance, have been proposed with the aim of gaining a better understanding of why Multiple Classifier Systems (MCS) perform as well as they do. While these measures provide different perspectives for MCS analysis, it is not clear how to use them for MCS design. In this paper a different measure based on a spectral representation is proposed for two-class problems. It incorporates terms representing positive and negative correlation of pairs of training patterns with respect to class labels. Experiments employing MLP base classifiers, in which parameters are fixed but systematically varied, demonstrate the sensitivity of the proposed measure to base classifier complexity.     

Surface and normal ensembles for surface reconstruction

The majority of the existing techniques for surface reconstruction and the closely related problem of normal reconstruction are deterministic. Their main advantages are the speed and, given a reasonably good initial input, the high quality of the reconstructed surfaces. Nevertheless, their deterministic nature may hinder them from effectively handling incomplete data with noise and outliers. An ensemble is a statistical technique which can improve the performance of deterministic algorithms by putting them into a statistics based probabilistic setting. In this paper, we study the suitability of ensembles in normal and surface reconstruction. We experimented with a widely used normal reconstruction technique [Hoppe H, DeRose T, Duchamp T, McDonald J, Stuetzle W. Surface reconstruction from unorganized points. Computer Graphics 1992;71-8] and Multi-level Partitions of Unity implicits for surface reconstruction [Ohtake Y, Belyaev A, Alexa M, Turk G, Seidel H-P. Multi-level partition of unity implicits. ACM Transactions on Graphics 2003;22(3):463-70], showing that normal and surface ensembles can successfully be combined to handle noisy point sets.     

Comparison among feature extraction methods for HIV-1 protease cleavage site prediction

Recently, several works have approached the HIV-1 protease specificity problem by applying a number of methods from the field of machine learning. However, it is still difficult for researchers to choose the best method due to the lack of an effective comparison. For the first time we have made an extensive study on methods for feature extraction for the problem of HIV-1 protease. We show that a fusion of classifiers trained in different feature spaces permits to obtain a drastically error reduction with respect to the performance of the state-of-the-art.     

基于特征选择的集成分类器抗噪性能分析

特征选择有助于增强集成分类器成员间的随机差异性,从而提高泛化精度。研究了随机子空间法（RandomSub-space）和旋转森林法（RotationForest）两种基于特征选择的集成分类器构造算法,分析讨论了两算法特征选择的方式与随机差异程度之间的关系。通过对UCI数据集引入噪声,比较两者在噪声环境下的分类精度。实验结果表明：当噪声增加及特征关联度下降时,基本学习算法及噪声程度对集成效果均有影响,当噪声增强到一定程度后。集成效果和单分类器的性能趋于一致。     

Random Forests with ensemble of feature spaces

Random Forests receive much attention from researchers because of their excellent performance. As Breiman suggested, the performance of Random Forests depends on the strength of the weak learners in the forests and the diversity among them. However, in the literature, many researchers only considered pre-processing of the data or post-processing of the Random Forests models. In this paper, we propose a new method to increase the diversity of each tree in the forests and thereby improve the overall accuracy. During the training process of each individual tree in the forest, different rotation spaces are concatenated into a higher space at the root node. Then the best split is exhaustively searched within this higher space. The location where the best split lies decides which rotation method to be used for all subsequent nodes. The performance of the proposed method here is evaluated on 42 benchmark data sets from various research fields and compared with the standard Random Forests. The results show that the proposed method improves the performance of the Random Forests in most cases.     

Ensemble clustering by means of clustering embedding in vector spaces

Ensemble clustering is a recently evolving research direction in cluster analysis and has found several different application domains. In this work the complex ensemble clustering problem is reduced to the well-known Euclidean median problem by clustering embedding in vector spaces. The Euclidean median problem is solved by the Weiszfeld algorithm and an inverse transformation maps the Euclidean median back into the clustering domain. In the experiment study different evaluation strategies are considered. The proposed embedding strategy is compared to several state-of-art ensemble clustering algorithms and demonstrates superior performance.     

面向目标的带先验概率的AdaBoost算法

针对集成学习算法研究中多个分类器的最佳组合问题,改进了传统的AdaBoost集成学习算法.用于组合的各个分类器通常是基于样本集通过一定的训练得到,样本集中不同类目标的比率可以反映分类目标的先验概率.使用该参数给出了新的组合参数和投票表决阈值计算公式,巧妙的利用样本权值并将其加入到样本属性上进行训练学习,采用新的策略来选择基分类器,给出了面向目标的带先验概率的AdaBoost算法(GWPP AdaBoost算法)和分类器的最佳组合.依据UCI实验数据对传统的AdaBoost算法、Bagging算法、GWPP AdaBoost算法的错误率和性能进行了比较分析,验证了GWPP AdaBoost的有效性.