A fast ensemble pruning algorithm based on pattern mining process

Ensemble pruning deals with the reduction of base classifiers prior to combination in order to improve generalization and prediction efficiency. Existing ensemble pruning algorithms require much pruning time. This paper presents a fast pruning approach: pattern mining based ensemble pruning (PMEP). In this algorithm, the prediction results of all base classifiers are organized as a transaction database, and FP-Tree structure is used to compact the prediction results. Then a greedy pattern mining method is explored to find the ensemble of size k. After obtaining the ensembles of all possible sizes, the one with the best accuracy is outputted. Compared with Bagging, GASEN, and Forward Selection, experimental results show that PMEP achieves the best prediction accuracy and keeps the size of the final ensemble small, more importantly, its pruning time is much less than other ensemble pruning algorithms.     

Classifier ensemble for an effective cytological image analysis

Breast cancer is the most common type of cancer among women. As early detection is crucial for the patient’s health, much attention has been paid to the development of tools for effective recognition of this disease. This article presents an application of image analysis and classification methods for fine needle biopsy. In our approach, each patient is described by nine microscopic images taken from the biopsy sample. The images are related to regions of the biopsy that seem interesting to the physician who selects them arbitrarily. We propose four different hybrid segmentation algorithms dedicated to processing these images and examine their effectiveness for the nuclei feature extraction task. Classification is carried out with the usage of a classifier ensemble based on the Random Subspaces approach. To boost its effectiveness, we use a linear combination of the support functions returned by the individual classifiers in the ensemble. In the proposed medical support system, the final decision about the patient is delivered after a fusion of nine separate outputs of the classifier – each for a different image. Experimental results carried out on a diverse dataset collected by the authors prove that the proposed solution outperforms state-of-the-art classifiers and shows itself to be a valuable tool for supporting day-to-day cytologist’s routine.     

Considering diversity and accuracy simultaneously for ensemble pruning

Diversity among individual classifiers is widely recognized to be a key factor to successful ensemble selection, while the ultimate goal of ensemble pruning is to improve its predictive accuracy. Diversity and accuracy are two important properties of an ensemble. Existing ensemble pruning methods always consider diversity and accuracy separately. However, in contrast, the two closely interrelate with each other, and should be considered simultaneously. Accordingly, three new measures, i.e., Simultaneous Diversity & Accuracy, Diversity-Focused-Two and Accuracy-Reinforcement, are developed for pruning the ensemble by greedy algorithm. The motivation for Simultaneous Diversity & Accuracy is to consider the difference between the subensemble and the candidate classifier, and simultaneously, to consider the accuracy of both of them. With Simultaneous Diversity & Accuracy, those difficult samples are not given up so as to further improve the generalization performance of the ensemble. The inspiration of devising Diversity-Focused-Two stems from the cognition that ensemble diversity attaches more importance to the difference among the classifiers in an ensemble. Finally, the proposal of Accuracy-Reinforcement reinforces the concern about ensemble accuracy. Extensive experiments verified the effectiveness and efficiency of the proposed three pruning measures. Through the investigation of this work, it is found that by considering diversity and accuracy simultaneously for ensemble pruning, well-performed selective ensemble with superior generalization capability can be acquired, which is the scientific value of this paper.     

Kernel matching pursuit classifier ensemble

Kernel Matching Pursuit Classifier (KMPC), a novel classification machine in pattern recognition, has an excellent advantage in solving classification problems for the sparsity of the solution. Unfortunately, the performance of the KMPC is far from the theoretically expected level of it. Ensemble Methods are learning algorithms that construct a collection of individual classifiers which are independent and yet accurate, and then classify a new data point by taking vote of their predictions. In such a way, the performance of classifiers can be improved greatly. In this paper, on a thorough investigation into the principle of KMPC and Ensemble Method, we expatiate on the theory of KMPC ensemble and pointed out the ways to construct it. The experiments performed on the artificial data and UCI data show KMPC ensemble combines the advantages of KMPC with ensemble method, and improves classification performance remarkably.     

基于自适应阈值的循环剪枝算法

针对YOLO系列目标检测算法中复杂的网络模型和大量冗余参数问题,提出了一种基于自适应阈值的循环剪枝算法：在经过基础训练和稀疏化训练后,进入到自适应阈值剪枝模块,该模块针对缩放因子分布情况,通过缩放因子对通道和卷积层的重要性进行评估,自主学习到一个剪枝阈值,再对网络模型进行剪枝,此过程可以循环进行,并在通道剪枝和层剪枝中应用。该算法中的阈值不是人为设定,而是针对当前网络结构学习获得,通过剪枝获得一个更优的精简模型。算法实验基于YOLOv3在三个数据集上验证,结果表明,该算法对不同数据集、不同网络结构表现出较强的适应性,与传统固定阈值相比,通过自适应阈值剪枝的模型在检测精度、压缩效果、推理速度等方面都取得了更优的效果。     

One-against-all ensemble for multiclass pattern classification

This paper presents a new method called one-against-all ensemble for solving multiclass pattern classification problems. The proposed method incorporates a neural network ensemble into the one-against-all method to improve the generalization performance of the classifier. The experimental results show that the proposed method can reduce the uncertainty of the decision and it is comparable to the other widely used methods.     

FuzzyBagging: A novel ensemble of classifiers

In this work, a new method for the creation of classifier ensembles is introduced. The patterns are partitioned into clusters to group together similar patterns, a training set is built using the patterns that belong to a cluster. Each of the new sets is used to train a classifier. We show that the approach here presented, called FuzzyBagging, obtains performance better than Bagging.     

改进Stacking集成学习的指纹识别算法

针对传统卷积神经网络对多传感器指纹识别泛化能力降低、准确率不高的问题,提出改进的Stacking集成学习算法。首先将AlexNet进行改进,在AlexNet中引入深度可分离卷积减少参数量,加快训练速度;引入空间金字塔池化,提升网络获取全局信息的能力;引入批归一化,加快网络收敛速度,同时提升网络在测试集上的准确率;使用全局平均池化替代全连接层,防止过拟合。然后将DenseNet和改进的AlexNet 2种卷积神经网络作为Stacking的基学习器对指纹进行分类,获得预测结果。最后对相同基学习器训练得到的各个模型,根据预测精度对各预测结果赋权,得到的预测结果再由元分类器分类。改进的Stacking算法在多传感器指纹数据库上进行实验,最终识别准确率达98.43%,相对AlexNet提升了20.05%,相对DenseNet提升了4.25%。     

基于联合动态剪枝的深度神经网络压缩算法

作为模型压缩的一个分支,网络剪枝算法通过移除深度神经网络中不重要的参数来降低计算消耗;然而,永久性的剪枝会导致模型容量不可逆转的损失.针对该问题,提出了一种联合动态剪枝的算法来综合分析卷积核与输入图像的特征.一方面,将部分卷积核置零,并允许其在训练过程中更新,直到网络收敛之后再永久性移除被置零的卷积核.另一方面,采样输...     

基于稀疏表示剪枝集成建模的烧结终点位置智能预测

烧结终点位置(BTP)是烧结过程至关重要的参数,直接决定着最终烧结矿的质量.由于BTP难以直接在线检测,因此,通过智能学习建模来实现BTP的在线预测并在此基础上进行操作参数调节对提高烧结矿质量具有重要意义.针对这一实际工程问题,首先提出一种基于遗传优化的Wrapper特征选择方法,可选取使后续预测建模性能最优的特征组合;在此基础上,为了解决单一学习器容易过拟合的问题,提出了基于随机权神经网络(RVFLNs)的稀疏表示剪枝(SRP)集成建模算法,即SRP-ERVFLNs算法.所提算法采用建模速度快、泛化性能好的RVFLNs作为个体基学习器,采用对基学习器基函数与隐层节点数等参数进行扰动的方式来增加集成学习子模型间的差异性;同时,为了进一步提高集成模型的泛化性能与计算效率,引入稀疏表示剪枝算法,实现对集成模型的高效剪枝;最后,将所提算法用于烧结过程BTP的预测建模.工业数据实验表明,所提方法相比于其他方法具有更好的预测精度、泛化性能和计算效率.     

基于颜色规格化的交通标志识别

采用神经网络群构成的分类器解决实景交通标志识别问题时,识别率普遍较低.分析可知,颜色复杂性造成的颜色失真是影响识别率的主要因素.遵循"简化复杂问题、基于颜色信息、采用智能方法"的基本思路,提出了一种新的解决方案:先通过颜色规格化方法将交通标志中复杂的颜色信息简化为5种基本颜色,然后再利用两级智能分类器实现分类.采用BP网络实现了颜色规格化;实验表明,这种方法具有很好的鲁棒性.     

基于子图策略的选择性分类器集成算法

为了去除集成学习中的冗余个体,提出了一种基于子图选择个体的分类器集成算法。训练出一批分类器,利用个体以及个体间的差异性构造出一个带权的完全无向图;利用子图方法选择部分差异性大的个体参与集成。通过使用支持向量机作为基学习器,在多个分类数据集上进行了实验研究,并且与常用的集成方法Bagging和Adaboost进行了比较,结果该方法获得了较好的集成效果。     

基于贝叶斯优化的无标签网络剪枝算法

针对深度神经网络（DNN）的参数和计算量过大问题,提出一种基于贝叶斯优化的无标签网络剪枝算法。首先,利用全局剪枝策略来有效避免以逐层方式修剪而导致的模型次优压缩率;其次,在网络剪枝过程中不依赖数据样本标签,并通过最小化剪枝网络与基线网络输出特征的距离对网络每层的压缩率进行优化;最后,利用贝叶斯优化算法寻找网络每一层的最优剪枝率,以提高子网搜索的效率和精度。实验结果表明,使用所提算法在CIFAR-10数据集上对VGG-16网络进行压缩,参数压缩率为85.32%,每秒浮点运算次数（FLOPS）压缩率为69.20%,而精度损失仅为0.43%。可见,所提算法可以有效地压缩DNN模型,且压缩后的模型仍能保持良好的精度。     

神经元模式分类器学习的进化计算算法

本文提出了一种用于神经元模式分类器学习的进化计算算法。该算法综合了非确定有限自动机和次群体的动态数据结构,可有效地完成神经网络模式分类器的结构学习,以获得最优的求解结果。该算法的有效性已由计算机仿真实验所证实,可被认为是一种很有发展前途的模式分类系统的机器学习算法。     

基于改进的BP网络模型的分类器的设计与实现

用改进的ＢＰ网络模型作了分类器。改进的模型拓扑结构最简，学习速率快，分类准确率高。     

一种新的RBF神经元网络分类算法

为了改善对人工神经网络行为的认识和研究中的"黑匣子"式的难以处理的状态,基于RBF神经元模型的几何解释,提出了一种新的RBF神经网络分类算法,算法把RBF神经元看作是高维空间里的超球面,从而将神经网络训练问题转化为点集"包含"问题.同传统的RBF网络相比,算法能够自动地优化RBF网络中核函数的个数、中心和宽度,同时,省去了传统RBF神经网络输出层线性连接权的计算,简化了网络的学习过程,大大缩短了训练时间,并且通过实验证明了算法的有效性.     

基于联邦集成算法对多源数据安全性的研究

联邦学习是隐私保护领域关注的热点内容,存在难以集中本地模型参数与因梯度更新造成数据泄露的问题。提出了一种联邦集成算法,使用256 B的密钥将不同类型的初始化模型传输至各数据源并训练,使用不同的集成算法来整合本地模型参数,使数据与模型的安全性得到很大提升。仿真结果表明,对于中小数据集而言,使用Adaboost集成算法得到的模型准确率达到92.505%,标准差约为8.6×10-8,对于大数据集而言,采用stacking集成算法得到的模型的准确率达到92.495%,标准差约为8.85×10-8,与传统整合多方数据集中训练模型的方法相比,在保证准确率的同时兼顾了数据与模型的安全性。     

A pruning algorithm with L1/2regularizer for extreme learning machine简

Compared with traditional learning methods such as the back propagation （BP） method, extreme learning machine provides much faster learning speed and needs less human intervention, and thus has been widely used. In this paper we combine the L1/2 regularization method with extreme learning machine to prune extreme learning machine. A variable learning coefficient is employed to prevent too large a learning increment. A numerical experiment demonstrates that a network pruned by L1/2 regularization has fewer hidden nodes but provides better performance than both the original network and the network pruned by L2 regularization.     

A generalized adaptive ensemble generation and aggregation approach for multiple classifier systems

In this paper, a generalized adaptive ensemble generation and aggregation (GAEGA) method for the design of multiple classifier systems (MCSs) is proposed. GAEGA adopts an “over-generation and selection” strategy to achieve a good bias-variance tradeoff. In the training phase, different ensembles of classifiers are adaptively generated by fitting the validation data globally with different degrees. The test data are then classified by each of the generated ensembles. The final decision is made by taking into consideration both the ability of each ensemble to fit the validation data locally and reducing the risk of overfitting. In this paper, the performance of GAEGA is assessed experimentally in comparison with other multiple classifier aggregation methods on 16 data sets. The experimental results demonstrate that GAEGA significantly outperforms the other methods in terms of average accuracy, ranging from 2.6% to 17.6%.     

Multi-class pattern classification using neural networks

Multi-class pattern classification has many applications including text document classification, speech recognition, object recognition, etc. Multi-class pattern classification using neural networks is not a trivial extension from two-class neural networks. This paper presents a comprehensive and competitive study in multi-class neural learning with focuses on issues including neural network architecture, encoding schemes, training methodology and training time complexity. Our study includes multi-class pattern classification using either a system of multiple neural networks or a single neural network, and modeling pattern classes using one-against-all, one-against-one, one-against-higher-order, and P-against-Q. We also discuss implementations of these approaches and analyze training time complexity associated with each approach. We evaluate six different neural network system architectures for multi-class pattern classification along the dimensions of imbalanced data, large number of pattern classes, large vs. small training data through experiments conducted on well-known benchmark data.