A survival ensemble of extreme learning machine

Due to the fast learning speed, simplicity of implementation and minimal human intervention, extreme learning machine has received considerable attentions recently, mostly from the machine learning community. Generally, extreme learning machine and its various variants focus on classification and regression problems. Its potential application in analyzing censored time-to-event data is yet to be verified. In this study, we present an extreme learning machine ensemble to model right-censored survival data by combining the Buckley-James transformation and the random forest framework. According to experimental and statistical analysis results, we show that the proposed model outperforms popular survival models such as random survival forest, Cox proportional hazard models on well-known low-dimensional and high-dimensional benchmark datasets in terms of both prediction accuracy and time efficiency.     

MR-ELM: a MapReduce-based framework for large-scale ELM training in big data era

In the big data era, extreme learning machine (ELM) can be a good solution for the learning of large sample data as it has high generalization performance and fast training speed. However, the emerging big and distributed data blocks may still challenge the method as they may cause large-scale training which is hard to be finished by a common commodity machine in a limited time. In this paper, we propose a MapReduce-based distributed framework named MR-ELM to enable large-scale ELM training. Under the framework, ELM submodels are trained parallelly with the distributed data blocks on the cluster and then combined as a complete single-hidden layer feedforward neural network. Both classification and regression capabilities of MR-ELM have been theoretically proven, and its generalization performance is shown to be as high as that of the original ELM and some common ELM ensemble methods through many typical benchmarks. Compared with the original ELM and the other parallel ELM algorithms, MR-ELM is a general and scalable ELM training framework for both classification and regression and is suitable for big data learning under the cloud environment where the data are usually distributed instead of being located in one machine.     

Polarimetric SAR image classification by Boosted Multiple-Kernel Extreme Learning Machines with polarimetric and spatial features

In this paper, we propose new approach: Boosted Multiple-Kernel Extreme Learning Machines (BMKELMs), a multiple kernel version of Kernel Extreme Learning Machine (KELM). We apply it to the classification of fully polarized SAR images using multiple polarimetric and spatial features. Compared with other conventional multiple kernel learning methods, BMKELMs exploit KELM with the boosting paradigm coming from ensemble learning (EL) to train multiple kernels. Additionally, different fusion strategies such as majority voting, weighted majority voting, MetaBoost, and ErrorPrune were used for selecting the classification result with the highest overall accuracy. To show the performance of BMKELMs against other state-of-the-art approaches, two L-band fully polarimetric airborne SAR images (Airborne Synthetic Aperture Radar (AIRSAR) data collected by NASA JPL over the Flevoland area of The Netherlands and Electromagnetics Institute Synthetic Aperture Radar (EMISAR) data collected by DLR over Foulum in Denmark) were considered. Experimental results indicate that the proposed technique achieves the highest classification accuracy values when dealing with multiple features, such as a combination of polarimetric coherency and multi-scale spatial features.     

Aerial scene classification via an ensemble extreme learning machine classifier based on discriminative hybrid convolutional neural networks features

Identifying a discriminative feature can effectively improve the classification performance of aerial scene classification. Deep convolutional neural networks (DCNN) have been widely used in aerial scene classification for its learning discriminative feature ability. The DCNN feature can be more discriminative by optimizing the training loss function and using transfer learning methods. To enhance the discriminative power of a DCNN feature, the improved loss functions of pretraining models are combined with a softmax loss function and a centre loss function. To further improve performance, in this article, we propose hybrid DCNN features for aerial scene classification. First, we use DCNN models with joint loss functions and transfer learning from pretrained deep DCNN models. Second, the dense DCNN features are extracted, and the discriminative hybrid features are created using linear connection. Finally, an ensemble extreme learning machine (EELM) classifier is adopted for classification due to its general superiority and low computational cost. Experimental results based on the three public benchmark data sets demonstrate that the hybrid features obtained using the proposed approach and classified by the EELM classifier can result in remarkable performance.     

Convex Hull Ensemble Machine for Regression and Classification

We propose a new ensemble algorithm called Convex Hull Ensemble Machine (CHEM). CHEM in Hilbert space is first developed and modified for regression and classification problems. We prove that the ensemble model converges to the optimal model in Hilbert space under regularity conditions. Empirical studies reveal that, for classification problems, CHEM has a prediction accuracy similar to that of boosting, but CHEM is much more robust with respect to output noise and never overfits datasets even when boosting does. For regression problems, CHEM is competitive with other ensemble methods such as gradient boosting and bagging.     

Composite kernel learning

The Support Vector Machine is an acknowledged powerful tool for building classifiers, but it lacks flexibility, in the sense that the kernel is chosen prior to learning. Multiple Kernel Learning enables to learn the kernel, from an ensemble of basis kernels, whose combination is optimized in the learning process. Here, we propose Composite Kernel Learning to address the situation where distinct components give rise to a group structure among kernels. Our formulation of the learning problem encompasses several setups, putting more or less emphasis on the group structure. We characterize the convexity of the learning problem, and provide a general wrapper algorithm for computing solutions. Finally, we illustrate the behavior of our method on multi-channel data where groups correspond to channels.     

Shared domains of competence of approximate learning models using measures of separability of classes

In this work we jointly analyze the performance of three classic Artificial Neural Network models and one Support Vector Machine with respect to a series of data complexity measures known as measures of separability of classes. In particular, we consider a Radial Basis Function Network, a Multi-Layer Perceptron, a Learning Vector Quantization, while the Sequential Minimal Optimization method is used to model the Support Vector Machine.We consider five measures of separability of classes over a wide range of data sets built from real data which have proved to be very discriminative when analyzing the performance of classifiers. We find that two of them allow us to extract common behavior patterns for the four learning methods due to their related nature. We obtain rules using these two metrics that describe both good or bad performance of the Artificial Neural Networks and the Support Vector Machine.With the obtained rules, we characterize the performance of the methods from the data set complexity metrics and therefore their common domains of competence are established. Using these domains of competence the shared good and bad capabilities of these four models can be used to know if the approximative models will perform well or poorly or if a more complex configuration of the model is needed for a given problem in advance.     

An improved extreme learning machine with self-recurrent hidden layer

Extreme learning machine (ELM) is widely used in complex industrial problems, especially the online-sequential extreme learning machine (OS-ELM) plays a good role in industrial online modeling. However, OS-ELM requires batch samples to be pre-trained to obtain initial weights, which may reduce the timeliness of samples. This paper proposes a novel model for the online process regression prediction, which is called the Recurrent Extreme Learning Machine (Recurrent-ELM). The nodes between the hidden layers are connected in Recurrent-ELM, thus the input of the hidden layer receives both the information from the current input layer and the previously hidden layer. Moreover, the weights and biases of the proposed model are generated by analysis rather than random. Six regression applications are used to verify the designed Recurrent-ELM, compared with extreme learning machine (ELM), fast learning network (FLN), online sequential extreme learning machine (OS-ELM), and an ensemble of online sequential extreme learning machine (EOS-ELM), the experimental results show that the Recurrent-ELM has better generalization and stability in several samples. In addition, to further test the performance of Recurrent-ELM, we employ it in the combustion modeling of a 330 MW coal-fired boiler compared with FLN, SVR and OS-ELM. The results show that Recurrent-ELM has better accuracy and generalization ability, and the theoretical model has some potential application value in practical application.     

基于C-AdaBoost模型的乳腺癌预测研究

机器学习和深度学习技术可用于解决医学分类预测中的许多问题,其中一些分类算法的预测精度较高,而另一些算法的精度有限。提出了基于C-AdaBoost模型的集成学习算法,对乳腺癌疾病进行预测,发现了判断乳腺癌是否复发、乳腺癌肿瘤是否为良性的最优特征组合。通过逐步回归方法对现有特征进行二次选取,并结合C-AdaBoost模型使得预测效果更优。大量实验表明,基于C-AdaBoost模型的算法的预测准确率比SVM、Naive Bayes、RandomForest以及传统的集成学习模型等机器学习分类器的准确率最多可提高19.5%,从而可以更好地帮助医生进行临床决策。     

A novel mobile robot localization approach based on topological maps using classification with reject option in omnidirectional images

Mobile robot localization, which allows a robot to identify its position, is one of main challenges in the field of Robotics. In this work, we provide an evaluation of consolidated feature extractions and machine learning techniques from omnidirectional images focusing on topological map and localization tasks. The main contributions of this work are a novel method for localization via classification with reject option using omnidirectional images, as well as two novel omnidirectional image data sets. The localization system was analyzed in both virtual and real environments. Based on the experiments performed, the Minimal Learning Machine with Nearest Neighbors classifier and Local Binary Patterns feature extraction proved to be the best combination for mobile robot localization with accuracy of 96.7% and an Fscore of 96.6%.     

Ensemble weighted extreme learning machine for imbalanced data classification based on differential evolution

Extreme learning machine for single-hidden-layer feedforward neural networks has been extensively applied in imbalanced data learning due to its fast learning capability. Ensemble approach can effectively improve the classification performance by combining several weak learners according to a certain rule. In this paper, a novel ensemble approach on weighted extreme learning machine for imbalanced data classification problem is proposed. The weight of each base learner in the ensemble is optimized by differential evolution algorithm. Experimental results on 12 datasets show that the proposed method could achieve more classification performance compared with the simple vote-based ensemble method and non-ensemble method.

     

流形极限学习机自编码特征表示

极限学习机（ELM）作为一种无监督分类方法,具有学习速度快、泛化性能高、逼近能力好的优点。随着无监督学习的发展,将ELM与自动编码器集成已成为无标签数据集提取特征的新视角,如极限学习机自动编码器（ELM-AE）是一种无监督的神经网络,无需迭代即可找到代表原始样本和其学习过程的主要成分。其重建输入信号获取原始样本的主要特征,且考虑了原始数据的全局信息以避免信息的丢失,然而这类方法未考虑数据的固有流形结构即样本间的近邻结构关系。借鉴极限学习机自动编码器的思想,提出了一种基于流形的极限学习机自动编码器算法（M-ELM）。该算法是一种非线性无监督特征提取方法,结合流形学习保持数据的局部信息,且在特征提取过程中同时对相似度矩阵进行学习。通过在IRIS数据集、脑电数据集和基因表达数据集上进行实验,将该算法与其他无监督学习方法PCA、LPP、NPE、LE和ELM-AE算法经过[k]-means聚类后的准确率进行了比较,以表明该算法的有效性。     

Tool wear monitoring and prognostics challenges: a comparison of connectionist methods toward an adaptive ensemble model

In a high speed milling operation the cutting tool acts as a backbone of machining process, which requires timely replacement to avoid loss of costly workpiece or machine downtime. To this aim, prognostics is applied for predicting tool wear and estimating its life span to replace the cutting tool before failure. However, the life span of cutting tools varies between minutes or hours, therefore time is critical for tool condition monitoring. Moreover, complex nature of manufacturing process requires models that can accurately predict tool degradation and provide confidence for decisions. In this context, a data-driven connectionist approach is proposed for tool condition monitoring application. In brief, an ensemble of Summation Wavelet-Extreme Learning Machine models is proposed with incremental learning scheme. The proposed approach is validated on cutting force measurements data from Computer Numerical Control machine. Results clearly show the significance of our proposition.     

基于CMAES集成学习方法的地表水质分类

为了提高人民生活质量,政府部门不断加强水质管理,然而人工分类方法无法满足实时处理的需求,传统机器学习方法的分类准确率又不够高。集成学习使用多种学习算法来获得比单一学习算法更好的预测性能。首先,对集成学习进行概述,简要介绍了Bagging和Boosting算法,并提出基于协方差自适应调整的进化策略算法(CMAES)的集成学习方法。接着,介绍了数据处理方式、模型评估方法和评价指标。最后,用CMAES集成学习方法对逻辑回归、线性判别分析、支持向量机、决策树、完全随机树、朴素贝叶斯、K-邻近算法、随机森林、完全随机树林、深度级联森林十种模型进行集成。实验结果表明,CMAES集成学习方法优于所有其他模型,该方法将继续被应用到未来的研究之中。     

面向贯序不均衡数据的混合采样极限学习机

针对现有机器学习算法难以有效提高贯序不均衡数据分类问题中少类样本分类精度的问题,提出一种基于混合采样策略的在线贯序极限学习机。该算法可在提高少类样本分类精度的前提下,减少多类样本的分类精度损失,主要包括离线和在线两个阶段:离线阶段采用均衡采样策略,利用主曲线分别构建多类和少类样本的可信区域,在不改变样本分布特性的前提下,利用可信区域扩充少类样本和削减多类样本,进而得到均衡的离线样本集,建立初始模型;在线阶段仅对贯序到达的多类数据进行欠采样,根据样本重要度挑选最具价值的多类样本,进而动态更新网络权值。通过理论分析证明所提算法在理论上存在损失信息上界。采用UCI标准数据集和实际的澳门空气污染预报数据进行仿真实验,结果表明,与现有在线贯序极限学习机(OS-ELM)、极限学习机(ELM)和元认知在线贯序极限学习机(MCOS-ELM)算法相比,所提算法对少类样本的预测精度更高,且数值稳定性良好。     

Generalized sparse metric learning with relative comparisons

The objective of sparse metric learning is to learn a distance measure from a set of data in addition to finding a low-dimensional representation. Despite demonstrated success, the performance of existing sparse metric learning approaches is usually limited because the methods assumes certain problem relaxations or they target the SML objective indirectly. In this paper, we propose a Generalized Sparse Metric Learning method. This novel framework offers a unified view for understanding many existing sparse metric learning algorithms including the Sparse Metric Learning framework proposed in (Rosales and Fung ACM International conference on knowledge discovery and data mining (KDD), pp 367–373, 2006), the Large Margin Nearest Neighbor (Weinberger et al. in Advances in neural information processing systems (NIPS), 2006; Weinberger and Saul in Proceedings of the twenty-fifth international conference on machine learning (ICML-2008), 2008), and the D-ranking Vector Machine (D-ranking VM) (Ouyang and Gray in Proceedings of the twenty-fifth international conference on machine learning (ICML-2008), 2008). Moreover, GSML also establishes a close relationship with the Pairwise Support Vector Machine (Vert et al. in BMC Bioinform, 8, 2007). Furthermore, the proposed framework is capable of extending many current non-sparse metric learning models to their sparse versions including Relevant Component Analysis (Bar-Hillel et al. in J Mach Learn Res, 6:937–965, 2005) and a state-of-the-art method proposed in (Xing et al. Advances in neural information processing systems (NIPS), 2002). We present the detailed framework, provide theoretical justifications, build various connections with other models, and propose an iterative optimization method, making the framework both theoretically important and practically scalable for medium or large datasets. Experimental results show that this generalized framework outperforms six state-of-the-art methods with higher accuracy and significantly smaller dimensionality for seven publicly available datasets.     

机器学习在故障检测与诊断领域应用综述

机器学习已经成为当前技术发展热点,由于机器学习具有快速处理大量数据、分析提取有效信息等优点,因此在故障检测与诊断技术中受到了越来越多的关注;文章系统介绍了机器学习和故障检测与诊断的概念、分类,深入了解了基于PCA和随机森林的故障检测方法和国内研究现状,以及基于决策树、支持向量机以及神经网络的故障诊断方法和国内外研究现状,其中重点介绍了卷积神经网络和递归神经网络的应用,并对机器学习算法在故障检测与诊断应用前景进行了展望,大数据时代下,机器学习在故障检测和诊断领域有着绝对优势。     

嵌入代价敏感的极限学习机相异性集成的基因表达数据分类

极限学习机的相异性集成算法(Dissimilarity Based Ensemble of Extreme Learning Machine,D-ELM)在基因表达数据分类中能够得到较稳定的分类效果,然而这种分类算法是基于分类精度的,当所给样本的误分类代价不相等时,不能直接实现代价敏感分类过程中的最小平均误分类代价的要求。通过在分类过程中引入概率估计以及误分类代价和拒识代价重新构造分类结果,提出了基于相异性集成极限学习机的代价敏感算法(CS-D-ELM)。该算法被运用到基因表达数据集上,得到了较好的分类效果。     

#br# 耦合样本先验分布信息的加权极限学习机

极限学习机广泛用于分类、聚类、回归等任务中，但在处理类不平衡分类问题时，前人未充分考虑样本先验分布信息对分类性能的影响。针对此问题，本文提出耦合样本先验分布信息的加权极限学习机（Coupling sample Prior distribution Weighted Extreme Learning Machine，CPWELM）算法。该算法基于加权极限学习机，充分探讨不同分布样本点的重要程度，以此构造代价矩阵，进而提升分类器性能。本文通过12个不平衡数据集，对CPWELM算法的可行性及有效性进行了验证。结果表明，相比同类其他算法，CPWELM算法的性能更优。