Performance enhancement of extreme learning machine for multi-category sparse data classification problems

This paper presents a performance enhancement scheme for the recently developed extreme learning machine (ELM) for multi-category sparse data classification problems. ELM is a single hidden layer neural network with good generalization capabilities and extremely fast learning capacity. In ELM, the input weights are randomly chosen and the output weights are analytically calculated. The generalization performance of the ELM algorithm for sparse data classification problem depends critically on three free parameters. They are, the number of hidden neurons, the input weights and the bias values which need to be optimally chosen. Selection of these parameters for the best performance of ELM involves a complex optimization problem.In this paper, we present a new, real-coded genetic algorithm approach called ‘RCGA-ELM’ to select the optimal number of hidden neurons, input weights and bias values which results in better performance. Two new genetic operators called ‘network based operator’ and ‘weight based operator’ are proposed to find a compact network with higher generalization performance. We also present an alternate and less computationally intensive approach called ‘sparse-ELM’. Sparse-ELM searches for the best parameters of ELM using K-fold validation. A multi-class human cancer classification problem using micro-array gene expression data (which is sparse), is used for evaluating the performance of the two schemes. Results indicate that the proposed RCGA-ELM and sparse-ELM significantly improve ELM performance for sparse multi-category classification problems.     

Direct adaptive neural control of nonlinear systems with extreme learning machine

A direct adaptive neural control scheme for a class of nonlinear systems is presented in the paper. The proposed control scheme incorporates a neural controller and a sliding mode controller. The neural controller is constructed based on the approximation capability of the single-hidden layer feedforward network (SLFN). The sliding mode controller is built to compensate for the modeling error of SLFN and system uncertainties. In the designed neural controller, its hidden node parameters are modified using the recently proposed neural algorithm named extreme learning machine (ELM), where they are assigned random values. However, different from the original ELM algorithm, the output weight is updated based on the Lyapunov synthesis approach to guarantee the stability of the overall control system. The proposed adaptive neural controller is finally applied to control the inverted pendulum system with two different reference trajectories. The simulation results demonstrate good tracking performance of the proposed control scheme.     

正负模糊规则系统、极限学习机与图像分类

传统的图像分类一般只利用了图像的正规则,忽略了负规则在图像分类中的作用。Nguyen将负规则引入图像分类,提出将正负模糊规则相结合形成正负模糊规则系统,并将其用于遥感图像和自然图像的分类。实验证明,其在图像分类过程中取得了很好的效果。他们提出的前馈神经网络模型在调整权值时利用了梯度下降法,由于步长选择不合理或陷入局部最优从而使训练速度受到了限制。极限学习机（ELM）是一种单隐层前馈神经网络（SLFN）学习算法,具有学习速度快,泛化性能好的优点。本文证明了极限学习机与正负模糊规则系统的实质是等价的,遂将其用于图像分类。实验结果说明了极限学习机能很好的利用正负模糊规则相结合的方法对图像进行分类,实验结果较为理想。     

A fast incremental extreme learning machine algorithm for data streams classification

Data streams classification is an important approach to get useful knowledge from massive and dynamic data. Because of concept drift, traditional data mining techniques cannot be directly applied in data streams environment. Extreme learning machine (ELM) is a single hidden layer feedforward neural network (SLFN), comparing with the traditional neural network (e.g. BP network), ELM has a faster speed, so it is very suitable for real-time data processing. In order to deal with the challenge in data streams classification, a new approach based on extreme learning machine is proposed in this paper. The approach utilizes ELMs as base classifiers and adaptively decides the number of the neurons in hidden layer, in addition, activation functions are also randomly selected from a series of functions to improve the performance of the approach. Finally, the algorithm trains a series of classifiers and the decision results for unlabeled data are made by weighted voting strategy. When the concept in data streams keeps stable, every classifier is incrementally updated by using new data; if concept drift is detected, the classifiers with weak performance will be cleared away. In the experiment, we used 7 artificial data sets and 9 real data sets from UCI repository to evaluate the performance of the proposed approach. The testing results showed, comparing with the conventional classification methods for data streams such as ELM, BP, AUE2 and Learn++.MF, on most data sets, the new approach could not only be simplest in the structure, but also get a higher and more stable accuracy with lower time consuming.     

Stable indirect adaptive neural controller for a class of nonlinear system

The paper presents an indirect adaptive neural control scheme for a general high-order nonlinear continuous system. In the proposed scheme a neural controller is constructed based on the single-hidden layer feedforward network (SLFN) for approximating the unknown nonlinearities of dynamic systems. A sliding mode controller is also incorporated to compensate for the modelling errors of SLFN. The parameters of the SLFN are modified using the recently proposed neural algorithm named extreme learning machine (ELM), where the parameters of the hidden nodes are assigned randomly. However different from the original ELM algorithm, the output weights are updated based on the Lyapunov synthesis approach to guarantee the stability of the overall control system, even in the presence of modelling errors which are offset using the sliding mode controller. Finally the proposed adaptive neural controller is applied to control the inverted pendulum system with two different reference trajectories. The simulation results demonstrate that good tracking performance is achieved by the proposed control scheme.     

A study on effectiveness of extreme learning machine

Extreme learning machine (ELM), proposed by Huang et al., has been shown a promising learning algorithm for single-hidden layer feedforward neural networks (SLFNs). Nevertheless, because of the random choice of input weights and biases, the ELM algorithm sometimes makes the hidden layer output matrix H of SLFN not full column rank, which lowers the effectiveness of ELM. This paper discusses the effectiveness of ELM and proposes an improved algorithm called EELM that makes a proper selection of the input weights and bias before calculating the output weights, which ensures the full column rank of H in theory. This improves to some extend the learning rate (testing accuracy, prediction accuracy, learning time) and the robustness property of the networks. The experimental results based on both the benchmark function approximation and real-world problems including classification and regression applications show the good performances of EELM.     

基于极限学习机的网页分类应用

极限学习机ELM不同于传统的神经网络学习算法（如BP算法）,是一种高效的单隐层前馈神经网络（SLFNs）学习算法。将极限学习机引入到中文网页分类任务中。对中文网页进行预处理,提取其特性信息,从而形成网页特征树,产生定长编码作为极限学习机的输入数据。实验结果表明该方法能够有效地分类网页。     

极限学习机在机场旅客吞吐量预测中的应用

极限学习机(ELM)是一种新型单馈层神经网络算法,在训练过程中只需要设置合适的隐藏层节点个数,随机赋值输入权值和隐藏层偏差,一次完成无需迭代.结合遗传算法在预测模型参数寻优方面的优势,找到极限学习机的最优参数取值,建立成都双流国际机场旅客吞吐量预测模型,通过对比支持向量机、BP神经网络,分析遗传-极限学习机算法在旅客吞吐量预测中的可行性和优势.仿真结果表明遗传-极限学习机算法不仅可行,并且与原始极限学习机算法相比,在预测精度和训练速度上具有比较明显的优势.     

Face recognition via local preserving average neighborhood margin maximization and extreme learning machine

Average neighborhood maximum margin (ANMM) is an effective method for feature extraction in appearance-based face recognition. In this paper, we extend ANMM to locality preserving average neighborhood margin maximization (LPANMM) in order to maintain the local structure on the original data manifold in the discriminant feature space. We also combine LPANMM with extreme learning machine (ELM) as a new scheme for face recognition, we train the single-hidden layer feedforward neural network (SLFN) in the ELM classifier with the discriminant features that are extracted by LPANMM, then we use the trained ELM classifer to classify the test data. In the process of training SLFN, ELM can not only achieve the smallest training error in theory, but is also not sensitive to the initial value selection of the parameters for the SLFN. Experimental results on ORL, Yale, CMU PIE and FERET face databases demonstrate the scheme LPANMM/ELM can achieve better performance than ANMM and other traditional schemes for face recognition.     

改进鱼群算法优化的ELM在乳腺肿瘤辅助诊断中的应用研究

针对传统极限学习机的输入权值矩阵和隐含层偏差是随机给定进而可能会导致在乳腺肿瘤的辅助诊断应用研究中存在精度明显不足的情况,提出用改进鱼群算法优化ELM方法。在完成对乳腺肿瘤有效的辅助诊断的过程中,本研究工作充分利用ELM能快速地完成训练过程且具有很好的泛化能力的特点,并结合用改进鱼群算法对ELM的隐含层偏差进行优化,构造出了乳腺肿瘤与从乳腺肿瘤样本数据中提取的10个特征向量之间的非线性映射关系。将本文提出的乳腺肿瘤识别方法的仿真结果与AFSA-ELM方法、ELM方法、LVQ方法、BP方法的仿真结果分别从识别准确率、假阴性率、学习速度三个方面做对比分析,仿真结果表明,本文所提方法对乳腺肿瘤诊断具有较高的分类识别准确率、假阴性率以及较快的学习速率。     

极限学习机综述

极限学习机是一种单隐层前向网络的训练算法,主要特点是训练速度极快,而且可以达到很高的泛化性能。回顾了极限学习机的发展历程,分析了极限学习机的数学模型,详细介绍了极限学习机的各种改进算法,并列举了极限学习机在识别、预测和医学诊断领域的应用。最后总结预测了极限学习机的改进方向。     

局部感知的类限制极限学习机

极限学习机(ELM)由于高效的训练方式被广泛应用于分类回归,然而不同的输入权值在很大程度上会影响其学习性能。为了进一步提高ELM的学习性能,针对ELM的输入权值进行了研究,充分利用图像局部感知的稀疏性,将局部感知的方法运用到基于自动编码器的ELM(ELM-AE)上,提出了局部感知的类限制极限学习机(RF-C₂ELM)。通过对MNIST数据集进行分类问题分析实验,实验结果表明,在具有相同隐层结点数的条件下,提出的方法能够获得更高的分类精度。     

极限学习机在岩性识别中的应用

基于传统支持向量机(SVM)训练速度慢、参数选择难等问题,提出了基于极限学习机(ELM)的岩性识别.该算法是一种新的单隐层前馈神经网络(SLFNs)学习算法,不但可以简化参数选择过程,而且可以提高网络的训练速度.在确定了最优参数的基础上,建立了ELM的岩性分类模型,并且将ELM的分类结果与SVM进行对比.实验结果表明,ELM以较少的神经元个数获得与SVM相当的分类正确率,并且ELM参数选择比SVM简便,有效降低了训练速度,表明了ELM应用于岩性识别的可行性和算法的有效性.     

Dynamic ensemble extreme learning machine based on sample entropy

Extreme learning machine (ELM) as a new learning algorithm has been proposed for single-hidden layer feed-forward neural networks, ELM can overcome many drawbacks in the traditional gradient-based learning algorithm such as local minimal, improper learning rate, and low learning speed by randomly selecting input weights and hidden layer bias. However, ELM suffers from instability and over-fitting, especially on large datasets. In this paper, a dynamic ensemble extreme learning machine based on sample entropy is proposed, which can alleviate to some extent the problems of instability and over-fitting, and increase the prediction accuracy. The experimental results show that the proposed approach is robust and efficient.     

Nonlinear regression in environmental sciences using extreme learning machines: A comparative evaluation

The extreme learning machine (ELM), a single-hidden layer feedforward neural network algorithm, was tested on nine environmental regression problems. The prediction accuracy and computational speed of the ensemble ELM were evaluated against multiple linear regression (MLR) and three nonlinear machine learning (ML) techniques – artificial neural network (ANN), support vector regression and random forest (RF). Simple automated algorithms were used to estimate the parameters (e.g. number of hidden neurons) needed for model training. Scaling the range of the random weights in ELM improved its performance. Excluding large datasets (with large number of cases and predictors), ELM tended to be the fastest among the nonlinear models. For large datasets, RF tended to be the fastest. ANN and ELM had similar skills, but ELM was much faster than ANN except for large datasets. Generally, the tested ML techniques outperformed MLR, but no single method was best for all the nine datasets.     

改进人工蜂群优化的极限学习机

针对极限学习机(ELM)在训练过程中需要大量隐含层节点的问题,提出了差分进化与克隆算法改进人工蜂群优化的极限学习机(DECABC-ELM),在人工蜂群算法的基础上,引入了差分进化算法的差分变异算子和免疫克隆算法的克隆扩增算子,改进了人工蜂群收敛速度慢等缺点,使用改进的人工蜂群算法计算ELM的隐含层节点参数.将算法应用于回归和分类数据集,并与其他算法进行比较,获得了良好的效果.     

改进式混合增量极限学习机算法

针对增量型极限学习机(I-ELM) 中存在大量降低学习效率及准确性的冗余节点的问题, 提出一种基于Delta 检验(DT) 和混沌优化算法(COA) 的改进式增量型核极限学习算法. 利用COA的全局搜索能力对I-ELM 中的隐含层节点参数进行寻优, 结合DT 算法检验模型输出误差, 确定有效的隐含层节点数量, 从而降低网络复杂程度, 提高算法的学习效率; 加入核函数可增强网络的在线预测能力. 仿真结果表明, 所提出的DCI-ELMK 算法具有较好的预测精度和泛化能力, 网络结构更为紧凑.

     

基于极限学习机的航空发动机传感器故障诊断

针对当前应用于航空发动机传感器故障诊断中的基于梯度的传统学习算法多存在参数选择困难、容易陷入局部最小化、过拟合等问题,提出了基于极限学习机(ELM)的航空发动机传感器故障诊断方法。算法只需设置隐含层神经元的个数,能够较好地避免上述问题,缩短故障诊断时间、提升诊断精度。通过仿真试验表明:基于ELM算法所建的航空发动机传感器故障诊断模型要比基于BP神经网络算法所建的模型耗时短且精度高。     

一种基于鲁棒估计的极限学习机方法

极限学习机(ELM)是一种单隐层前馈神经网络(single-hidden layer feedforward neural networks,SLFNs),它相较于传统神经网络算法来说结构简单,具有较快的学习速度和良好的泛化性能等优点。ELM的输出权值是由最小二乘法(least square,LE)计算得出,然而经典的LS估计的抗差能力较差,容易夸大离群点和噪声的影响,从而造成训练出的参数模型不准确甚至得到完全错误的结果。为了解决此问题,提出一种基于M估计的采用加权最小二乘方法来取代最小二乘法计算输出权值的鲁棒极限学习机算法(RBELM),通过对多个数据集进行回归和分类分析实验,结果表明,该方法能够有效降低异常值的影响,具有良好的抗差能力。     

Credit score classification using spiking extreme learning machine

Credit score classification is a prominent research problem in the banking or financial industry, and its predictive performance is responsible for the profitability of financial industry. This paper addresses how Spiking Extreme Learning Machine (SELM) can be effectively used for credit score classification. A novel spike-generating function is proposed in Leaky Nonlinear Integrate and Fire Model (LNIF). Its interspike period is computed and utilized in the extreme learning machine (ELM) for credit score classification. The proposed model is named as SELM and is validated on five real-world credit scoring datasets namely: Australian, German-categorical, German-numerical, Japanese, and Bankruptcy. Further, results obtained by SELM are compared with back propagation, probabilistic neural network, ELM, voting-based Q-generalized extreme learning machine, Radial basis neural network and ELM with some existing spiking neuron models in terms of classification accuracy, Area under curve (AUC), H-measure and computational time. From the experimental results, it has been noticed that improvement in accuracy and execution time for the proposed SELM is highly statistically important for all aforementioned credit scoring datasets. Thus, integrating a biological spiking function with ELM makes it more efficient for categorization.