A fast learning algorithm of neural network with tunable activation function

This paper presents a modified structure of a neural network with tunable activation function and provides a new learning algorithm for the neural network training. Simulation results of XOR problem, Feigenbaum function, and Henon map show that the new algorithm has better performance than BP (back propagation) algorithm in terms of shorter convergence time and higher convergence accuracy. Further modifications of the structure of the neural network with the faster learning algorithm demonstrate simpler structure with even faster convergence speed and better convergence accuracy.     

Variable activation function extreme learning machine based on residual prediction compensation

For solving the problem that extreme learning machine (ELM) algorithm uses fixed activation function and cannot be residual compensation, a new learning algorithm called variable activation function extreme learning machine based on residual prediction compensation is proposed. In the learning process, the proposed method adjusts the steep degree, position and mapping scope simultaneously. To enhance the nonlinear mapping capability of ELM, particle swarm optimization algorithm is used to optimize variable parameters according to root-mean square error for the prediction accuracy of the mode. For further improving the predictive accuracy, the auto-regressive moving average model is used to model the residual errors between actual value and predicting value of variable activation function extreme learning machine (V-ELM). The prediction of residual errors is used to rectify the prediction value of V-ELM. Simulation results verified the effectiveness and feasibility of this method by using Pole, Auto-Mpg, Housing, Diabetes, Triazines and Stock benchmark datasets. Also, it was implemented to develop a soft sensor model for the gasoline dry point in delayed coking and some satisfied results were obtained.     

基于微分同胚的鲁棒激活函数的极端学习机

极端学习机作为一种分类算法已被广泛应用于各个领域,并取得较好的效果。然而在实际问题中数据的不规则分布、带有噪音以及离群点,都严重影响了极端学习机算法的分类准确率。针对这些问题,深入分析不同激活函数的特性,提出了一种基于角度优化和微分同胚理论的鲁棒激活函数（Robust activation function）。该鲁棒激活函数通过角度优化及微分同胚揭示数据的内在流形,从理论上证明了中心化样本长度与其偏离主空间角度为子空间偏离的主要因素,进而解决了噪音造成的主空间偏离问题,并且可尽量避免激活函数的输出值趋于零的情况。实验结果表明本文提出的激活函数优于其他的激活函数。     

A modified extreme learning machine with sigmoidal activation functions

This paper proposes a modified ELM algorithm that properly selects the input weights and biases before training the output weights of single-hidden layer feedforward neural networks with sigmoidal activation function and proves mathematically the hidden layer output matrix maintains full column rank. The modified ELM avoids the randomness compared with the ELM. The experimental results of both regression and classification problems show good performance of the modified ELM algorithm.     

改进式混合增量极限学习机算法

针对增量型极限学习机(I-ELM) 中存在大量降低学习效率及准确性的冗余节点的问题, 提出一种基于Delta 检验(DT) 和混沌优化算法(COA) 的改进式增量型核极限学习算法. 利用COA的全局搜索能力对I-ELM 中的隐含层节点参数进行寻优, 结合DT 算法检验模型输出误差, 确定有效的隐含层节点数量, 从而降低网络复杂程度, 提高算法的学习效率; 加入核函数可增强网络的在线预测能力. 仿真结果表明, 所提出的DCI-ELMK 算法具有较好的预测精度和泛化能力, 网络结构更为紧凑.

     

Hybrid evolutionary algorithm with extreme machine learning fitness function evaluation for two-stage capacitated facility location problems

This paper considers the two-stage capacitated facility location problem (TSCFLP) in which products manufactured in plants are delivered to customers via storage depots. Customer demands are satisfied subject to limited plant production and limited depot storage capacity. The objective is to determine the locations of plants and depots in order to minimize the total cost including the fixed cost and transportation cost. However, the problem is known to be NP-hard. A practicable exact algorithm is impossible to be developed. In order to solve large-sized problems encountered in the practical decision process, an efficient alternative approximate method becomes more valuable. This paper aims to propose a hybrid evolutionary algorithm framework with machine learning fitness approximation for delivering better solutions in a reasonable amount of computational time. In our study, genetic operators are adopted to perform the search process and a local search strategy is used to refine the best solution found in the population. To avoid the expensive consumption of computational time during the fitness evaluating process, the framework uses extreme machine learning to approximate the fitness of most individuals. Moreover, two heuristics based on the characteristics of the problem is incorporated to generate a good initial population. Computational experiments are performed on two sets of test instances from the recent literature. The performance of the proposed algorithm is evaluated and analyzed. Compared with other algorithms in the literature, the proposed algorithm can find the optimal or near-optimal solutions in a reasonable amount of computational time. By employing the proposed algorithm, facilities can be positioned more efficiently, which means the fixed cost and the transportation cost can be decreased significantly, and organizations can enhance competitiveness by using the optimized facility location scheme.     

Absent extreme learning machine algorithm with application to packed executable identification

Extreme learning machine (ELM) has been an important research topic over the last decade due to its high efficiency, easy-implementation, unification of classification and regression, and unification of binary and multi-class learning tasks. Though integrating these advantages, existing ELM algorithms cannot directly handle the case where some features of the samples are missing or unobserved, which is usually very common in practical applications. The work in this paper fills this gap by proposing an absent ELM (A-ELM) algorithm to address the above issue. By observing the fact that some structural characteristics of a part of packed malware instances hold unreasonable values, we cast the packed executable identification tasks into an absence learning problem, which can be efficiently addressed via the proposed A-ELM algorithm. Extensive experiments have been conducted on six UCI data sets and a packed data set to evaluate the performance of the proposed algorithm. As indicated, the proposed A-ELM algorithm is superior to other imputation algorithms and existing state-of-the-art ones.     

一种鲁棒非平衡极速学习机算法

极速学习机（ELM）算法只对平衡数据集分类较好,对于非平衡数据集,它通常偏向多数样本类,对于少数样本类性能较低。针对这一问题,提出了一种处理不平衡数据集分类的ELM模型（ELM-CIL）,该模型按照代价敏感学习的原则为少数类样本赋予较大的惩罚系数,并引入模糊隶属度值减小了外围噪声点的影响。实验表明,提出的方法不仅对提高不平衡数据集中少数类的分类精度效果较明显,而且提高了对噪声的鲁棒性。     

遗传优化核极限学习机的数据分类算法

为了提高核极限学习机(KELM)数据分类的精度,提出了一种结合K折交叉验证(K-CV)与遗传算法(GA)的KELM分类器参数优化方法(GA-KELM),将CV训练所得多个模型的平均精度作为GA的适应度评价函数,为KELM的参数优化提供评价标准,用获得GA优化最优参数的KELM算法进行数据分类.利用UCI中数据集进行仿真,实验结果表明:所提方法在整体性能上优于GA结合支持向量机法(GA-SVM)和GA结合反向传播(GA-BP)算法,具有更高的分类精度.     

弹性网络核极限学习机的多标记学习算法

将正则化极限学习机或者核极限学习机理论应用到多标记分类中,一定程度上提高了算法的稳定性。但目前这些算法关于损失函数添加的正则项都基于L2正则,导致模型缺乏稀疏性表达。同时,弹性网络正则化既保证模型鲁棒性且兼具模型稀疏化学习,但结合弹性网络的极限学习机如何解决多标记问题鲜有研究。基于此,本文提出一种对核极限学习机添加弹性网络正则化的多标记学习算法。首先,对多标记数据特征空间使用径向基核函数映射;随后,对核极限学习机损失函数施加弹性网络正则项;最后,采用坐标下降法迭代求解输出权值以得到最终预测标记。通过对比试验和统计分析表明,提出的算法具有更好的性能表现。     

Evolutionary extreme learning machine

Extreme learning machine (ELM) [G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: a new learning scheme of feedforward neural networks, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN2004), Budapest, Hungary, 25-29 July 2004], a novel learning algorithm much faster than the traditional gradient-based learning algorithms, was proposed recently for single-hidden-layer feedforward neural networks (SLFNs). However, ELM may need higher number of hidden neurons due to the random determination of the input weights and hidden biases. In this paper, a hybrid learning algorithm is proposed which uses the differential evolutionary algorithm to select the input weights and Moore-Penrose (MP) generalized inverse to analytically determine the output weights. Experimental results show that this approach is able to achieve good generalization performance with much more compact networks.     

基于改进极限学习机算法的行为识别

重点研究了极限学习机ELM对行为识别检测的效果。针对在线学习和行为分类上存在计算复杂性和时间消耗大的问题,提出了一种新的行为识别学习算法(ELM-Cholesky)。该算法首先引入了基于Cholesky分解求ELM的方法,接着依据在线学习期间核函数矩阵的更新特点,将分块矩阵Cholesky分解算法用于ELM的在线求解,使三角因子矩阵实现在线更新,从而得出一种新的ELM-Cholesky在线学习算法。新算法充分利用了历史训练数据,降低了计算的复杂性,提高了行为识别的准确率。最后,在基准数据库上采用该算法进行了大量实验,实验结果表明了这种在线学习算法的有效性。     

Symmetric extreme learning machine

Extreme learning machine (ELM) can be considered as a black-box modeling approach that seeks a model representation extracted from the training data. In this paper, a modified ELM algorithm, called symmetric ELM (S-ELM), is proposed by incorporating a priori information of symmetry. S-ELM is realized by transforming the original activation function of hidden neurons into a symmetric one with respect to the input variables of the samples. In theory, S-ELM can approximate N arbitrary distinct samples with zero error. Simulation results show that, in the applications where there exists the prior knowledge of symmetry, S-ELM can obtain better generalization performance, faster learning speed, and more compact network architecture.     

基于改进极限学习机的软测量建模方法

针对生物发酵过程中一些生物参量难以测量的问题,提出一种基于改进极限学习机（IELM）的软测量建模方法。该方法通过最小二乘方法和误差反馈原理计算出最优的网络输入到隐含层的学习参数,以提高模型的稳定性和预测精度。通过双对角化方法计算出最优的输出权值,解决输出矩阵的病态问题,进一步提高模型的稳定性。将所提方法应用于红霉素发酵过程生物量浓度的软测量。结果表明,与ELM、PL-ELM、IRLS-ELM软测量建模方法相比,IELM在线软测量建模方法具有更高的预测精度和更强的泛化能力。     

A new dandelion algorithm and optimization for extreme learning machine

In this paper, we proposed the Dandelion Algorithm (DA), based on the behaviour of dandelion sowing. In DA, the dandelion is sown in a certain range based on dynamic radius. Meanwhile the dandelion has self-learning ability; it could select a number of excellent seeds to learn. We compare the proposed algorithm with other existing algorithms. Simulations show that the proposed algorithm seems much superior to other algorithms. Moreover, the proposed algorithm can be applied to optimise extreme learning machine (ELM), which has a very good classification and prediction capability.     

狮群优化核极限学习机的分类算法

在核极限学习机(Kernel Based Extreme Learning Machine,KELM)分类应用的基础上,结合狮群算法(Loin Swarm Optimization,LSO)强全局寻优能力与收敛快的特性,提出一种LSO优化KELM算法.将测试准确率作为LSO优化KELM的适应度函数,根据移动位置获取最优...     

基于灵敏度分析法的ELM剪枝算法

针对极端学习机(ELM)网络结构设计问题,提出基于灵敏度分析法的ELM剪枝算法.利用隐含层节点输出和相对应的输出层权值向量,定义学习残差对于隐含层节点的灵敏度和网络规模适应度,根据灵敏度大小判断隐含层节点的重要性,利用网络规模适应度确定隐含层节点个数,删除重要性较低的节点.仿真结果表明,所提出的算法能够较为准确地确定与学习样本相匹配的网络规模,解决了ELM网络结构设计问题.     

Self-adaptive extreme learning machine

In order to overcome the disadvantage of the traditional algorithm for SLFN (single-hidden layer feedforward neural network), an improved algorithm for SLFN, called extreme learning machine (ELM), is proposed by Huang et al. However, ELM is sensitive to the neuron number in hidden layer and its selection is a difficult-to-solve problem. In this paper, a self-adaptive mechanism is introduced into the ELM. Herein, a new variant of ELM, called self-adaptive extreme learning machine (SaELM), is proposed. SaELM is a self-adaptive learning algorithm that can always select the best neuron number in hidden layer to form the neural networks. There is no need to adjust any parameters in the training process. In order to prove the performance of the SaELM, it is used to solve the Italian wine and iris classification problems. Through the comparisons between SaELM and the traditional back propagation, basic ELM and general regression neural network, the results have proven that SaELM has a faster learning speed and better generalization performance when solving the classification problem.     

A pruning algorithm with L1/2regularizer for extreme learning machine简

Compared with traditional learning methods such as the back propagation （BP） method, extreme learning machine provides much faster learning speed and needs less human intervention, and thus has been widely used. In this paper we combine the L1/2 regularization method with extreme learning machine to prune extreme learning machine. A variable learning coefficient is employed to prevent too large a learning increment. A numerical experiment demonstrates that a network pruned by L1/2 regularization has fewer hidden nodes but provides better performance than both the original network and the network pruned by L2 regularization.     

基于多层学习克隆选择的改进式增量型超限学习机算法

针对增量型超限学习机(incremental extreme learning machine,I-ELM)中大量冗余节点可导致算法学习效率降低,网络结构复杂化等问题,提出基于多层学习(multi-learning)优化克隆选择算法(clone selection algorithm,CSA)的改进式I-ELM.利用Baldwinian learning操作改变抗体信息的搜索范围,结合Lamarckian learning操作提高CSA的搜索能力.改进后的算法能够有效控制I-ELM的隐含层节点数,使网络结构更加紧凑,提高算法精度.仿真结果表明,所提出的基于多层学习克隆选择的增量型核超限学习机(multi-learning clonal selection I-ELMK,MLCSIELMK)算法能够有效简化网络结构,并保持较好的泛化能力,较强的学习能力和在线预测能力.