基于径向基神经网络和正则化极限学习机的多标签学习模型*

相比径向基(RBF)神经网络,极限学习机(ELM)训练速度更快,泛化能力更强.同时,近邻传播聚类算法(AP)可以自动确定聚类个数.因此,文中提出融合AP聚类、多标签RBF(ML-RBF)和正则化ELM(RELM)的多标签学习模型(ML-AP-RBF-RELM).首先,在该模型中输入层使用ML-RBF进行映射,且通过AP聚类算法自动确定每一类标签的聚类个数,计算隐层节点个数.然后,利用每类标签的聚类个数通过K均值聚类确定隐层节点RBF函数的中心.最后,通过RELM快速求解隐层到输出层的连接权值.实验表明,ML-AP-RBF-RELM效果较好.     

ELM网络结构自适应正交搜索算法

由于具有灵活的非线性建模能力和良好的模式识别能力,单隐藏层前馈神经网络(Single Hidden Layer Feedforward Neural Network,SLFN)一直是机器学习和数据挖掘领域关注的焦点.众所周知,网络结构是影响SLFN泛化能力的重要因素之一.给定一个具体应用,如何在训练过程中自动选取最优的隐节点个数,仍是一大挑战.极限学习机(Extreme Learning Machine,ELM)通过随机生成隐藏层节点参数,并利用最小二乘法求解输出层权值的方式来训练SLFN,在一定程度上克服了传统的基于梯度类学习方法收敛速度慢、容易陷入局部最小值等问题.然而,ELM仍需要人为确定隐节点个数,不仅过程繁琐,而且无法保证得到最优或者次优的网络结构.在不影响泛化能力的前提下,为了进一步降低网络的复杂度,本文对ELM进行了改进,通过将网络结构学习转化为子集模型选择,提出了一种隐节点自适应正交搜索方法.首先,利用标准ELM构建隐节点候选池.然后,采用正交前向选择算法选择与网络期望输出相关度最大的候选隐节点加入到模型中.同时,每向前引入一个新的隐节点,就要向后对已选入的隐节点进行逐个检查,将变得不重要的隐节点从网络中删除.最后,设计了一种增强的向后移除策略来纠正前面步骤中所犯的错误,进一步剔除模型内残留的冗余隐节点.本文方法充分考虑了隐节点间的内在联系和相互影响,实验结果表明,该方法不仅具有良好的泛化性能,而且能够产生比较紧凑的网络结构.     

基于微分同胚优化极端学习机的人脸识别

极端学习机(ELM)以其快速高效和良好的泛化能力在模式识别领域得到了广泛应用。然而当前的ELM及其改进算法并没有充分考虑到隐层节点输出矩阵对极端学习机泛化能力的影响。通过实验发现激活函数选取不当及数据维数过高将导致隐层节点输出值趋于零,使得输出权值矩阵求解不准,降低ELM的分类性能。为此,提出一种微分同胚优化的极端学习机算法。该算法结合降维和微分同胚技术提高激活函数的鲁棒性,克服隐层节点输出值趋于零的问题。为验证所提算法的有效性使用人脸数据进行实验。实验结果表明所提算法具有良好的泛化性能。     

基于隐特征空间的极限学习机模型选择

针对极限学习机（ELM）中冗余的隐神经元会削弱模型泛化能力的缺点,提出了一种基于隐特征空间的ELM模型选择算法。首先,为了寻找合适的ELM隐层,在ELM中添加正则项,该项为现有隐层空间到低维隐特征空间的映射函数矩阵的Frobenius范数;其次,为解决该非凸问题,采用交替优化的策略,并通过凸二次型优化学习该隐空间;最终自适应得到最优映射函数和ELM模型。分别采用UCI标准数据集和载荷识别工程数据对所提算法进行测试,结果表明,与经典ELM相比,该算法可有效提高预测精度和数值稳定性,与现有模型选择算法相比,该算法预测精度相当,但运行时间则大幅降低。     

基于近似结构风险的ELM隐层节点数优化

隐层节点数是影响极端学习机(ELM)泛化性能的关键参数,针对传统的ELM隐层节点数确定算法中优化过程复杂、容易过学习或陷入局部最优的问题,提出结构风险最小化-极端学习机(SRM-ELM)算法。通过分析VC维与隐层节点数量之间的关联,对VC信任函数进行近似改进,使其为凹函数,并结合经验风险重构近似的SRM。在此基础上,将粒子群优化的位置值直接作为ELM的隐层节点数,利用粒子群算法最小化结构风险函数获得极端学习机的隐层节点数,作为最优节点数。使用6组UCI数据和胶囊缺陷数据进行仿真验证,结果表明,该算法能获得极端学习机的最优节点数,并具有更好的泛化能力。     

ELM算法中随机映射作用的实验研究

通过实验研究ELM算法中随机映射的作用及神经网络中隐含层结点个数对网络泛化能力的影响.在35个数据集上进行实验,针对不同的数据集,找到网络的最优精度所对应的隐含层结点个数.实验结果表明,当随机映射使数据升维到一定维数时,网络性能得到提高.     

基于影响度剪枝的极端学习机分类算法研究

针对极端学习机(ELM)网络规模控制问题,从剪枝思路出发,提出了一种基于影响度剪枝的ELM分类算法。利用ELM网络单个隐节点连接输入层和输出层的权值向量、该隐节点的输出、初始隐节点个数以及训练样本个数,定义单个隐节点相对于整个网络学习的影响度,根据影响度判断隐节点的重要性并将其排序,采用与ELM网络规模相匹配的剪枝步长删除冗余节点,最后更新隐含层与输入层和输出层连接的权值向量。通过对多个UCI机器学习数据集进行分类实验,并将提出的算法与EM-ELM、PELM和ELM算法相比较,结果表明,该算法具有较高的稳定性和测试精度,训练速度较快,并能有效地控制网络规模。     

基于结点敏感度的单隐含层前馈神经网络结构选择

提出了一种基于结点敏感度的单隐含层前馈神经网络结构选择方法。该方法从一个隐含层结点个数较多的网络开始,首先利用结点敏感度度量隐含层结点的重要性,然后按重要性对隐含层结点由大到小排序,最后逐个剪去不重要的隐含层结点,直到满足预定义的停止条件。该算法的特点是不需要重复训练神经网络,得到的网络结构紧凑,具有较高的泛化能力。在实际数据集和UCI数据集上的实验结果显示,提出的算法是行之有效的。     

基于免疫RBF神经网络的逆运动学求解

求解机械臂逆运动学问题可以采用神经网络来建立逆运动学模型,通过遗传算法或BP算法训练神经网络的权值从而得到问题的解,在求解精度和收敛速度上有待进一步改进。采用人工免疫原理对RBF网络训练数据集的泛化能力在线调整隐层结构,生成RBF网络隐层。当网络结构确定时,采用递推最小二乘法确定网络连接权值。由此对神经网络的网络结构和连接权进行自适应调整和学习。通过仿真可以看出,用免疫原理训练的神经网络收敛速度快,泛化能力强,可大幅提高机械臂逆运动学求解精度。     

分布式训练异构任务调度算法研究

分布式机器学习中的工作结点在训练过程中经常需要处理异构任务,但任务发布者可能无法根据有效的先验知识确定边缘服务器集群中哪些是处于训练状态的工作结点。针对边缘服务器集群无法同时满足训练性能与服务质量最大化的问题,对异构任务调度算法进行了研究。首先在集群资源约束下分析了分布式训练收敛性能的影响因素;其次建立了最大化训练性能的优化目标;最后转化为多维多选择背包问题进行求解。仿真结果表明,所提异构任务调度算法能够在保证服务质量的同时,最大化分布式训练性能。     

Model selection of extreme learning machine based on multi-objective optimization

As a novel learning algorithm for single-hidden-layer feedforward neural networks, extreme learning machines (ELMs) have been a promising tool for regression and classification applications. However, it is not trivial for ELMs to find the proper number of hidden neurons due to the nonoptimal input weights and hidden biases. In this paper, a new model selection method of ELM based on multi-objective optimization is proposed to obtain compact networks with good generalization ability. First, a new leave-one-out (LOO) error bound of ELM is derived, and it can be calculated with negligible computational cost once the ELM training is finished. Furthermore, the hidden nodes are added to the network one-by-one, and at each step, a multi-objective optimization algorithm is used to select optimal input weights by minimizing this LOO bound and the norm of output weight simultaneously in order to avoid over-fitting. Experiments on five UCI regression data sets are conducted, demonstrating that the proposed algorithm can generally obtain better generalization performance with more compact network than the conventional gradient-based back-propagation method, original ELM and evolutionary ELM.     

An integrated PSO for parameter determination and feature selection of ELM and its application in classification of power system disturbances

This paper presents a performance enhancement scheme for the recently developed extreme learning machine (ELM) for classifying power system disturbances using particle swarm optimization (PSO). Learning time is an important factor while designing any computational intelligent algorithms for classifications. ELM is a single hidden layer neural network with good generalization capabilities and extremely fast learning capacity. In ELM, the input weights are chosen randomly and the output weights are calculated analytically. However, ELM may need higher number of hidden neurons due to the random determination of the input weights and hidden biases. One of the advantages of ELM over other methods is that the parameter that the user must properly adjust is the number of hidden nodes only. But the optimal selection of its parameter can improve its performance. In this paper, a hybrid optimization mechanism is proposed which combines the discrete-valued PSO with the continuous-valued PSO to optimize the input feature subset selection and the number of hidden nodes to enhance the performance of ELM. The experimental results showed the proposed algorithm is faster and more accurate in discriminating power system disturbances.     

逐层可加的急速学习机

已有的急速学习机（Extreme Learning Machine）的学习精度受隐节点数目的影响很大。无论是已提出的单隐层急速学习机还是多隐层神经网络,都是先确定隐藏层数,再通过增加每一层的神经元个数来提高精度。但当训练集规模很大时,往往需要引入很多的隐节点,导致违逆矩阵计算复杂度大,从而不利于学习效率的提高。提出逐层可加的急速学习机MHL-ELM（Extreme Learning Machine with Incremental Hidden Layers）,其思想是首先对当前隐藏层神经元（数目不大且不寻优,因而复杂度小）的权值进行随机赋值,用ELM思想求出逼近误差;若误差达不到要求,再增加一个隐含层。然后运用ELM的思想对当前隐含层优化。逐渐增加隐含层,直至满足误差精度为止。除此以外,MHL-ELM的算法复杂度为[l=1MO(N3l)]。实验使用10个UCI,keel真实数据集,通过与BP,OP-ELM等传统方法进行比较,表明MHL-ELM学习方法具有更好的泛化性,在学习精度和学习速度方面都有很大的提升。     

改进式混合增量极限学习机算法

针对增量型极限学习机(I-ELM) 中存在大量降低学习效率及准确性的冗余节点的问题, 提出一种基于Delta 检验(DT) 和混沌优化算法(COA) 的改进式增量型核极限学习算法. 利用COA的全局搜索能力对I-ELM 中的隐含层节点参数进行寻优, 结合DT 算法检验模型输出误差, 确定有效的隐含层节点数量, 从而降低网络复杂程度, 提高算法的学习效率; 加入核函数可增强网络的在线预测能力. 仿真结果表明, 所提出的DCI-ELMK 算法具有较好的预测精度和泛化能力, 网络结构更为紧凑.

     

Uncertainty evaluation and model selection of extreme learning machine based on Riemannian metric

Considering the uncertainty of hidden neurons, choosing significant hidden nodes, called as model selection, has played an important role in the applications of extreme learning machines(ELMs). How to define and measure this uncertainty is a key issue of model selection for ELM. From the information geometry point of view, this paper presents a new model selection method of ELM for regression problems based on Riemannian metric. First, this paper proves theoretically that the uncertainty can be characterized by a form of Riemannian metric. As a result, a new uncertainty evaluation of ELM is proposed through averaging the Riemannian metric of all hidden neurons. Finally, the hidden nodes are added to the network one by one, and at each step, a multi-objective optimization algorithm is used to select optimal input weights by minimizing this uncertainty evaluation and the norm of output weight simultaneously in order to obtain better generalization performance. Experiments on five UCI regression data sets and cylindrical shell vibration data set are conducted, demonstrating that the proposed method can generally obtain lower generalization error than the original ELM, evolutionary ELM, ELM with model selection, and multi-dimensional support vector machine. Moreover, the proposed algorithm generally needs less hidden neurons and computational time than the traditional approaches, which is very favorable in engineering applications.     

Performance enhancement of extreme learning machine for multi-category sparse data classification problems

This paper presents a performance enhancement scheme for the recently developed extreme learning machine (ELM) for multi-category sparse data classification problems. ELM is a single hidden layer neural network with good generalization capabilities and extremely fast learning capacity. In ELM, the input weights are randomly chosen and the output weights are analytically calculated. The generalization performance of the ELM algorithm for sparse data classification problem depends critically on three free parameters. They are, the number of hidden neurons, the input weights and the bias values which need to be optimally chosen. Selection of these parameters for the best performance of ELM involves a complex optimization problem.In this paper, we present a new, real-coded genetic algorithm approach called ‘RCGA-ELM’ to select the optimal number of hidden neurons, input weights and bias values which results in better performance. Two new genetic operators called ‘network based operator’ and ‘weight based operator’ are proposed to find a compact network with higher generalization performance. We also present an alternate and less computationally intensive approach called ‘sparse-ELM’. Sparse-ELM searches for the best parameters of ELM using K-fold validation. A multi-class human cancer classification problem using micro-array gene expression data (which is sparse), is used for evaluating the performance of the two schemes. Results indicate that the proposed RCGA-ELM and sparse-ELM significantly improve ELM performance for sparse multi-category classification problems.     

基于蚁群优化的极限学习机选择性集成学习算法

针对现有极限学习机集成学习算法分类精度低、泛化能力差等缺点,提出了一种基于蚁群优化思想的极限学习机选择性集成学习算法。该算法首先通过随机分配隐层输入权重和偏置的方法生成大量差异的极限学习机分类器,然后利用一个二叉蚁群优化搜索算法迭代地搜寻最优分类器组合,最终使用该组合分类测试样本。通过12个标准数据集对该算法进行了测试,该算法在9个数据集上获得了最优结果,在另3个数据集上获得了次优结果。采用该算法可显著提高分类精度与泛化性能。