一种新颖的径向基函数(RBF)网络学习算法

以提高RBF网络泛化能力为着眼点，提出了一种新型的网络结构自适应学习算法．该算法采用衰减聚类半径的聚类算法来确定初始的隐层结构，然后通过调整包含样本类别信息的扩展聚类不纯度来修正隐层结构，直至满足所有扩展聚类不纯度均小于等于不纯度均值以及所有扩展聚类方差均不超过方差均值这两个条件．这样就确定了隐层的最终结构．在确定隐层结构之后，采用反向传播算法来训练隐层与输出层之间的连接权重．经双螺旋线问题仿真试验验证，该算法确实具有较强的泛化能力．     

基于层次聚类的主动学习方法——HC_AL

针对传统主动学习(AL)方法对大规模的无标记样本分类收敛速度过慢的问题,提出了基于层次聚类(HC)的主动学习训练算法--HC_AL方法。通过对大规模的未标记数据进行层次聚类,并对每个层次上的类中心打标记来代替该层次上的类标记,然后将该层次上具有错误标记的类中心加入训练集。在数据集上的实验取得了较好的泛化能力和较快的收敛速度。实验结果表明通过采用分层细化、逐步求精的方法,可使主动学习的收敛速度大大提高,同时获得较为满意的学习能力。     

量子门Elman神经网络及其梯度扩展的量子反向传播学习算法

针对Elman神经网络的学习速度和泛化性能, 提出一种具有量子门结构的新型Elman神经网络模型及其梯度扩展反向传播(Back-propagation)学习算法, 新模型由量子比特神经元和经典神经元构成. 新网络结构采用量子映射层以确保来自上下文单元的局部反馈与隐藏层输入之间的模式一致; 通过量子比特神经元输出与相关量子门参数的修正互补关系以提高网络更新动力. 新学习算法采用搜索然后收敛的策略自适应地调整学习率参数以提高网络学习速度; 通过将上下文单元的权值扩展到隐藏层的权值矩阵, 使其在与隐藏层权值同步更新过程中获取时间序列的额外信息, 从而提高网络上下文单元输出与隐藏层输入之间的匹配程度. 以峰值检波为例的数值实验结果显示, 在量子反向传播学习过程中, 量子门Elman神经网络具有较快的学习速度和良好的泛化性能.     

基于核学习方法的短时交通流量预测

基于核学习的强大非线性映射性能,针对短时交通流量预测,提出一类基于核学习方法的预测模型。核递推最小二乘(KRLS)基于近似线性依赖(approximate linear dependence,ALD) 技术可降低计算复杂度及存储量,是一种在线核学习方法,适用于较大规模数据集的学习;核偏最小二乘(KPLS)方法将输入变量投影在潜在变量上,利用输入与输出变量之间的协方差信息提取潜在特征;核极限学习机(KELM)方法用核函数表示未知的隐含层非线性特征映射,通过正则化最小二乘算法计算网络的输出权值,能以极快的学习速度获得良好的推广性。为验证所提方法的有效性,将KELM、KPLS、ALD-KRLS用于不同实测交通流数据中,在同等条件下,与现有方法进行比较。实验结果表明,不同核学习方法的预测精度和训练速度均有提高,体现了核学习方法在短时交通流量预测中的应用潜力。     

融合认知心理学理论的新型教与学优化算法及应用

针对教与学优化算法(teaching-learning-based optimization, TLBO)寻优精度低、易陷入局部最优的问题,提出了一种融合认知心理学理论的新型教与学优化算法(cognitive psychology teaching-learning-based optimization, CPTLBO)。在教阶段融入登门槛效应理论,对于学习有困难的学生设置阶段性学习目标,从而提高学生的整体水平;在学阶段加入老师引导机制,提高算法收敛速度;随后,加入自我调整阶段,学生根据心理控制源理论可被分为内控型和外控型,不同类型的学生对自身成绩采取不同的归因方式并采取相应措施。利用经典的基准测试函数对CPTLBO进行测验,结果表明改进算法在寻优精度和收敛速度方面具有优势。构建CPTLBO-ELM自来水供水量预测模型,采用CPTLBO算法优化极端学习机的输入权值和隐含层阈值参数,以提高模型的预测精度和泛化能力。仿真结果表明:用CPTLBO算法优化后的模型预测结果更准确。     

Robust Adaptive Gradient-Descent Training Algorithm for Recurrent Neural Networks in Discrete Time Domain

For a recurrent neural network (RNN), its transient response is a critical issue, especially for real-time signal processing applications. The conventional RNN training algorithms, such as backpropagation through time (BPTT) and real-time recurrent learning (RTRL), have not adequately addressed this problem because they suffer from low convergence speed. While increasing the learning rate may help to improve the performance of the RNN, it can result in unstable training in terms of weight divergence. Therefore, an optimal tradeoff between RNN training speed and weight convergence is desired. In this paper, a robust adaptive gradient-descent (RAGD) training algorithm of RNN is developed based on a novel RNN hybrid training concept. It switches the training patterns between standard real-time online backpropagation (BP) and RTRL according to the derived convergence and stability conditions. The weight convergence and $L_2$-stability of the algorithm are derived via the conic sector theorem. The optimized adaptive learning maximizes the training speed of the RNN for each weight update without violating the stability and convergence criteria. Computer simulations are carried out to demonstrate the applicability of the theoretical results.      

量子小波神经网络及其在漏钢预报中的应用

针对传统神经网络收敛速度慢,收敛精度低,以及用于模式识别泛化能力差的问题。提出了将量子神经网络与小波理论相结合的量子小波神经网络模型。该模型隐层量子神经元采用小波基函数的线性叠加作为激励函数,称之为多层小波激励函数,这样隐层神经元既能表示更多的状态和量级,又能提高网络收敛精度和速度。给出了网络学习算法。并以之在漏钢预报波形识别中的应用验证了该模型和学习算法的有效性。     

改进粒子群优化的极限学习机软测量建模方法

工业过程常含有显著的非线性、时变等复杂特性,传统的极限学习机有时无法充分利用数据信息,所建软测量模型预测性能较差。为了提高极限学习机的泛化能力和预测精度,提出一种改进粒子群优化的极限学习机软测量建模方法。首先,利用高斯函数正态分布的特点实现惯性权重的自适应更新,并线性变化学习因子以提高粒子群优化算法的收敛速度和搜索性能;然后将该算法用于优化极限学习机的惩罚系数和核宽,得到一组最优超参数;最后将该方法应用于脱丁烷塔过程软测量建模中。仿真结果表明,优化后的极限学习机模型预测精度有明显的提高,验证了所提方法不仅是可行的,而且具有良好的预测精度和泛化性能。     

一种通过反馈提高神经网络学习性能的新算法

为了有效提高前向神经网络的学习性能，需要从一个新的角度考虑神经网络的学习训练．基于此，提出了一种基于结果反馈的新算法——FBBP算法．将神经网络输入调整与通常的权值调整的反向传播算法结合起来，通过调整权值和输入矢量值的双重作用来最小化神经网络的误差函数．并通过几个函数逼近和模式分类问题的实例仿真，将FBBP算法与加动量项BP算法、最新的一种加快收敛的权值更新的算法进行了比较，来验证所提出的算法的有效性．实验结果表明，所提出的算法具有训练速度快和泛化能力高的双重优点，是一种非常有效的学习方法．     

Fast learning network: a novel artificial neural network with a fast learning speed

This paper proposes a novel artificial neural network called fast learning network (FLN). In FLN, input weights and hidden layer biases are randomly generated, and the weight values of the connection between the output layer and the input layer and the weight values connecting the output node and the input nodes are analytically determined based on least squares methods. In order to test the FLN validity, it is applied to nine regression applications, and experimental results show that, compared with support vector machine, back propagation, extreme learning machine, the FLN with much more compact networks can achieve very good generalization performance and stability at a very fast training speed and a quick reaction of the trained network to new observations. In addition, in order to further test the FLN validity, it is applied to model the thermal efficiency and NO _x emissions of a 330 WM coal-fired boiler and achieves very good prediction precision and generalization ability at a high learning speed.     

New dynamical optimal learning for linear multilayer FNN

This letter presents a new dynamical optimal learning (DOL) algorithm for three-layer linear neural networks and investigates its generalization ability. The optimal learning rates can be fully determined during the training process. The mean squared error (mse) is guaranteed to be stably decreased and the learning is less sensitive to initial parameter settings. The simulation results illustrate that the proposed DOL algorithm gives better generalization performance and faster convergence as compared to standard error back propagation algorithm.     

A robust recurrent simultaneous perturbation stochastic approximation training algorithm for recurrent neural networks

Training of recurrent neural networks (RNNs) introduces considerable computational complexities due to the need for gradient evaluations. How to get fast convergence speed and low computational complexity remains a challenging and open topic. Besides, the transient response of learning process of RNNs is a critical issue, especially for online applications. Conventional RNN training algorithms such as the backpropagation through time and real-time recurrent learning have not adequately satisfied these requirements because they often suffer from slow convergence speed. If a large learning rate is chosen to improve performance, the training process may become unstable in terms of weight divergence. In this paper, a novel training algorithm of RNN, named robust recurrent simultaneous perturbation stochastic approximation (RRSPSA), is developed with a specially designed recurrent hybrid adaptive parameter and adaptive learning rates. RRSPSA is a powerful novel twin-engine simultaneous perturbation stochastic approximation (SPSA) type of RNN training algorithm. It utilizes three specially designed adaptive parameters to maximize training speed for a recurrent training signal while exhibiting certain weight convergence properties with only two objective function measurements as the original SPSA algorithm. The RRSPSA is proved with guaranteed weight convergence and system stability in the sense of Lyapunov function. Computer simulations were carried out to demonstrate applicability of the theoretical results.     

基于Softplus+HKELM的彩色图像超分辨率算法

在极限学习机网络结构中,对于隐藏层节点,一般都采用Sigmoid激活函数。为了使基于极限学习机的彩色图像超分辨率算法具有更快的学习速度,更好的泛化能力,论文提出一种具有生物学激活模型且有一定稀疏表达能力的Soft plus激活函数与学习能力更好的HKELM相结合的彩色图像超分辨率算法。实验证明,与传统的极限学机相比,基于Soft plus+HKELM的彩色图像超分辨率算法不仅具有更好的泛化能力,而且最大程度降低了极限学习机训练的复杂度,提高了超分辨算法的实时应用性。     

基于免疫RBF神经网络的逆运动学求解

求解机械臂逆运动学问题可以采用神经网络来建立逆运动学模型,通过遗传算法或BP算法训练神经网络的权值从而得到问题的解,在求解精度和收敛速度上有待进一步改进。采用人工免疫原理对RBF网络训练数据集的泛化能力在线调整隐层结构,生成RBF网络隐层。当网络结构确定时,采用递推最小二乘法确定网络连接权值。由此对神经网络的网络结构和连接权进行自适应调整和学习。通过仿真可以看出,用免疫原理训练的神经网络收敛速度快,泛化能力强,可大幅提高机械臂逆运动学求解精度。     

ACO—BP在神经网络训练中的研究与应用

针对神经网络收敛速度慢、易于陷入局部最优等问题,可将蚁群算法与人工神经网络相融合的方法来解决,但容易出现训练时间与训练精度、泛化能力之间的矛盾.为解决上述矛盾,提出将蚁群优化算法与反向传播算法相融合共同完成神经网络训练的方法.算法首先采用蚁群优化算法对网络权值进行整体寻优,克服反向传播算法容易陷入局部最优的不足再以找到的较优的权值为初值,采用反向传播算法做进一步的寻优,克服单一训练网络时间较长、精度不高的缺点.最后对ACO-BP与反向传播算法进行了比较,给出两种算法在不同隐结点数目下的检验误差值和两种网络在矿选指标中的应用效果.通过对实验结果的分析.表明ACO-BP算法要优于反向传播算法.     

基于共轭梯度的极速学习机

极速学习机(ELM)由于具有较快的训练速度和较好的泛化能力而被广泛的应用到很多的领域,然而在计算数据样例个数较大的情况下,它的训练速度就会下降,甚至会出现程序报错,因此提出在ELM模型中用改进的共轭梯度算法代替广义逆的计算方法。实验结果表明,与求逆矩阵的ELM算法相比,在同等泛化精度的条件下,共轭梯度ELM有着更快的训练速度。通过研究发现:基于共轭梯度的极速学习机算法不需要计算一个大型矩阵的广义逆,而大部分广义逆的计算依赖于矩阵的奇异值分解(SVD),但这种奇异值分解对于阶数很高的矩阵具有很低的效率;因为已经证明共轭梯度算法可通过有限步迭代找到其解,所以基于共轭剃度的极速学习机有着较高的训练速度,而且也比较适用于处理大数据。     

一种改进BP网络学习算法

针对BP神经网络的原始算法收敛速率慢、学习精度低、训练过程易陷入局部极小值问题,为解决上述问题,提出一种以变学习率BP算法为基础的改进算法,通过区分隐层和输出层的学习率,并用交叉熵作性能函数,提高算法的学习精度和训练速度,并经过数学推导,得到改进箅法的实现公式.将改进算法应用于奇偶数判别问题进行仿真,仿真实验结果与其它类似的方法进行比较后,发现改进算法大大降低了网络迭代次数,缩短了网络的训练时间,提高了训练精度,验证了该算法的有效性.     

Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks

Online learning algorithms have been preferred in many applications due to their ability to learn by the sequentially arriving data. One of the effective algorithms recently proposed for training single hidden-layer feedforward neural networks (SLFNs) is online sequential extreme learning machine (OS-ELM), which can learn data one-by-one or chunk-by-chunk at fixed or varying sizes. It is based on the ideas of extreme learning machine (ELM), in which the input weights and hidden layer biases are randomly chosen and then the output weights are determined by the pseudo-inverse operation. The learning speed of this algorithm is extremely high. However, it is not good to yield generalization models for noisy data and is difficult to initialize parameters in order to avoid singular and ill-posed problems. In this paper, we propose an improvement of OS-ELM based on the bi-objective optimization approach. It tries to minimize the empirical error and obtain small norm of network weight vector. Singular and ill-posed problems can be overcome by using the Tikhonov regularization. This approach is also able to learn data one-by-one or chunk-by-chunk. Experimental results show the better generalization performance of the proposed approach on benchmark datasets.     

改进PSO-BP神经网络对储层参数的动态预测研究

为提高BP神经网络的收敛速度和泛化能力,防止其陷入局部最优值,在前人工作基础上对传统粒子群算法进行了改进,具体包括：设定最大限制速度、改变惯性权重因子和改进适应度函数,并把改进粒子群算法应用于BP神经网络权值和阈值的优化。之后利用改进粒子群算法优化的BP神经网络实现对储层参数的动态预测,具体步骤为：确定神经网络的输入、输出神经元,定量化时间参数[T],利用训练样本构建神经网络模型并进行检验。最后通过平均训练误差对仿真过程进行分析,结果表明改进PSO-BP算法的收敛性与泛化能力均优于BP算法和PSO-BP算法。     

基于面向对象自适应粒子群算法的神经网络训练*

针对传统的神经网络训练算法收敛速度慢和泛化性能低的缺陷,提出一种新的基于面向对象的自适应粒子群优化算法(OAPSO)用于神经网络的训练。该算法通过改进PSO的编码方式和自适应搜索策略以提高网络的训练速度与泛化性能,并结合Iris和Ionosphere分类数据集进行测试。实验结果表明：基于OAPSO算法训练的神经网络在分类准确率上明显优于BP算法及标准PSO算法,极大地提高了网络泛化能力和优化效果,具有快速全局收敛的性能。