Optimization of wavelet neural networks with the firefly algorithm for approximation problems

The effectiveness of swarm intelligence has been proven to be at the heart of various optimization problems. In this study, a recently developed nature-inspired algorithm, specifically the firefly algorithm (FA), is integrated in the learning strategy of wavelet neural networks (WNNs). The FA, which systematically optimizes the initial location of the translation parameters for WNNs, has reduced the number of hidden nodes while simultaneously improved the generalization capability of WNNs significantly. The applicability of the proposed model was demonstrated through empirical simulations for function approximation study, with both synthetic and real-world data. Performance assessment demonstrated its enhancement over the K-means clustering and random initialization approaches, as well as to the other neural network models reported in the literature, whereby a noteworthy decrease in the approximation error was observed.

     

Adaptive annealing learning algorithm-based robust wavelet neural networks for function approximation with outliers

In this study, a robust wavelet neural network (WNN) is proposed to approximate functions with outliers. In the proposed methodology, firstly, support vector machine with wavelet kernel function (WSVM) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs. Then, an adaptive annealing learning algorithm (AALA) is adopted to accommodate the translations, the dilations, and the weights of the WNNs. In the learning procedure, the AALA is proposed to overcome the problems of initialization and the cut-off points in the robust learning algorithm. Hence, when an initial structure of the WNNs is determined by a support vector regression (SVR) approach, the WNNs with AALA (AALA-WNNs) have fast convergence speed and can robust against outliers. Two examples are simulated to verify the feasibility and efficiency of the proposed algorithm.     

Identification of nonlinear systems with outliers using wavelet neural networks based on annealing dynamical learning algorithm

This paper presents an annealing dynamical learning algorithm (ADLA) to train wavelet neural networks (WNNs) for identifying nonlinear systems with outliers. In ADLA–WNNs, wavelet-based support vector regression (WSVR) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs due to the similarity between WSVR and WNNs. After initialization, ADLA with nonlinear time-varying learning rates is applied to train the WNNs. In the ADLA, the determination of the learning rates would be a key work for the trade-off between stability and speed of convergence. A computationally efficient optimization method, particle swarm optimization (PSO), is adopted to find the optimal learning rates to overcome the stagnation in the training procedure of WNNs. Due to the advantages of WSVR and ADLA (WSVR–ADLA), the WSVR-based ADLA–WNNs (WSVR–ADLA–WNNs) can robust against outliers and achieve the promising efficiency of system identifications. Three examples are simulated to confirm the performance of the proposed algorithm. From the simulated results, the feasibility and superiority of the proposed WSVR–ADLA–WNNs for identifying nonlinear systems with artificial outliers are verified.     

Reliable multiclass cancer classification of microarray gene expression profiles using an improved wavelet neural network

Properly designing a wavelet neural network (WNN) is crucial for achieving the optimal generalization performance. In this paper, two different approaches were proposed for improving the predictive capability of WNNs. First, the types of activation functions used in the hidden layer of the WNN were varied. Second, the proposed enhanced fuzzy c-means clustering algorithm—specifically, the modified point symmetry-based fuzzy c-means (MSFCM) algorithm—was employed in selecting the locations of the translation vectors of the WNN. The modified WNN was then applied to heterogeneous cancer classification using four different microarray benchmark datasets. The comparative experimental results showed that the proposed methodology achieved an almost 100% classification accuracy in multiclass cancer prediction, leading to superior performance with respect to other clustering algorithms. Subsequently, performance comparisons with other classifiers were made. An assessment analysis showed that this proposed approach outperformed most of the other classifiers.     

基于小波神经网络的混沌时间序列分析与相空间重构

探讨了小波神经网络在混沌时间序列分析与相空间重构中的应用,通过混沌时间序列单步预测与多步预测的例子,比较了小波神经网络与MLP的逼近和收敛性能,对最近提出的一种多分辨率学习策略进行了改进,利用连续3次样条小和正交Daubechies小波代替Haar小波对时间序列做小波分解;用改进的学习算法训练网络并应用到混沌序列相空间重构中,实验结果表明,小波神经网络比MLP和ARMA模型具有更强大的逼近能力,因而十分适合应用于时间序列分析中;多分辨率学习算法可作为分析复杂混沌时间序列的一种重要工具。     

基于影响度剪枝的极端学习机分类算法研究

针对极端学习机(ELM)网络规模控制问题,从剪枝思路出发,提出了一种基于影响度剪枝的ELM分类算法。利用ELM网络单个隐节点连接输入层和输出层的权值向量、该隐节点的输出、初始隐节点个数以及训练样本个数,定义单个隐节点相对于整个网络学习的影响度,根据影响度判断隐节点的重要性并将其排序,采用与ELM网络规模相匹配的剪枝步长删除冗余节点,最后更新隐含层与输入层和输出层连接的权值向量。通过对多个UCI机器学习数据集进行分类实验,并将提出的算法与EM-ELM、PELM和ELM算法相比较,结果表明,该算法具有较高的稳定性和测试精度,训练速度较快,并能有效地控制网络规模。     

基于Gabor小波和HMM的人像鉴别算法

探讨了利用Gabor小波和隐马尔可夫模型（HMM）进行人脸识别的方法,首先对人脸图像进行多分辨率的Gabor小波变换;然后在图像上放置一组网格结点,每个结点用该结点处的多尺度Gabor幅度特征描述,采用独立元分析法对每个结点进行去相关和降维;最后形成特征结,把每个特征结作为观测向量,对隐马尔可夫模型进行训练,并将优化的模型参数用于人脸识别,ORL人脸库的实验结果表明,该方法识别率高,工程上易于应用。     

改进BP神经网络的光伏系统发电功率预测

为了提高光伏发电功率的预测精度,提出一种改进BP神经网络的光伏发电功率预测模型.首先采用包括室外温度、光照辐射量、风速等作为输入层节点,交流发电功率作为输出节点,引入RMSE作为衡量最优模型指标,确定了隐含层节点数,然后采用BP神经网络对其进行学习,并采用布谷鸟搜索算法对BP神经网络进行优化,最后采用仿真实验对其有效性进行测试.结果表明,改进神经网络提高了光伏发电功率预测精度,具有一定的推广价值.     

基于灵敏度分析法的ELM剪枝算法

针对极端学习机(ELM)网络结构设计问题,提出基于灵敏度分析法的ELM剪枝算法.利用隐含层节点输出和相对应的输出层权值向量,定义学习残差对于隐含层节点的灵敏度和网络规模适应度,根据灵敏度大小判断隐含层节点的重要性,利用网络规模适应度确定隐含层节点个数,删除重要性较低的节点.仿真结果表明,所提出的算法能够较为准确地确定与学习样本相匹配的网络规模,解决了ELM网络结构设计问题.     

Fault diagnosis of rolling bearing based on back propagation neural network optimized by cuckoo search algorithm

In order to improve the accuracy of rolling bearing fault diagnosis in mechanical equipment, a new fault diagnosis method based on back propagation neural network optimized by cuckoo search algorithm is proposed. This method use the global search ability of the cuckoo search algorithm to constantly search for the best weights and thresholds, and then give it to the back propagation neural network. In this paper, wavelet packet decomposition is used for feature extraction of vibration signals. The energy values of different frequency bands are obtained through wavelet packet decomposition, and they are input as feature vectors into optimized back propagation neural network to identify different fault types of rolling bearings. Through the three sets of simulation comparison experiments of Matlab, the experimental results show that, Under the same conditions, compared with the other five models, the proposed back propagation neural network optimized by cuckoo search algorithm has the least number of training iterations and the highest diagnostic accuracy rate. And in the complex classification experiment with the same fault location but different bearing diameters, the fault recognition correct rate of the back propagation neural network optimized by cuckoo search algorithm is 96.25%.

     

Application of bacterial foraging technique trained artificial and wavelet neural networks in load forecasting

A new load forecasting (LF) approach using bacterial foraging technique (BFT) trained wavelet neural network (WNN) is proposed in this paper. Artificial neural network (ANN) is combined with wavelet transform called wavelet neural network is applied for LF. The parameters of translation and dilation in the wavelet nodes and the weighting factors in the weighting nodes are tuned using BFT optimization. With the advantages of global search abilities of BFT as well as the multiresolution and localizing natures of wavelets, the networks are constructed which identifies the inherent non-linear characteristics of power system loads. The proposed approach is validated with Tamil Nadu Electricity Board (TNEB) system, India. The comparison of Delta Rule and BFT-based LF for different periods are depicted with their mean absolute percentage errors (MAPE).     

一种结构自适应的径向基函数神经网络

提出了一种新的结构自适应的径向基函数(RBF)神经网络模型。在该模型中,自组织映射(SOM)神经网络作为聚类网络,采用无监督学习算法对输入样本进行自组织分类,并将分类中心及其对应的权值向量传递给RBF神经网络,分别作为径向基函数的中心和相应的权值向量;RBF神经网络作为基础网络,采用高斯函数实现输入层到隐层的非线性映射,输出层则采用有监督学习算法训练网络的权值,从而实现输入层到输出层的非线性映射。通过对字母数据集进行仿真,表明该网络具有较好的性能。     

Construction of a new adaptive wavelet network and its learning algorithm

A new adaptive learning algorithm for constructing and training wavelet networks is proposed based on the time-frequency localization properties of wavelet frames and the adaptive projection algorithm. The exponential convergence of the adaptive projection algorithm in finite-dimensional Hilbert spaces is constructively proved, with exponential decay ratios given with high accuracy. The learning algorithm can sufficiently utilize the time-frequency information contained in the training data, iteratively determines the number of the hidden layer nodes and the weights of wavelet networks, and solves the problem of structure optimization of wavelet networks. The algorithm is simple and efficient, as illustrated by examples of signal representation and denoising.     

利用基于小波特征提取的网络模型解析色谱重叠峰

提出了一种新的色谱重叠峰解析模型——基于小波特征提取的RBF神经网络模型。首先利用小波变换奇异性的检测原理,从原始色谱信号中提取特征点,这些特征点蕴含了反映色谱峰形状的信息,包括重叠峰个数、保留时间等信息。由小波变换获得的特征点来确定RBF网络的隐节点数目和网络参数的初值,即将拐点对数作为隐节点数目,将峰宽估计值作为输出层连接权的初值,将峰高估计值作为隐节点宽度的初值。再用RBF网络来拟合原始重叠色谱信号,梯度下降法训练后获得的网络参数作为解析结果,实现了重叠色谱峰的分离。实验结果表明:本方法快速、准确、可靠,能有效解析未知组分数的重叠峰。     

Nonlinear systems control using self-constructing wavelet networks

This paper describes a self-constructing wavelet network (SCWN) controller for nonlinear systems control. The proposed SCWN controller has a four-layer structure. We adopt the orthogonal wavelet functions as its node functions. An online learning algorithm, structure learning and parameter learning, allows the dynamic determining of the number of wavelet bases, and adjusting the shape of the wavelet bases and the connection weights. The SCWN controller is a highly autonomous system. Initially, there are no hidden nodes. They are created and begin to grow as learning proceeds. Computer simulations have been conducted to illustrate the performance and applicability of the proposed learning scheme.     

前馈神经网隐层节点的动态删除法

本文首先针对ＢＰ算法中存在的缺陷对误差函数作了简单的修改，使网络的收敛速度比原来的大大提高，此外本文提提出了一种基于线性回归分析算法来确定隐层节点数。当已训练好的网络具有过多的隐层单元，可以用这种算法来计算隐层节点输出之间的线性相关性，并估计多余隐层单元数目，然后删除这部分多余的节点，就能获得一个合适的网络结构。     

求解连续函数优化问题的合作协同进化布谷鸟搜索算法

为改善布谷鸟搜索算法求解连续函数优化问题的性能,提出合作协同进化的布谷鸟搜索算法.改进算法通过应用合作协同进化框架,将种群的解向量分解成若干子向量,并构成相应子群体.利用标准布谷鸟算法更新各子群体的解向量.各子群体为其它子群体提供最优个体,组合成问题解向量并完成子群体评价.经10个测试函数实验仿真,结果说明改进算法能有效改善求解连续函数优化问题的性能.同时,针对连续函数优化问题,该算法与其它算法相比是有竞争力的优化算法.     

基于资源分配网络和语义特征选取的文本分类

针对资源分配网络(RAN)算法存在隐含层节点受初始学习数据影响大、收敛速度低等问题,提出一种新的RAN学习算法。通过均值算法确定初始隐含层节点,在原有的“新颖性准则”基础上增加RMS窗口,更好地判定隐含层节点是否增加。同时,采用最小均方(LMS)算法与扩展卡尔曼滤波器(EKF)算法相结合调整网络参数,提高算法学习速度。由于基于词向量空间文本模型很难处理文本的高维特性和语义复杂性,为此通过语义特征选取方法对文本输入空间进行语义特征的抽取和降维。实验结果表明,新的RAN学习算法具有学习速度快、网络结构紧凑、分类效果好的优点,而且,在语义特征选取的同时实现了降维,大幅度减少文本分类时间,有效提高了系统分类准确性。     

城市灾区无人机网络自适应覆盖优化算法

城市灾区中,地面用户节点的移动特性使得应急网络覆盖成为难题。针对城市灾区移动用户节点的应急网络覆盖优化问题,提出一种无人机网络自适应覆盖优化算法。对布谷鸟搜索算法进行改进,并对目标函数进行优化调整,将城市灾区地面用户节点的移动模型应用于改进的布谷鸟算法模拟中,最终实现对城市灾区重点区域移动用户的自适应覆盖优化。仿真结果表明,所提算法与相同实验环境下的标准布谷鸟算法（CSA）和模拟退火算法（SAA）相比,对重点区域的覆盖率分别提升了2.98个百分点和1.87个百分点。多次实验表明无人机网络的覆盖率、连通性及路径损耗稳定,且随着仿真时间变化,应急网络的性能稳定。证明了该算法不仅能够对城市灾区移动节点提供稳定的动态网络覆盖,有较强的全局以及局部寻优能力且能够更加有效地提高对重点区域的覆盖率。