A new architecture selection method based on tabu search for artificial neural networks

Artificial neural networks (ANN) have been used in various applications in recent years. One of these applications is time series forecasting. Although ANN produces accurate forecasts in many time series implementations, there are still some problems with using ANN. ANN consist of some components such as architecture structure, learning algorithm and activation function. These components have important effect on the performance of ANN. An important decision is the selection of architecture structure that consists of determining the numbers of neurons in the layers of a network. Therefore, various approaches have been proposed to determine the best ANN architecture in the literature. However, the most preferred method is still trial and error method for finding a good architecture. In this study, a new architecture selection method based on tabu search algorithm is proposed. In the implementation, five real time series are analyzed by using ANN and the proposed method is employed to select the best architecture. For the comparison, these time series are also forecasted by using ANN when trial and error method is utilized to determine the best architecture. As a result of the implementation, it is clearly seen that better results are obtained when the proposed method is used for the selection of architecture.     

A combination of Genetic Algorithm,Particle Swarm Optimization and Neural Network for palmprint recognition

In this study, a new approach to the palmprint recognition phase is presented. 2D Gabor filters are used for feature extraction of palmprints. After Gabor filtering, standard deviations are computed in order to generate the palmprint feature vector. Genetic Algorithm-based feature selection is used to select the best feature subset from the palmprint feature set. An Artificial Neural Network (ANN) based on hybrid algorithm combining Particle Swarm Optimization (PSO) algorithm with back-propagation algorithms has been applied to the selected feature vectors for recognition of the persons. Network architecture and connection weights of ANN are evolved by a PSO method, and then, the appropriate network architecture and connection weights are fed into ANN. Recognition rate equal to 96% is obtained by using conjugate gradient descent algorithm.

     

基于WS小世界模型的蝙蝠优化算法

蝙蝠算法是在对微型蝙蝠回声观察研究的基础上发现蝙蝠回声和优化目标功能之间的关系而提出的一种新算法。蝙蝠算法具有强大的搜索性能,但是其局部搜索相对简单,个体间缺乏信息互通,搜索能力差。尽管目前也提出了一些相关改进算法,但高维优化方面较少涉及。考虑到蝙蝠群体中个体相互联系与作用的方式有动态复杂的感知网络结构,具有“小世界”特性,所以首先把有“小世界”特性的WS小世界模型引入蝙蝠算法,利用WS小世界模型断边重连的特点生成动态的邻域结构,这种邻域结构能够提高整体的搜索能力。实例验证表明借助一般的蝙蝠算法可以进行局部搜索。     

Evolving neural network using real coded genetic algorithm for permeability estimation of the reservoir

In this work we investigate how artificial neural network (ANN) evolution with genetic algorithm (GA) improves the reliability and predictability of artificial neural network. This strategy is applied to predict permeability of Mansuri Bangestan reservoir located in Ahwaz, Iran utilizing available geophysical well log data. Our methodology utilizes a hybrid genetic algorithm–neural network strategy (GA–ANN). The proposed algorithm combines the local searching ability of the gradient–based back-propagation (BP) strategy with the global searching ability of genetic algorithms. Genetic algorithms are used to decide the initial weights of the gradient decent methods so that all the initial weights can be searched intelligently. The genetic operators and parameters are carefully designed and set avoiding premature convergence and permutation problems. For an evaluation purpose, the performance and generalization capabilities of GA–ANN are compared with those of models developed with the common technique of BP. The results demonstrate that carefully designed genetic algorithm-based neural network outperforms the gradient descent-based neural network.     

Fast and improved backpropagation learning of multi-layer artificial neural network using adaptive activation function

In the conventional backpropagation (BP) learning algorithm used for the training of the connecting weights of the artificial neural network (ANN), a fixed slope−based sigmoidal activation function is used. This limitation leads to slower training of the network because only the weights of different layers are adjusted using the conventional BP algorithm. To accelerate the rate of convergence during the training phase of the ANN, in addition to updates of weights, the slope of the sigmoid function associated with artificial neuron can also be adjusted by using a newly developed learning rule. To achieve this objective, in this paper, new BP learning rules for slope adjustment of the activation function associated with the neurons have been derived. The combined rules both for connecting weights and slopes of sigmoid functions are then applied to the ANN structure to achieve faster training. In addition, two benchmark problems: classification and nonlinear system identification are solved using the trained ANN. The results of simulation-based experiments demonstrate that, in general, the proposed new BP learning rules for slope and weight adjustments of ANN provide superior convergence performance during the training phase as well as improved performance in terms of root mean square error and mean absolute deviation for classification and nonlinear system identification problems.     

自适应多普勒补偿与变异选择的蝙蝠算法

为克服蝙蝠算法在高维优化问题上求解精度低和早熟收敛的缺点,提出一种改进的蝙蝠算法。首先根据蝙蝠相对猎物距离的远近程度,对频率引入自适应多普勒补偿策略,并结合速度偏移机制修正飞行方向,产生靠近最优个体的新位置;其次对最优个体构造自适应变异选择策略,先利用柯西变异产生的较大步长摆脱局部极值的束缚,后利用高斯变异产生的较小步长精细搜寻最优区域;最后通过调整响度和脉冲发射率,平衡算法的全局探索和局部开发能力。从理论上分析了算法的收敛性和运算复杂性,对12个标准函数在不同维度下进行仿真实验,并与近年来其他蝙蝠算法进行比较,结果表明改进的算法在求解高维优化问题上具有较优的收敛速度和精度。     

An efficient hybrid learning algorithm for neural network–based speech recognition systems on FPGA chip

This paper implemented an artificial neural network (ANN) on a field programmable gate array (FPGA) chip for Mandarin speech measurement and recognition of nonspecific speaker. A three-layer hybrid learning algorithm (HLA), which combines genetic algorithm (GA) and steepest descent method, was proposed to fulfill a faster global search of optimal weights in ANN. Some other popular evolutionary algorithms, such as differential evolution, particle swarm optimization and improve GA, were compared to the proposed HLA. It can be seen that the proposed HLA algorithm outperforms the other algorithms. Finally, the designed system was implemented on an FPGA chip with an SOC architecture to measure and recognize the speech signals.     

基于动态扫描周期的多属性垂直切换算法

当终端处于动态的异构无线网络环境下,为在减少终端能耗的同时,及时检测到周围可用的无线网络并从中快速选择一个合适的接入网络,提出一种基于动态扫描周期的多属性垂直切换算法。在网络发现阶段根据终端接收到的当前接入网信号强度和终端的移动速度动态调整扫描周期,减少终端接口激活次数。在网络选择阶段采用多属性判决法综合考虑多种网络属性,采用层次分析法和熵权法根据业务类型不同对各参考属性进行权值分配,引入微小阈值对网络进行过滤。仿真结果表明,与现有算法相比,所提算法能够在减少终端能耗的同时,及时地为终端选择出最佳的接入网络。     

A hybrid particle swarm optimization and its application in neural networks

In this paper, a novel particle swarm optimization model for radial basis function neural networks (RBFNN) using hybrid algorithms to solve classification problems is proposed. In the model, linearly decreased inertia weight of each particle (ALPSO) can be automatically calculated according to fitness value. The proposed ALPSO algorithm was compared with various well-known PSO algorithms on benchmark test functions with and without rotation. Besides, a modified fisher ratio class separability measure (MFRCSM) was used to select the initial hidden centers of radial basis function neural networks, and then orthogonal least square algorithm (OLSA) combined with the proposed ALPSO was employed to further optimize the structure of the RBFNN including the weights and controlling parameters. The proposed optimization model integrating MFRCSM, OLSA and ALPSO (MOA-RBFNN) is validated by testing various benchmark classification problems. The experimental results show that the proposed optimization method outperforms the conventional methods and approaches proposed in recent literature.     

Neural network classifier optimization using Differential Evolution with Global Information and Back Propagation algorithm for clinical datasets

A Computer-Aided Diagnostic (CAD) system that uses Artificial Neural Network (ANN) trained by drawing in the relative advantages of Differential Evolution (DE), Particle Swarm Optimization (PSO) and gradient descent based backpropagation (BP) for classifying clinical datasets is proposed. The DE algorithm with a modified best mutation operation is used to enhance the search exploration of PSO. The ANN is trained using PSO and the global best value obtained is used as a seed by the BP. Local search is performed using BP, in which the weights of the Neural Network (NN) are adjusted to obtain an optimal set of NN weights. Three benchmark clinical datasets namely, Pima Indian Diabetes, Wisconsin Breast Cancer and Cleveland Heart Disease, obtained from the University of California Irvine (UCI) machine learning repository have been used. The performance of the trained neural network classifier proposed in this work is compared with the existing gradient descent backpropagation, differential evolution with backpropagation and particle swarm optimization with gradient descent backpropagation algorithms. The experimental results show that DEGI-BP provides 85.71% accuracy for diabetes, 98.52% for breast cancer and 86.66% for heart disease datasets. This CAD system can be used by junior clinicians as an aid for medical decision support.     

混合粒子群优化算法优化前向神经网络结构和参数*

提出了综合利用粒子群优化算法（PSO）和离散粒子群优化算法（D-PSO）同时优化前向神经网络结构和参数的新方法。该算法使用离散粒子群优化算法优化神经网络连接结构,用多维空间中0或1取值的粒子来描述所有可能的神经网络连接,同时使用粒子群优化算法优化神经网络权值。将经过该算法训练的神经网络应用于故障诊断,能够有效消除冗余连接结构对网络诊断能力的影响。仿真试验的结果表明,相比遗传算法等其他算法,该算法能够有效改善神经网络结构和参数的优化效率,提高故障模式识别的准确率。     

Modified Smith predictor with a robust disturbance reduction scheme for linear systems with small time delays

This paper presents a robust disturbance reduction scheme using an artificial neural network (ANN) for linear systems with small time delays. It is assumed that the nominal linear systems are stable, minimum phase and relative degree one systems. The proposed structure is an integration of a modified Smith predictor and an ANN‐based disturbance reduction scheme. Unlike other disturbance rejection methods, the proposed approach does not require information about unknown load disturbance frequencies. An ANN is used to approximate the unknown load disturbances and to enhance the robustness of the proposed disturbance reduction scheme against modelling errors in the estimated time delay and the process model. Connective weights of the ANN are trained on‐line using a back‐propagation algorithm until uncertainties resulting from unknown load disturbances and modelling errors are minimized. The simulation results show the effectiveness of the presented disturbance reduction scheme for controlling linear delay systems subjected to step or periodic unknown load disturbances.     

改进利益驱动神经网络在欺诈信息的应用研究

为了研究个体收益和代价实现总体净收益的最大化问题,提出了利益驱动的人工神经网络（ANN）分类方法。该方法引入了惩罚函数,根据实例不同的重要程度对不同实例的误分类给予可变惩罚,并在之后对净利益进行最大化处理。为了生成对个体的惩罚,参照每个实例的收益,通过改变函数值对误差平方和函数进行了修改,提出了七个不同版本的ANN模型。两个欺诈信息的实验结果表明,与原ANN、决策树和朴素贝叶斯分类器相比,所提模型的不同版本在净利润项上的性能优于其他方法,而且能够针对不同的数据集采用不同的权值生成方式。     

一种新的自适应粒子群优化算法

基于粒子分工与合作的思想,提出一种自适应粒子群优化(PSO)算法。该算法为不同的粒子分配不同的任务,对性能较好的粒子使用较大的惯性权,对性能较差的粒子采用较小的惯性权,加速系数根据惯性权自适应调整。将标准PSO算法中的全局最优位置与个体最优位置分别替换为相关个体最优位置的加权平均,更好地平衡了算法的全局与局部搜索能力,提高了算法的多样性与搜索效率。5个经典测试函数的仿真结果及与其他PSO算法的比较结果验证了该算法的有效性。     

无线局域网MAC层协作通信改进方案

针对现有无线局域网MAC层协作通信的不足,提出一种根据即时信道信息选择最优中继的策略,引入固定退避时隙和改进的随机退避时隙解决中继碰撞问题,增加CTC(Compete to Clear,取消竞争)控制帧来协调源-中继-目的节点三者之间的关系。本文以吞吐量和服务延迟为性能指标对新算法和现有MAC算法进行仿真和比较,结果表明新算法能够带来网络性能的显著提升。     

Computational modification of neural systems using a novel stochastic search scheme,namely evaporation rate-based water cycle algorithm: an application in geotechnical issues

Recent studies have demonstrated the high efficiency of metaheuristic algorithms for various optimization engineering problems. The main focus of the present study is to apply a novel notion of stochastic search methods, namely evaporation rate-based water cycle algorithm (ER-WCA) to the problem of soil shear strength (SSS) prediction. The ER-WCA, as the name indicates, is a modified version of the water cycle algorithm that is used to computationally modify an artificial neural network (ANN) for the mentioned purpose. The sensitivity analysis showed that the most proper values for the number of rivers + sea and the population size are 5 and 300, respectively. The performance of the ER-WCA–ANN hybrid is compared to an ANN typically trained by the Levenberg–Marquardt algorithm to evaluate the effectiveness of the proposed metaheuristic technique. The findings showed that incorporation of the ER-WCA results in reducing the root-mean-square error by 5.87% and 4.92% in the training and testing phases, respectively. Meanwhile, the coefficient of determination rose from 84.27 to 86.11% and from 78.80 to 80.83% in these phases. It indicates that the weights and biases suggested by the ER-WCA can construct a considerably more reliable ANN. Therefore, the introduced method is recommended for practical uses in the early prediction of the SSS in civil engineering projects.

     

Fast Affine Projection Algorithms for Filtered-x Multichannel Active Noise Control

In recent years, affine projection algorithms have been proposed for adaptive system applications as an efficient alternative to the slow convergence speed of least mean square (LMS)-type algorithms. Whereas much attention has been focused on the development of efficient versions of affine projection algorithms for echo cancellation applications, the similar adaptive problem presented by active noise control (ANC) systems has not been studied so deeply. This paper is focused on the necessity to reduce even more the computational complexity of affine projection algorithms for real-time ANC applications. We present some alternative efficient versions of existing affine projection algorithms that do not significantly degrade performance in practice. Furthermore, while in the ANC context the commonly used affine projection algorithm is based on the modified filtered-x structure, an efficient affine projection algorithm based on the (nonmodified) conventional filtered-x structure, as well as efficient methods to reduce its computational burden, are discussed throughout this paper. Although the modified filtered-x scheme exhibits better convergence speed than the conventional filtered-x structure and allows recovery of all the signals needed in the affine projection algorithm for ANC, the conventional filtered-x scheme provides a significant computational saving, avoiding the additional filtering needed by the modified filtered-x structure. In this paper, it is shown that the proposed efficient versions of affine projection algorithms based on the conventional filtered-x structure show good performance, comparable to the performance exhibited by the efficient approaches of modified filtered-x affine projection algorithms, and also achieve meaningful computational savings. Experimental results are presented to validate the use of the algorithms introduced in the paper for practical applications.      

A hybrid algorithm for artificial neural network training

Artificial neural network (ANN) training is one of the major challenges in using a prediction model based on ANN. Gradient based algorithms are the most frequent training algorithms with several drawbacks. The aim of this paper is to present a method for training ANN. The ability of metaheuristics and greedy gradient based algorithms are combined to obtain a hybrid improved opposition based particle swarm optimization and a back propagation algorithm with the momentum term. Opposition based learning and random perturbation help population diversification during the iteration. Use of time-varying parameter improves the search ability of standard PSO, and constriction factor guarantees particles convergence. Since several contingent local minima conditions may happen in the weight space, a new cross validation method is proposed to prevent overfitting. Effectiveness and efficiency of the proposed method are compared with several other famous ANN training algorithms on the various benchmark problems.     

A weight initialization method for improving training speed in feedforward neural network

An algorithm for determining the optimal initial weights of feedforward neural networks based on the Cauchy's inequality and a linear algebraic method is developed. The algorithm is computational efficient. The proposed method ensures that the outputs of neurons are in the active region and increases the rate of convergence. With the optimal initial weights determined, the initial error is substantially smaller and the number of iterations required to achieve the error criterion is significantly reduced. Extensive tests were performed to compare the proposed algorithm with other algorithms. In the case of the sunspots prediction, the number of iterations required for the network initialized with the proposed method was only 3.03% of those started with the next best weight initialization algorithm.     

A filter model for feature subset selection based on genetic algorithm

This paper describes a novel feature subset selection algorithm, which utilizes a genetic algorithm (GA) to optimize the output nodes of trained artificial neural network (ANN). The new algorithm does not depend on the ANN training algorithms or modify the training results. The two groups of weights between input-hidden and hidden-output layers are extracted after training the ANN on a given database. The general formula for each output node (class) of ANN is then generated. This formula depends only on input features because the two groups of weights are constant. This dependency is represented by a non-linear exponential function. The GA is involved to find the optimal relevant features, which maximize the output function for each class. The dominant features in all classes are the features subset to be selected from the input feature group.