Enhancement of artificial neural network learning using centripetal accelerated particle swarm optimization for medical diseases diagnosis

In recent decades, artificial neural networks (ANNs) have been extensively applied in different areas such as engineering, medicine, business, education, manufacturing and so on. Nowadays, ANNs are as a hot research in medicine especially in the fields of medical disease diagnosis. To have a high efficiency in ANN, selection of an appropriate architecture and learning algorithm is very important. ANN learning is a complex task and an efficient learning algorithm has a significant role to enhance ANN performance. In this paper, a new meta-heuristic algorithm, centripetal accelerated particle swarm optimization (CAPSO), is applied to evolve the ANN learning and accuracy. The algorithm is based on an improved scheme of particle swarm algorithm and Newton’s laws of motion. The hybrid learning of CAPSO and multi-layer perceptron (MLP) network, CAPSO-MLP, is used to classify the data of nine standard medical datasets of Hepatitis, Heart Disease, Pima Indian Diabetes, Wisconsin Prognostic Breast Cancer, Parkinson’s disease, Echocardiogram, Liver Disorders, Laryngeal 1 and Acute Inflammations. The performance of CAPSO-MLP is compared with those of PSO, gravitational search algorithm and imperialist competitive algorithm on MLP. The efficiency of methods are evaluated based on mean square error, accuracy, sensitivity, specificity, area under the receiver operating characteristics curve and statistical tests of \(t\) -test and Wilcoxon’s signed ranks test. The results indicate that CAPSO-MLP provides more effective performance than the others for medical disease diagnosis especially in term of unseen data (testing data) and datasets with high missing data values.     

Neural identification of dynamic systems on FPGA with improved PSO learning

This work introduces hardware implementation of artificial neural networks (ANNs) with learning ability on field programmable gate array (FPGA) for dynamic system identification. The learning phase is accomplished by using the improved particle swarm optimization (PSO). The improved PSO is obtained by modifying the velocity update function. Adding an extra term to the velocity update function reduced the possibility of stucking in a local minimum. The results indicates that ANN, trained using improved PSO algorithm, converges faster and produces more accurate results with a little extra hardware utilization cost.     

Seeker optimization algorithm for tuning the structure and parameters of neural networks

Seeker optimization algorithm (SOA) is a novel population-based heuristic stochastic search algorithm, which is based on the concept of simulating the act of human searching. In the SOA, the search direction is determined by seeker’s egotistic behavior, altruistic behavior and pro-activeness behavior, while step length is given by uncertainty reasoning behavior. In this paper, the application of the SOA to tuning the structures and parameters of artificial neural networks (ANNs) is presented as a new evolutionary method of ANN training. Simulation experiments for pattern classification and function approximation are performed. The comparisons of the SOA between BP algorithms and other evolutionary algorithms (EAs) are studied. The simulation results show that the performance of the SOA is better than or, at least, equivalent to that of other EAs (i.e., DE and two variations of PSO) for all the listed problems. Moreover, the ANNs with link switches trained by the SOA can provide better or comparable learning capabilities with much less number of links than ones by BP algorithms (i.e., GDX, RP, OSS and SCG). Hence, SOA can simultaneously tune the structures and the weight values, and, though SOA is more computationally intensive, it is believed that SOA will become a promising candidate for training ANNs.     

Mutation-based genetic neural network

Evolving gradient-learning artificial neural networks (ANNs) using an evolutionary algorithm (EA) is a popular approach to address the local optima and design problems of ANN. The typical approach is to combine the strength of backpropagation (BP) in weight learning and EA's capability of searching the architecture space. However, the BP's "gradient descent" approach requires a highly computer-intensive operation that relatively restricts the search coverage of EA by compelling it to use a small population size. To address this problem, we utilized mutation-based genetic neural network (MGNN) to replace BP by using the mutation strategy of local adaptation of evolutionary programming (EP) to effect weight learning. The MGNN's mutation enables the network to dynamically evolve its structure and adapt its weights at the same time. Moreover, MGNN's EP-based encoding scheme allows for a flexible and less restricted formulation of the fitness function and makes fitness computation fast and efficient. This makes it feasible to use larger population sizes and allows MGNN to have a relatively wide search coverage of the architecture space. MGNN implements a stopping criterion where overfitness occurrences are monitored through "sliding-windows" to avoid premature learning and overlearning. Statistical analysis of its performance to some well-known classification problems demonstrate its good generalization capability. It also reveals that locally adapting or scheduling the strategy parameters embedded in each individual network may provide a proper balance between the local and global searching capabilities of MGNN.     

混合粒子群优化算法优化前向神经网络结构和参数

提出了综合利用粒子群优化算法（PSO）和离散粒子群优化算法（D-PSO）同时优化前向神经网络结构和参数的新方法。该算法使用离散粒子群优化算法优化神经网络连接结构,用多维空间中0或1取值的粒子来描述所有可能的神经网络连接,同时使用粒子群优化算法优化神经网络权值。将经过该算法训练的神经网络应用于故障诊断,能够有效消除冗余连接结构对网络诊断能力的影响。仿真试验的结果表明,相比遗传算法等其他算法,该算法能够有效改善神经网络结构和参数的优化效率,提高故障模式识别的准确率。     

Artificial neural networks with evolutionary instance selection for financial forecasting

In this paper, I propose a genetic algorithm (GA) approach to instance selection in artificial neural networks (ANNs) for financial data mining. ANN has preeminent learning ability, but often exhibit inconsistent and unpredictable performance for noisy data. In addition, it may not be possible to train ANN or the training task cannot be effectively carried out without data reduction when the amount of data is so large. In this paper, the GA optimizes simultaneously the connection weights between layers and a selection task for relevant instances. The globally evolved weights mitigate the well-known limitations of gradient descent algorithm. In addition, genetically selected instances shorten the learning time and enhance prediction performance. This study applies the proposed model to stock market analysis. Experimental results show that the GA approach is a promising method for instance selection in ANN.     

A combination of Genetic Algorithm,Particle Swarm Optimization and Neural Network for palmprint recognition

In this study, a new approach to the palmprint recognition phase is presented. 2D Gabor filters are used for feature extraction of palmprints. After Gabor filtering, standard deviations are computed in order to generate the palmprint feature vector. Genetic Algorithm-based feature selection is used to select the best feature subset from the palmprint feature set. An Artificial Neural Network (ANN) based on hybrid algorithm combining Particle Swarm Optimization (PSO) algorithm with back-propagation algorithms has been applied to the selected feature vectors for recognition of the persons. Network architecture and connection weights of ANN are evolved by a PSO method, and then, the appropriate network architecture and connection weights are fed into ANN. Recognition rate equal to 96% is obtained by using conjugate gradient descent algorithm.

     

粒子群优化算法中惯性权值调整的一种新策略

惯性权值的设置对粒子群优化（PSO）算法的性能起着关键作用,现有的基于惯性权值的改进算法提高了算法的性能,但都把惯性权值作为全局参数,很难控制算法的搜索能力。本文在充分分析惯性权值的关键作用基础上给出一种新的惯性权值调整策略及其相应的粒子群优化算法,使用不同的惯性权值更新同一代种群。测试结果表明,新算法提高了算算法的性能,并具有更快的收敛速度和跳出局部最优的能力。     

Artificial Neural Network trained by Particle Swarm Optimization for non-linear channel equalization

In this paper, we apply Artificial Neural Network (ANN) trained with Particle Swarm Optimization (PSO) for the problem of channel equalization. Existing applications of PSO to Artificial Neural Networks (ANN) training have only been used to find optimal weights of the network. Novelty in this paper is that it also takes care of appropriate network topology and transfer functions of the neuron. The PSO algorithm optimizes all the variables, and hence network weights and network parameters. Hence, this paper makes use of PSO to optimize the number of layers, input and hidden neurons, the type of transfer functions etc. This paper focuses on optimizing the weights, transfer function, and topology of an ANN constructed for channel equalization. Extensive simulations presented in this paper shows that, as compared to other ANN based equalizers as well as Neuro-fuzzy equalizers, the proposed equalizer performs better in all noise conditions.     

An improved cooperative particle swarm optimization and its application

A powerful cooperative evolutionary particle swarm optimization (PSO) algorithm based on two swarms with different behaviors to improve the global performance of PSO is proposed. In this method, one swarm tracks the best position and the other leaves the worst position of them; the best and the worst solutions of the two swarms are exchanged in the common blackboard and the information can be flowed mutually between them. The diversity is maintained if the two swarms are regarded as a whole. To show the effectiveness of the given algorithm, five benchmark functions and two forward ANNs with three layers are performed; the results of the proposed algorithms are compared with standard PSO, MCPSO and NPSO.     

A novel improved particle swarm optimization algorithm based on individual difference evolution

As a well-known stochastic optimization algorithm, the particle swarm optimization (PSO) algorithm has attracted the attention of many researchers all over the world, which has resulted in many variants of the basic algorithm, in addition to a vast number of parameter selection/control strategies. However, most of these algorithms evolve their population using a single fixed pattern, thereby reducing the intelligence of the entire swarm. Some PSO-variants adopt a multimode evolutionary strategy, but lack dynamic adaptability. Furthermore, competition among particles is ignored, with no consideration of individual thinking or decision-making ability. This paper introduces an evolution mechanism based on individual difference, and proposes a novel improved PSO algorithm based on individual difference evolution (IDE-PSO). This algorithm allocates a competition coefficient called the emotional status to each particle for quantifying individual differences, separates the entire swarm into three subgroups, and selects the specific evolutionary method for each particle according to its emotional status and current fitness. The value of the coefficient is adjusted dynamically according to the evolutionary performance of each particle. A modified restarting strategy is employed to regenerate corresponding particles and enhance the diversity of the population. For a series of benchmark functions, simulation results show the effectiveness of the proposed IDE-PSO, which outperforms many state-of-the-art evolutionary algorithms in terms of convergence, robustness, and scalability.     

Optimization of neural network model using modified bat-inspired algorithm

The success of an artificial neural network (ANN) strongly depends on the variety of the connection weights and the network structure. Among many methods used in the literature to accurately select the network weights or structure in isolate; a few researchers have attempted to select both the weights and structure of ANN automatically by using metaheuristic algorithms. This paper proposes modified bat algorithm with a new solution representation for both optimizing the weights and structure of ANNs. The algorithm, which is based on the echolocation behaviour of bats, combines the advantages of population-based and local search algorithms. In this work, ability of the basic bat algorithm and some modified versions which are based on the consideration of the personal best solution in the velocity adjustment, the mean of personal best and global best solutions through velocity adjustment and the employment of three chaotic maps are investigated. These modifications are aimed to improve the exploration and exploitation capability of bat algorithm. Different versions of the proposed bat algorithm are incorporated to handle the selection of the structure as well as weights and biases of the ANN during the training process. We then use the Taguchi method to tune the parameters of the algorithm that demonstrates the best ability compared to the other versions. Six classifications and two time series benchmark datasets are used to test the performance of the proposed approach in terms of classification and prediction accuracy. Statistical tests demonstrate that the proposed method generates some of the best results in comparison with the latest methods in the literature. Finally, our best method is applied to a real-world problem, namely to predict the future values of rainfall data and the results show satisfactory of the method.     

Dynamic neural network method-based improved PSO and BR algorithms for transient probabilistic analysis of flexible mechanism

To improve the computing efficiency and precision of transient probabilistic analysis of flexible mechanism, dynamic neural network method (DNNM)-based improved particle swarm optimization (PSO)/Bayesian regularization (BR) (called as PSO/BR-DNNM) is proposed based on the developed DNNM with the integration of extremum response surface method (ERSM) and artificial neural network (ANN). The mathematical model of DNNM is established based on ANN on the foundation of investigating ERSM. Aiming at the high nonlinearity and strong coupling characteristics of limit state function of flexible mechanism, accurate weights and thresholds of PSO/BR-DNNM function are discussed by searching initial weights and thresholds based on the improved PSO and training final weights and thresholds by the BR-based training performance function. The probabilistic analysis of two-link flexible robot manipulator (TFRM) was investigated with the proposed method. Reliability degree, distribution characteristics and major factors (section sizes of link-2) of TFRM are obtained, which provides a useful reference for a more effective TFRM design. Through the comparison of three methods (Monte Carlo method, DNNM, PSO/BR-DNNM), it is demonstrated that PSO/BR-DNNM reshapes the probability of flexible mechanism probabilistic analysis and improves the computing efficiency while keeping acceptable computational precision. Moreover, the proposed method offers a useful insight for reliability-based design optimization of flexible mechanism and thereby also enriches the theory and method of mechanical reliability design.     

Exploring comprehensible classification rules from trained neural networks integrated with a time-varying binary particle swarm optimizer

Purpose

Extracting comprehensible classification rules is the most emphasized concept in data mining researches. In order to obtain accurate and comprehensible classification rules from databases, a new approach is proposed by combining advantages of artificial neural networks (ANN) and swarm intelligence.

Method

Artificial neural networks (ANNs) are a group of very powerful tools applied to prediction, classification and clustering in different domains. The main disadvantage of this general purpose tool is the difficulties in its interpretability and comprehensibility. In order to eliminate these disadvantages, a novel approach is developed to uncover and decode the information hidden in the black-box structure of ANNs. Therefore, in this paper a study on knowledge extraction from trained ANNs for classification problems is carried out. The proposed approach makes use of particle swarm optimization (PSO) algorithm to transform the behaviors of trained ANNs into accurate and comprehensible classification rules. Particle swarm optimization with time varying inertia weight and acceleration coefficients is designed to explore the best attribute-value combination via optimizing ANN output function.

Results

The weights hidden in trained ANNs turned into comprehensible classification rule set with higher testing accuracy rates compared to traditional rule based classifiers.     

Artificial intelligence approach to classify unipolar and bipolar depressive disorders

Machine learning approaches for medical decision-making processes are valuable when both high classification accuracy and less feature requirements are satisfied. Artificial neural networks (ANNs) successfully meet the first goal with its adaptive engine, while nature-inspired algorithms are focusing on the feature selection (FS) process in order to eliminate less informative and less discriminant features. Besides engineering applications of ANN and FS algorithms, medical informatics is another emerging field using similar methods for medical data processing. Classification of psychiatric disorders is one of the major focus of medical informatics using artificial intelligence approaches. Being one of the most debilitating psychiatric diseases, bipolar disorder (BD) is frequently misdiagnosed as unipolar disorder (UD), leading to suboptimal treatment and poor outcomes. Thus, discriminating UD and BD at earlier stages of illness could therefore help to facilitate efficient and specific treatment. The use of quantitative electroencephalography (EEG) cordance as a biomarker has greatly enhanced the clinical utility of EEG in psychiatric and neurological subjects. In this context, the paper puts forward a study using two-step hybridized methodology: particle swarm optimization (PSO) algorithm for FS process and ANN for training process. The noteworthy performance of ANN–PSO approach stated that it is possible to discriminate 31 bipolar and 58 unipolar subjects using selected features from alpha and theta frequency bands with 89.89 % overall classification accuracy.     

Forecasting of Software Reliability Using Neighborhood Fuzzy Particle Swarm Optimization Based Novel Neural Network

This paper proposes an artificial neural network (ANN) based software reliability model trained by novel particle swarm optimization (PSO) algorithm for enhanced forecasting of the reliability of software. The proposed ANN is developed considering the fault generation phenomenon during software testing with the fault complexity of different levels. We demonstrate the proposed model considering three types of faults residing in the software. We propose a neighborhood based fuzzy PSO algorithm for competent learning of the proposed ANN using software failure data. Fitting and prediction performances of the neighborhood fuzzy PSO based proposed neural network model are compared with the standard PSO based proposed neural network model and existing ANN based software reliability models in the literature through three real software failure data sets. We also compare the performance of the proposed PSO algorithm with the standard PSO algorithm through learning of the proposed ANN. Statistical analysis shows that the neighborhood fuzzy PSO based proposed neural network model has comparatively better fitting and predictive ability than the standard PSO based proposed neural network model and other ANN based software reliability models. Faster release of software is achievable by applying the proposed PSO based neural network model during the testing period.      

Generation of optimal artificial neural networks using a pattern search algorithm: application to approximation of chemical systems

A pattern search optimization method is applied to the generation of optimal artificial neural networks (ANNs). Optimization is performed using a mixed variable extension to the generalized pattern search method. This method offers the advantage that categorical variables, such as neural transfer functions and nodal connectivities, can be used as parameters in optimization. When used together with a surrogate, the resulting algorithm is highly efficient for expensive objective functions. Results demonstrate the effectiveness of this method in optimizing an ANN for the number of neurons, the type of transfer function, and the connectivity among neurons. The optimization method is applied to a chemistry approximation of practical relevance. In this application, temperature and a chemical source term are approximated as functions of two independent parameters using optimal ANNs. Comparison of the performance of optimal ANNs with conventional tabulation methods demonstrates equivalent accuracy by considerable savings in memory storage. The architecture of the optimal ANN for the approximation of the chemical source term consists of a fully connected feedforward network having four nonlinear hidden layers and 117 synaptic weights. An equivalent representation of the chemical source term using tabulation techniques would require a 500 x 500 grid point discretization of the parameter space.     

Implementation of hybrid ANN–PSO algorithm on FPGA for harmonic estimation

Harmonic estimation is the main process in active filters for harmonic reduction. A hybrid Adaptive Neural Network–Particle Swarm Optimization (ANN–PSO) algorithm is being proposed for harmonic isolation. Originally Fourier Transformation is used to analyze a distorted wave. In order to improve the convergence rate and processing speed an Adaptive Neural Network Algorithm called Adaline has then been used. A further improvement has been provided to reduce the error and increase the fineness of harmonic isolation by combining PSO algorithm with Adaline algorithm. The inertia weight factor of PSO is combined along with the weight factor of Adaline and trained in Neural Network environment for better results. ANN–PSO provides uniform convergence with the convergence rate comparable that of Adaline algorithm. The proposed ANN–PSO algorithm is implemented on an FPGA. To validate the performance of ANN–PSO; results are compared with Adaline algorithm and presented herein.     

Neural Network and Classification Approach in Identifying Customer Behavior in the Banking Sector: A Case Study of an International Bank

The customer relationship focus for banks is in development of main competencies and strategies of building strong profitable customer relationships through considering and managing the customer impression, influence on the culture of the bank, satisfactory treatment, and assessment of valued relationship building. Artificial neural networks (ANNs) are used after data segmentation and classification, where the designed model register records into two class sets, that is, the training and testing sets. ANN predicts new customer behavior from previously observed customer behavior after executing the process of learning from existing data. This article proposes an ANN model, which is developed using a six‐step procedure. The back‐propagation algorithm is used to train the ANN by adjusting its weights to minimize the difference between the current ANN output and the desired output. An evaluation process is conducted to determine whether the ANN has learned how to perform. The training process is halted periodically, and its performance is tested until an acceptable result is obtained. The principles underlying detection software are grounded in classical statistical decision theory.     

Time-series forecasting using a system of ordinary differential equations

This paper presents a hybrid evolutionary method for identifying a system of ordinary differential equations (ODEs) to predict the small-time scale traffic measurements data. We used the tree-structure based evolutionary algorithm to evolve the architecture and a particle swarm optimization (PSO) algorithm to fine tune the parameters of the additive tree models for the system of ordinary differential equations. We also illustrate some experimental comparisons with genetic programming, gene expression programming and a feedforward neural network optimized using PSO algorithm. Experimental results reveal that the proposed method is feasible and efficient for forecasting the small-scale traffic measurements data.