Structure optimization of neural networks for evolutionary design optimization

We study the use of neural networks as approximate models for the fitness evaluation in evolutionary design optimization. To improve the quality of the neural network models, structure optimization of these networks is performed with respect to two different criteria: One is the commonly used approximation error with respect to all available data, and the other is the ability of the networks to learn different problems of a common class of problems fast and with high accuracy. Simulation results from turbine blade optimizations using the structurally optimized neural network models are presented to show that the performance of the models can be improved significantly through structure optimization.We would like to thank the BMBF, grant LOKI, number 01 IB 001 C, for their financial support of our research.     

Towards probabilistic footy tipping: a hybrid approach utilising genetically defined neural networks and linear programming

Using readily available data from the 1992–1995 Australian Football League season, we have developed a model that will readily predict the winner of a game, together with the probability of that win. This model has been developed using a genetically modified neural network to calculate the likely winner, combined with a linear program optimisation to determine the probability of that occurring in the context of the tipping competition scoring regime. This model has then been tested against 484 tippers in a probabilistic tipping competition for the 2002 season. We have found that the performance of the combined neural network, linear program model compared most favorably with other model based tipping programs and human tippers.     

Credit scoring and rejected instances reassigning through evolutionary computation techniques

The credit industry is concerned with many problems of interest to the computation community. This study presents a work involving two interesting credit analysis problems and resolves them by applying two techniques, neural networks (NNs) and genetic algorithms (GAs), within the field of evolutionary computation. The first problem is constructing NN-based credit scoring model, which classifies applicants as accepted (good) or rejected (bad) credits. The second one is better understanding the rejected credits, and trying to reassign them to the preferable accepted class by using the GA-based inverse classification technique. Each of these problems influences on the decisions relating to the credit admission evaluation, which significantly affects risk and profitability of creditors. From the computational results, NNs have emerged as a computational tool that is well-matched to the problem of credit classification. Using the GA-based inverse classification, creditors can suggest the conditional acceptance, and further explain the conditions to rejected applicants. In addition, applicants can evaluate the option of minimum modifications to their attributes.     

Enhancing neural control systems by fuzzy logic and evolutionary reinforcement

Neural networks are widely used for system modelling and control because of their ability to approximate complex non-linear functions. Fuzzy systems, similarly, have been shown to be able to approximate or model any nonlinear system. Fuzzy-logic and neural systems, however, have very contrasting application requirements and it has been said that their integration offers a facility to bridge symbolic knowledge processing and connectionist learning. The significance of the integration becomes more apparent by considering their disparities. Neural networks do not provide a strong scheme for knowledge representation, while fuzzy systems do not possess capabilities for automated learning. On the other hand, another learning method has emerged recently, as an alternative to inductive techniques used with neural networks, namely, genetic or evolutionary learning. This paper will present a technique for the fusion of the three paradigms in a learning control context. It will describe a type of learning, known as Evolutionary Algorithm Reinforcement Learning (EARL), which is used to optimise a fuzzy neural control system. An application case study is also presented.     

A neural networks approach to minority game

The minority game (MG) comes from the so-called “El Farol bar” problem by W.B. Arthur. The underlying idea is competition for limited resources and it can be applied to different fields such as: stock markets, alternative roads between two locations and in general problems in which the players in the “minority” win. Players in this game use a window of the global history for making their decisions, we propose a neural networks approach with learning algorithms in order to determine players strategies. We use three different algorithms to generate the sequence of minority decisions and consider the prediction power of a neural network that uses the Hebbian algorithm. The case of sequences randomly generated is also studied. Research supported by Local Project 2004–2006 (EX 40%) Università di Foggia. A. Sfrecola is a researcher financially supported by Dipartimento di Scienze Economiche, Matematiche e Statistiche, Università degli Studi di Foggia, Foggia, Italy.     

Parallel processing, neural networks and genetic algorithms

In an earlier paper[1] some recent developments in computational technology to structural engineering were described. The developments included: parallel and distributed computing; neural networks; and genetic algorithms. In this paper, the authors concentrate on parallel implementations of neural networks and genetic algorithms. In the final section of the paper the authors show how a parallel finite element analysis may be undertaken in an efficient manner by preprocessing of the finite element model using a genetic algorithm utilizing a neural network predictor. This preprocessing is the partitioning of the finite element mesh into sub-domains to ensure load balancing and minimum interprocessor communication during the parallel finite element analysis on a MIMD distributed memory computer. © 1998 Published by Elsevier Science Limited. All rights reserved.     

Design and comparison of adaptive power system stabilizers based on neural fuzzy networks and genetic algorithms

This paper presents two different power system stabilizers (PSSs) which are designed making use of neural fuzzy network and genetic algorithms (GAs). In both cases, GAs tune a conventional PSS on different operating conditions and then, the relationship between these points and the PSS parameters is learned by the ANFIS. ANFIS will select the PSS parameters based on machine loading conditions. The first stabilizer is adjusted minimizing an objective function based on ITAE index, while second stabilizer is adjusted minimizing an objective function based on pole-placement technique. The proposed stabilizers have been tested by performing simulations of the overall nonlinear system. Preliminary experimental results are shown.     

Rules extraction from constructively trained neural networks based on genetic algorithms

The application of neural networks in the data mining has become wider. Although neural networks may have complex structure, long training time, and the representation of results is not comprehensible, neural networks have high acceptance ability for noisy data, high accuracy and are preferable in data mining. On the other hand, It is an open question as to what is the best way to train and extract symbolic rules from trained neural networks in domains like classification. In this paper, we train the neural networks by constructive learning and present the analysis of the convergence rate of the error in a neural network with and without threshold which have been learnt by a constructive method to obtain the simple structure of the network.The response of ANN is acquired but its result is not in understandable form or in a black box form. It is frequently desirable to use the model backwards and identify sets of input variable which results in a desired output value. The large numbers of variables and nonlinear nature of many materials models that can help finding an optimal set of difficult input variables. We will use a genetic algorithm to solve this problem. The method is evaluated on different public-domain data sets with the aim of testing the predictive ability of the method and compared with standard classifiers, results showed comparatively high accuracy.     

Optimum design of structures by an improved genetic algorithm using neural networks

Optimum design of large-scale structures by standard genetic algorithm (GA) makes the computational burden of the process very high. To reduce the computational cost of standard GA, two different strategies are used. The first strategy is by modifying the standard GA, called virtual sub-population method (VSP). The second strategy is by using artificial neural networks for approximating the structural analysis. In this study, radial basis function (RBF), counter propagation (CP) and generalized regression (GR) neural networks are used. Using neural networks within the framework of VSP creates a robust tool for optimum design of structures.     

Pruning backpropagation neural networks using modern stochastic optimisation techniques

Approaches combining genetic algorithms and neural networks have received a great deal of attention in recent years. As a result, much work has been reported in two major areas of neural network design: training and topology optimisation. This paper focuses on the key issues associated with the problem of pruning a multilayer perceptron using genetic algorithms and simulated annealing. The study presented considers a number of aspects associated with network training that may alter the behaviour of a stochastic topology optimiser. Enhancements are discussed that can improve topology searches. Simulation results for the two mentioned stochastic optimisation methods applied to non-linear system identification are presented and compared with a simple random search.     

Thalassaemia classification by neural networks and genetic programming

This paper presents the use of a neural network and a decision tree, which is evolved by genetic programming (GP), in thalassaemia classification. The aim is to differentiate between thalassaemic patients, persons with thalassaemia trait and normal subjects by inspecting characteristics of red blood cells, reticulocytes and platelets. A structured representation on genetic algorithms for non-linear function fitting or STROGANOFF is the chosen architecture for genetic programming implementation. For comparison, multilayer perceptrons are explored in classification via a neural network. The classification results indicate that the performance of the GP-based decision tree is approximately equal to that of the multilayer perceptron with one hidden layer. But the multilayer perceptron with two hidden layers, which is proven to have the most suitable architecture among networks with different number of hidden layers, outperforms the GP-based decision tree. Nonetheless, the structure of the decision tree reveals that some input features have no effects on the classification performance. The results confirm that the classification accuracy of the multilayer perceptron with two hidden layers can still be maintained after the removal of the redundant input features. Detailed analysis of the classification errors of the multilayer perceptron with two hidden layers, in which a reduced feature set is used as the network input, is also included. The analysis reveals that the classification ambiguity and misclassification among persons with minor thalassaemia trait and normal subjects is the main cause of classification errors. These results suggest that a combination of a multilayer perceptron with a blood cell analysis may give rise to a guideline/hint for further investigation of thalassaemia classification.     

Evolution of neural control structures: some experiments on mobile robots

From perception to action and from action to perception, all elements of an autonomous agent are interdependent and need to be strongly coherent. The final behavior of the agent is the result of the global activity of this loop and every weakness or incoherence of a single element has strong consequences on the performances of the agent. We think that, for the purpose of building autonomous robots, all these elements need to be developed together in continuous interaction with the environment. We describe the implementation of a possible solution (artificial neural networks and genetic algorithms) on a real mobile robot through a set of three different experiments. We focus our attention on three different aspects of the control structure: perception, internal representation and action. In all the experiments these aspects are not considered as single processing elements, but as part of an agent. For every experiment, the advantages and disadvantages of this approach are presented and discussed. The results show that the combination of genetic algorithms and neural networks is a very interesting technique for the development of control structures in autonomous agents. The time necessary for evolution, on the other hand, is a very important limitation of the evolutionary approach.     

Memetic evolutionary training for recurrent neural networks: an application to time-series prediction

Abstract: Artificial neural networks are bio-inspired mathematical models that have been widely used to solve complex problems. The training of a neural network is an important issue to deal with, since traditional gradient-based algorithms become easily trapped in local optimal solutions, therefore increasing the time taken in the experimental step. This problem is greater in recurrent neural networks, where the gradient propagation across the recurrence makes the training difficult for long-term dependences. On the other hand, evolutionary algorithms are search and optimization techniques which have been proved to solve many problems effectively. In the case of recurrent neural networks, the training using evolutionary algorithms has provided promising results. In this work, we propose two hybrid evolutionary algorithms as an alternative to improve the training of dynamic recurrent neural networks. The experimental section makes a comparative study of the algorithms proposed, to train Elman recurrent neural networks in time-series prediction problems.     

Training neural networks by stochastic optimisation

We present a stochastic learning algorithm for neural networks. The algorithm does not make any assumptions about transfer functions of individual neurons and does not depend on a functional form of a performance measure. The algorithm uses a random step of varying size to adapt weights. The average size of the step decreases during learning. The large steps enable the algorithm to jump over local maxima/minima, while the small ones ensure convergence in a local area. We investigate convergence properties of the proposed algorithm as well as test the algorithm on four supervised and unsupervised learning problems. We have found a superiority of this algorithm compared to several known algorithms when testing them on generated as well as real data.     

Fuzzy assisted learning in backpropagation neural networks

This paper reports on studies to overcome difficulties associated with setting the learning rates of backpropagation neural networks by using fuzzy logic. Building on previous research, a fuzzy control system is designed which is capable of dynamically adjusting the individual learning rates of both hidden and output neurons, and the momentum term within a back-propagation network. Results show that the fuzzy controller not only eliminates the effort of configuring a global learning rate, but also increases the rate of convergence in comparison with a conventional backpropagation network. Comparative studies are presented for a number of different network configurations. The paper also presents a brief overview of fuzzy logic and backpropagation learning, highlighting how the two paradigms can enhance each other.     

Learning Dynamic Obstacle Avoidance for a Robot Arm Using Neuroevolution

Neural networks can be evolved to control robot manipulators in tasks like target tracking and obstacle avoidance in complex environments. Neurocontrollers are robust to noise and can be adapted to different environments and robot configurations. In this paper, neurocontrollers were evolved to position the end effector of a robot arm close to a target in three different environments: environments without obstacles, environments with stationary obstacles, and environments with moving obstacles. The evolved neurocontrollers perform qualitatively like inverse kinematic controllers in environments with no obstacles and like path-planning controllers based on Rapidly-exploring random trees in environments with obstacles. Unlike inverse kinematic controllers and path planners, the approach reliably generalizes to environments with moving obstacles, making it possible to use it in natural environments.     

Location of amide I mode of vibration in computed data utilizing constructed neural networks

An automatic location method of amide I mode of vibration between computed data is proposed, based on the well established neural network model of artificial intelligence. This method was developed by constructing and testing the neural network on previously computed and characterized data which were divided in the training and the testing set, respectively. The results show high level of success since the majority of amide I modes of vibration in the testing set were located 99.5%.     

软件测试数据智能化生成的研究

软件测试数据自动生成是软件测试中的重要难题之一。测试数据自动生成问题可归结为测试数据的搜索或组合优化问题，通常具有不连续、不可微和非线性等特征，适合于采用遗传算法、神经网络等人工智能技术进行解决。国内外学者在此方面作了不少研究并取得一定的成果，但也存在一些问题。该文系统地综述了近年来软件测试数据智能化生成的研究和存在的问题，并对未来的发展进行了展望。     

Automated fuzzy knowledge acquisition with connectionist adaptation

This paper presents an automated knowledge acquisition architecture for the truck docking problem. The architecture consists of a neural network block, a fuzzy rule generation block and a genetic optimisation block. The neural network block is used to quickly and adaptively learn from trials the driving knowledge. The fuzzy rule generation block then extracts the driving knowledge to form a knowledge rule base. The driving knowledge rule base is further optimised in the genetic optimisation block using a genetic algorithm. Computer simulations are presented to show the effectiveness of the architecture.     

A neural network based multi-classifier system for gene identification in DNA sequences

The paper presents a neural network based multi-classifier system for the identification of Escherichia coli promoter sequences in strings of DNA. As each gene in DNA is preceded by a promoter sequence, the successful location of an E. coli promoter leads to the identification of the corresponding E. coli gene in the DNA sequence. A set of 324 known E. coli promoters and a set of 429 known non-promoter sequences were encoded using four different encoding methods. The encoded sequences were then used to train four different neural networks. The classification results of the four individual neural networks were then combined through an aggregation function, which used a variation of the logarithmic opinion pool method. The weights of this function were determined by a genetic algorithm. The multi-classifier system was then tested on 159 known promoter sequences and 171 non-promoter sequences not contained in the training set. The results obtained through this study proved that the same data set, when presented to neural networks in different forms, can provide slightly varying results. It also proves that when different opinions of more classifiers on the same input data are integrated within a multi-classifier system, we can obtain results that are better than the individual performances of the neural networks. The performances of our multi-classifier system outperform the results of other prediction systems for E. coli promoters developed so far.