Evolving modular networks with genetic algorithms: application to nonlinear time series

Abstract: A key problem of modular neural networks is finding the optimal aggregation of the different subtasks (or modules) of the problem at hand. Functional networks provide a partial solution to this problem, since the inter‐module topology is obtained from domain knowledge (functional relationships and symmetries). However, the learning process may be too restrictive in some situations, since the resulting modules (functional units) are assumed to be linear combinations of selected families of functions. In this paper, we present a non‐parametric learning approach for functional networks using feedforward neural networks for approximating the functional modules of the resulting architecture; we also introduce a genetic algorithm for finding the optimal intra‐module topology (the appropriate balance of neurons for the different modules according to the complexity of their respective tasks). Some benchmark examples from nonlinear time‐series prediction are used to illustrate the performance of the algorithm for finding optimal modular network architectures for specific problems.     

Multi-objective genetic algorithm based approaches for mining optimized fuzzy association rules

Association rules form one of the most widely used techniques to discover correlations among attribute in a database. So far, some efficient methods have been proposed to obtain these rules with respect to an optimal goal, such as: to maximize the number of large itemsets and interesting rules or the values of support and confidence for the discovered rules. This paper first introduces optimized fuzzy association rule mining in terms of three important criteria; strongness, interestingness and comprehensibility. Then, it proposes multi-objective Genetic Algorithm (GA) based approaches for discovering these optimized rules. Optimization technique according to given criterion may be one of two different forms; The first tries to determine the appropriate fuzzy sets of quantitative attributes in a prespecified rule, which is also called as certain rule. The second deals with finding both uncertain rules and their appropriate fuzzy sets. Experimental results conducted on a real data set show the effectiveness and applicability of the proposed approach.     

Multi-objective evolutionary algorithms for energy-aware scheduling on distributed computing systems

The ongoing increase of energy consumption by IT infrastructures forces data center managers to find innovative ways to improve energy efficiency. The latter is also a focal point for different branches of computer science due to its financial, ecological, political, and technical consequences. One of the answers is given by scheduling combined with dynamic voltage scaling technique to optimize the energy consumption. The way of reasoning is based on the link between current semiconductor technologies and energy state management of processors, where sacrificing the performance can save energy.This paper is devoted to investigate and solve the multi-objective precedence constrained application scheduling problem on a distributed computing system, and it has two main aims: the creation of general algorithms to solve the problem and the examination of the problem by means of the thorough analysis of the results returned by the algorithms.The first aim was achieved in two steps: adaptation of state-of-the-art multi-objective evolutionary algorithms by designing new operators and their validation in terms of performance and energy. The second aim was accomplished by performing an extensive number of algorithms executions on a large and diverse benchmark and the further analysis of performance among the proposed algorithms. Finally, the study proves the validity of the proposed method, points out the best-compared multi-objective algorithm schema, and the most important factors for the algorithms performance.     

Multi-objective optimization of sensor array using genetic algorithm

A multi-objective optimization method using genetic algorithm was proposed for sensor array optimization. Based on information theory, selectivity and diversity were used as the criteria for constructing two objective functions. A statistic measurement of resolving power, general resolution factor, and visual inspection were used to evaluate the optimization results with the aid of principal component analysis. In each Pareto set, most nondominated solutions had better statistics than the combination using all potential sensors. Also the principal component plots showed that different vapor classes were generally better separated after optimization. The experiment results indicated that the proposed method could successfully identify a set of Pareto optimal solutions of small size; and most optimized sensor arrays provided input with improved quality, i.e. better separation of target analytes. The running time for implementing the multi-objective optimization was satisfactory.     

Optimization methodology of dynamic data structures based on genetic algorithms for multimedia embedded systems

Modern multimedia application exhibit high resource utilization. In order to efficiently run this kind of applications in embedded systems, the dynamic memory subsystem needs to be optimized. A key role in this optimization is played by the dynamic data structures that reside in every real-life application. This paper presents a novel and automated way to optimize dynamic data structures. The search space is pruned using genetic algorithms that converge to the best multilayered data structure implementation for the targeted applications.     

Improved genetic algorithm inspired by biological evolution

The process of mutation has been studied extensively in the field of biology and it has been shown that it is one of the major factors that aid the process of evolution. Inspired by this a novel genetic algorithm (GA) is presented here. Various mutation operators such as small mutation, gene mutation and chromosome mutation have been applied in this genetic algorithm. In order to facilitate the implementation of the above-mentioned mutation operators a modified way of representing the variables has been presented. It resembles the way genetic information is coded in living beings. Different mutation operators pose a challenge as regards the determination of the optimal rate of mutation. This problem is overcome by using adaptive mutation operators. The main purpose behind this approach was to improve the efficiency of GAs and to find widely distributed Pareto-optimal solutions. This algorithm was tested on some benchmark test functions and compared with other GAs. It was observed that the introduction of these mutations do improve the genetic algorithms in terms of convergence and the quality of the solutions.     

基于DBNs理论的多传感器信息融合方法

为了提高多传感器信息融合系统的动态优化和容错能力,应用动态贝叶斯网络理论对传统联邦滤波器进行改造,得到了一种可以实现高动态环境条件下多传感器信息主动动态优化配置的信息融合方法．仿真结果表明,这种信息融合方法具有较强的动态适应和容错能力．     

Evolutionary algorithms and dynamic programming

Recently, it has been proven that evolutionary algorithms produce good results for a wide range of combinatorial optimization problems. Some of the considered problems are tackled by evolutionary algorithms that use a representation which enables them to construct solutions in a dynamic programming fashion. We take a general approach and relate the construction of such algorithms to the development of algorithms using dynamic programming techniques. Thereby, we give general guidelines on how to develop evolutionary algorithms that have the additional ability of carrying out dynamic programming steps. Finally, we show that for a wide class of the so-called DP-benevolent problems (which are known to admit FPTAS) there exists a fully polynomial-time randomized approximation scheme based on an evolutionary algorithm.     

Nonlinear time series forecasting with Bayesian neural networks

The Bayesian learning provides a natural way to model the nonlinear structure as the artificial neural networks due to their capability to cope with the model complexity. In this paper, an evolutionary Monte Carlo (MC) algorithm is proposed to train the Bayesian neural networks (BNNs) for the time series forecasting. This approach called as Genetic MC is based on Gaussian approximation with recursive hyperparameter. Genetic MC integrates MC simulations with the genetic algorithms and the fuzzy membership functions. In the implementations, Genetic MC is compared with the traditional neural networks and time series techniques in terms of their forecasting performances over the weekly sales of a Finance Magazine.     

Parallel processing, neural networks and genetic algorithms

In an earlier paper[1] some recent developments in computational technology to structural engineering were described. The developments included: parallel and distributed computing; neural networks; and genetic algorithms. In this paper, the authors concentrate on parallel implementations of neural networks and genetic algorithms. In the final section of the paper the authors show how a parallel finite element analysis may be undertaken in an efficient manner by preprocessing of the finite element model using a genetic algorithm utilizing a neural network predictor. This preprocessing is the partitioning of the finite element mesh into sub-domains to ensure load balancing and minimum interprocessor communication during the parallel finite element analysis on a MIMD distributed memory computer. © 1998 Published by Elsevier Science Limited. All rights reserved.     

Introducing assignment functions to Bayesian optimization algorithms

In this paper, we improve Bayesian optimization algorithms by introducing proportionate and rank-based assignment functions. A Bayesian optimization algorithm builds a Bayesian network from a selected sub-population of promising solutions, and this probabilistic model is employed to generate the offspring of the next generation. Our method assigns each solution a relative significance based on its fitness, and this information is used in building the Bayesian network model. These assignment functions can improve the quality of the model without performing an explicit selection on the population. Numerical experiments demonstrate the effectiveness of this method compared to a conventional BOA.     

Dynamic vehicle routing using genetic algorithms

Many difficult combinatorial optimization problems have been modeled as static problems. However, in practice, many problems are dynamic and changing, while some decisions have to be made before all the design data are known. For example, in the Dynamic Vehicle Routing Problem (DVRP), new customer orders appear over time, and new routes must be reconfigured while executing the current solution. Montemanni et al. [1] considered a DVRP as an extension to the standard vehicle routing problem (VRP) by decomposing a DVRP as a sequence of static VRPs, and then solving them with an ant colony system (ACS) algorithm. This paper presents a genetic algorithm (GA) methodology for providing solutions for the DVRP model employed in [1]. The effectiveness of the proposed GA is evaluated using a set of benchmarks found in the literature. Compared with a tabu search approach implemented herein and the aforementioned ACS, the proposed GA methodology performs better in minimizing travel costs. Franklin T. Hanshar is currently a M.Sc. student in the Department of Computing and Information Science at the University of Guelph, Ontario, Canada. He received a B.Sc. degree in Computer Science from Brock University in 2005. His research interests include uncertain reasoning, optimization and evolutionary computation. Beatrice Ombuki-Berman is currently an Associate Professor in the Department of Computer Science at Brock University, Ontario, Canada. She obtained a PhD and ME in Information Engineering from University of The Ryukyus, Okinawa, Japan in 2001 and 1998, respectively. She received a B.Sc. in Mathematics and Computer Science from Jomo Kenyatta University, Nairobi, Kenya. Her primary research interest is evolutionary computation and applied optimization. Other research interests include neural networks, machine learning and ant colony optimization.     

Evolving rule induction algorithms with multi-objective grammar-based genetic programming

Multi-objective optimization has played a major role in solving problems where two or more conflicting objectives need to be simultaneously optimized. This paper presents a Multi-Objective grammar-based genetic programming (MOGGP) system that automatically evolves complete rule induction algorithms, which in turn produce both accurate and compact rule models. The system was compared with a single objective GGP and three other rule induction algorithms. In total, 20 UCI data sets were used to generate and test generic rule induction algorithms, which can be now applied to any classification data set. Experiments showed that, in general, the proposed MOGGP finds rule induction algorithms with competitive predictive accuracies and more compact models than the algorithms it was compared with.

Multi-objective evolutionary biclustering of gene expression data

Biclustering or simultaneous clustering of both genes and conditions have generated considerable interest over the past few decades, particularly related to the analysis of high-dimensional gene expression data in information retrieval, knowledge discovery, and data mining. The objective is to find sub-matrices, i.e., maximal subgroups of genes and subgroups of conditions where the genes exhibit highly correlated activities over a range of conditions. Since these two objectives are mutually conflicting, they become suitable candidates for multi-objective modeling. In this study, a novel multi-objective evolutionary biclustering framework is introduced by incorporating local search strategies. A new quantitative measure to evaluate the goodness of the biclusters is developed. The experimental results on benchmark datasets demonstrate better performance as compared to existing algorithms available in literature.     

Adaptive reconfiguration of data networks using genetic algorithms

Genetic algorithms are applied to an important, but little investigated, network design problem, that of reconfiguring the topology and link capacities of an operational network to adapt to changes in its operating conditions. These conditions include: which nodes and links are unavailable; the traffic patterns; and the quality of service (QoS) requirements and priorities of different users and applications. Dynamic reconfiguration is possible in networks that contain links whose endpoints can be easily changed, such as satellite channels, terrestrial wireless connections, and certain types of optical connections. We report preliminary results that demonstrate the feasibility of performing genetic search quickly enough for online adaptation.     

Cooperative binary-real coded genetic algorithms for generating and adapting artificial neural networks

This paper describes a genetic system for designing and training feed-forward artificial neural networks to solve any problem presented as a set of training patterns. This system, called GANN, employs two interconnected genetic algorithms that work parallelly to design and train the better neural network that solves the problem. Designing neural architectures is performed by a genetic algorithm that uses a new indirect binary codification of the neural connections based on an algebraic structure defined in the set of all possible architectures that could solve the problem. A crossover operation, known as Hamming crossover, has been designed to obtain better performance when working with this type of codification. Training neural networks is also accomplished by genetic algorithms but, this time, real number codification is employed. To do so, morphological crossover operation has been developed inspired on the mathematical morphology theory. Experimental results are reported from the application of GANN to the breast cancer diagnosis within a complete computer-aided diagnosis system.     

Performance evaluation of subsea BOP control systems using dynamic Bayesian networks with imperfect repair and preventive maintenance

The work presents a dynamic Bayesian networks (DBN) modeling of series, parallel and 2-out-of-3 (2oo3) voting systems, taking account of common-cause failure, imperfect coverage, imperfect repair and preventive maintenance. Seven basic events of one, two or three component failure are proposed to model the common-cause failure of the three-components-systems. The imperfect coverage is modeled in the conditional probability table by defining a coverage factor. A multi-state degraded component is used to model the imperfect repair and preventive maintenance. Using the proposed method, a DBN modeling of a subsea blowout preventer (BOP) control system is built, and the reliability and availability are evaluated. The mutual information is researched in order to assess the important degree of basic events. The effects of degradation probability, failure rate and mean time to repair (MTTR) on the performances are studied. The results show that the repairs and maintenance can improve the system performance significantly, whereas the imperfect repair cannot degrade the system performance significantly in comparison with the perfect repair, and the preventive maintenance can improve the system performance slightly in comparison with the imperfect repair. In order to improve the performance of subsea BOP control system, the single surface components and the components with all-common-cause failure should given more attention. The influence of degradation probability on the performance is in the order of PLC, PC and ES. The influence of failure rate and MTTR on the performance is in the order of PLC, ES, PC, DO, DI and AI.     

A self-organizing random immigrants genetic algorithm for dynamic optimization problems

In this paper a genetic algorithm is proposed where the worst individual and individuals with indices close to its index are replaced in every generation by randomly generated individuals for dynamic optimization problems. In the proposed genetic algorithm, the replacement of an individual can affect other individuals in a chain reaction. The new individuals are preserved in a subpopulation which is defined by the number of individuals created in the current chain reaction. If the values of fitness are similar, as is the case with small diversity, one single replacement can affect a large number of individuals in the population. This simple approach can take the system to a self-organizing behavior, which can be useful to control the diversity level of the population and hence allows the genetic algorithm to escape from local optima once the problem changes due to the dynamics.     

A novel hybrid learning algorithm for full Bayesian approach of artificial neural networks

The Bayesian neural networks are useful tools to estimate the functional structure in the nonlinear systems. However, they suffer from some complicated problems such as controlling the model complexity, the training time, the efficient parameter estimation, the random walk, and the stuck in the local optima in the high-dimensional parameter cases. In this paper, to alleviate these mentioned problems, a novel hybrid Bayesian learning procedure is proposed. This approach is based on the full Bayesian learning, and integrates Markov chain Monte Carlo procedures with genetic algorithms and the fuzzy membership functions. In the application sections, to examine the performance of proposed approach, nonlinear time series and regression analysis are handled separately, and it is compared with the traditional training techniques in terms of their estimation and prediction abilities.