Adaptability analysis of genetic network programming with reinforcement learning in dynamically changing environments

Genetic network programming (GNP) has been proposed as one of the evolutionary algorithms and extended with reinforcement learning (GNP-RL). The combination of evolution and learning can efficiently evolve programs and the fitness improvement has been confirmed in the simulations of tileworld problems, elevator group supervisory control systems, stock trading models and wall following behavior of Khepera robot. However, its adaptability in testing environments, where the situations dynamically change, has not been analyzed in detail yet. In this paper, the adaptation mechanism in the testing environment is introduced and it is confirmed that GNP-RL can adapt to the environmental changes using a robot simulator WEBOTS, especially when unexperienced sensor troubles suddenly occur. The simulation results show that GNP-RL works well in the testing even if wrong sensor information is given because GNP-RL has a function to automatically change programs using alternative actions. In addition, the analysis on the effects of the parameters of GNP-RL is carried out in both training and testing simulations.     

Coevolutionary bid-based genetic programming for problem decomposition in classification

In this work a cooperative, bid-based, model for problem decomposition is proposed with application to discrete action domains such as classification. This represents a significant departure from models where each individual constructs a direct input-outcome map, for example, from the set of exemplars to the set of class labels as is typical under the classification domain. In contrast, the proposed model focuses on learning a bidding strategy based on the exemplar feature vectors; each individual is associated with a single discrete action and the individual with the maximum bid ‘wins’ the right to suggest its action. Thus, the number of individuals associated with each action is a function of the intra-action bidding behaviour. Credit assignment is designed to reward correct but unique bidding strategies relative to the target actions. An advantage of the model over other teaming methods is its ability to automatically determine the number of and interaction between cooperative team members. The resulting model shares several traits with learning classifier systems and as such both approaches are benchmarked on nine large classification problems. Moreover, both of the evolutionary models are compared against the deterministic Support Vector Machine classification algorithm. Performance assessment considers the computational, classification, and complexity characteristics of the resulting solutions. The bid-based model is found to provide simple yet effective solutions that are robust to wide variations in the class representation. Support Vector Machines and classifier systems tend to perform better under balanced datasets albeit resulting in black-box solutions.

Modular neural network programming with genetic optimization

This study proposes a modular neural network (MNN) that is designed to accomplish both artificial intelligent prediction and programming. Each modular element adopts a high-order neural network to create a formula that considers both weights and exponents. MNN represents practical problems in mathematical terms using modular functions, weight coefficients and exponents. This paper employed genetic algorithms to optimize MNN parameters and designed a target function to avoid over-fitting. Input parameters were identified and modular function influences were addressed in manner that significantly improved previous practices. In order to compare the effectiveness of results, a reference study on high-strength concrete was adopted, which had been previously studied using a genetic programming (GP) approach. In comparison with GP, MNN calculations were more accurate, used more concise programmed formulas, and allowed the potential to conduct parameter studies. The proposed MNN is a valid alternative approach to prediction and programming using artificial neural networks.     

The structure of the genetic programming collaboration network

The genetic programming bibliography aims to be the most complete reference of papers on genetic programming. In addition to locating publications, it contains coauthor and coeditor relationships which have not previously been studied. These reveal some similarities and differences between our field and collaborative social networks in other scientific fields.

A survey and taxonomy of performance improvement of canonical genetic programming

The genetic programming (GP) paradigm, which applies the Darwinian principle of evolution to hierarchical computer programs, has been applied with breakthrough success in various scientific and engineering applications. However, one of the main drawbacks of GP has been the often large amount of computational effort required to solve complex problems. Much disparate research has been conducted over the past 25 years to devise innovative methods to improve the efficiency and performance of GP. This paper attempts to provide a comprehensive overview of this work related to Canonical Genetic Programming based on parse trees and originally championed by Koza (Genetic programming: on the programming of computers by means of natural selection. MIT, Cambridge, 1992). Existing approaches that address various techniques for performance improvement are identified and discussed with the aim to classify them into logical categories that may assist with advancing further research in this area. Finally, possible future trends in this discipline and some of the open areas of research are also addressed.     

一种基于改进遗传规划的泛函网络结构逼近模型

基于泛函网络的结构特点和遗传规划的全局搜索能力,提出了广义基函数概念,通过改进遗传规划的编码方式对广义基函数进行学习,用最小二乘法设计适应度函数,从而确定泛函网络的最佳逼近结构模型。最后,4个数值仿真实例表明,该方法是有效可行的,具有较强的泛化特性。     

Continuous probabilistic model building genetic network programming using reinforcement learning

Recently, a novel probabilistic model-building evolutionary algorithm (so called estimation of distribution algorithm, or EDA), named probabilistic model building genetic network programming (PMBGNP), has been proposed. PMBGNP uses graph structures for its individual representation, which shows higher expression ability than the classical EDAs. Hence, it extends EDAs to solve a range of problems, such as data mining and agent control. This paper is dedicated to propose a continuous version of PMBGNP for continuous optimization in agent control problems. Different from the other continuous EDAs, the proposed algorithm evolves the continuous variables by reinforcement learning (RL). We compare the performance with several state-of-the-art algorithms on a real mobile robot control problem. The results show that the proposed algorithm outperforms the others with statistically significant differences.     

Human-competitive results produced by genetic programming

Genetic programming has now been used to produce at least 76 instances of results that are competitive with human-produced results. These human-competitive results come from a wide variety of fields, including quantum computing circuits, analog electrical circuits, antennas, mechanical systems, controllers, game playing, finite algebras, photonic systems, image recognition, optical lens systems, mathematical algorithms, cellular automata rules, bioinformatics, sorting networks, robotics, assembly code generation, software repair, scheduling, communication protocols, symbolic regression, reverse engineering, and empirical model discovery. This paper observes that, despite considerable variation in the techniques employed by the various researchers and research groups that produced these human-competitive results, many of the results share several common features. Many of the results were achieved by using a developmental process and by using native representations regularly used by engineers in the fields involved. The best individual in the initial generation of the run of genetic programming often contains only a small number of operative parts. Most of the results that duplicated the functionality of previously issued patents were novel solutions, not infringing solutions. In addition, the production of human-competitive results, as well as the increased intricacy of the results, are broadly correlated to increased availability of computing power tracked by Moore’s law. The paper ends by predicting that the increased availability of computing power (through both parallel computing and Moore’s law) should result in the production, in the future, of an increasing flow of human-competitive results, as well as more intricate and impressive results.     

Assessment of artificial neural network and genetic programming as predictive tools

Soft computing techniques have been widely used during the last two decades for nonlinear system modeling, specifically as predictive tools. In this study, the performances of two well-known soft computing predictive techniques, artificial neural network (ANN) and genetic programming (GP), are evaluated based on several criteria, including over-fitting potential. A case study in punching shear prediction of RC slabs is modeled here using a hybrid ANN (which includes simulated annealing and multi-layer perception) and an established GP variant called gene expression programming. The ANN and GP results are compared to values determined from several design codes. For more verification, external validation and parametric studies were also conducted. The results of this study indicate that model acceptance criteria should include engineering analysis from parametric studies.     

A model of portfolio optimization using time adapting genetic network programming

This paper describes a decision-making model of dynamic portfolio optimization for adapting to the change of stock prices based on an evolutionary computation method named genetic network programming (GNP). The proposed model, making use of the information from technical indices and candlestick chart, is trained to generate portfolio investment advice. Experimental results on the Japanese stock market show that the decision-making model using time adapting genetic network programming (TA-GNP) method outperforms other traditional models in terms of both accuracy and efficiency. A comprehensive analysis of the results is provided, and it is clarified that the TA-GNP method is effective on the portfolio optimization problem.     

Information transformation by virus-evolutionary genetic programming

This paper deals with genetic programming (GP) for information translation. The GP can generate a structured computer program, but it is difficult to define recursive functions automatically. Therefore, we propose a virus-evolutionary genetic programming (VE-GP) composed of two populations; host and virus. Here, a virus plays the role of an automatically defined function. First, the VE-GP is applied to a function approximation problem, and the simulation result shows that the VE-GP can generate a function to approximate the given function with small errors. Next, the VE-GP is applied to the information transformation for a classification task, and the simulation result shows that the VE-GP can generate a function to classify a given data set. This work was presented in part at the Fourth International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–22, 1999     

Robot path planning by genetic programming

This paper describes a method to determine the path of a robot that travels around between machine tools in a production line FA factory. This decision is made by the genetic algorithm with Lisp language programming. In the algorithm, the building block method to decide fitness is adopted. The method is applied to a flexible manufacturing system (FMS) that has four machine tools and a robot. This work was presented, in part, at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

Modelling environments in call-by-value programming languages

In categorical semantics, there have traditionally been two approaches to modelling environments, one by use of finite products in cartesian closed categories, the other by use of the base categories of indexed categories with structure. Each requires modifications in order to account for environments in call-by-value programming languages. There have been two more general definitions along both of these lines: the first generalising from cartesian to symmetric premonoidal categories, the second generalising from indexed categories with specified structure to κ-categories. In this paper, we investigate environments in call-by-value languages by analysing a fine-grain variant of Moggi’s computational λ-calculus, giving two equivalent sound and complete classes of models: one given by closed Freyd categories, which are based on symmetric premonoidal categories, the other given by closed κ-categories.     

Modifying genetic programming for artificial neural network development for data mining

The development of artificial neural networks (ANNs) is usually a slow process in which the human expert has to test several architectures until he finds the one that achieves best results to solve a certain problem. However, there are some tools that provide the ability of automatically developing ANNs, many of them using evolutionary computation (EC) tools. One of the main problems of these techniques is that ANNs have a very complex structure, which makes them very difficult to be represented and developed by these tools. This work presents a new technique that modifies genetic programming (GP) so as to correctly and efficiently work with graph structures in order to develop ANNs. This technique also allows the obtaining of simplified networks that solve the problem with a small group of neurons. In order to measure the performance of the system and to compare the results with other ANN development methods by means of evolutionary computation (EC) techniques, several tests were performed with problems based on some of the most used test databases in the Data Mining domain. These comparisons show that the system achieves good results that are not only comparable to those of the already existing techniques but, in most cases, improve them.     

Target detection in SAR imagery by genetic programming

The automatic detection of ships in low-resolution synthetic aperture radar (SAR) imagery is investigated in this article. The detector design objectives are to maximise detection accuracy across multiple images, to minimise the computational effort during image processing, and to minimise the effort during the design stage. The results of an extensive numerical study show that a novel approach, using genetic programming (GP), successfully evolves detectors which satisfy the earlier objectives. Each detector represents an algebraic formula and thus the principles of detection can be discovered and reused. This is a major advantage over artificial intelligence techniques which use more complicated representations, e.g. neural networks.     

Emergence in genetic programming

Banzhaf explores the concept of emergence and how and where it happens in genetic programming [1]. Here we consider the question: what shall we do with it? We argue that given our ultimate goal to produce genetic programming systems that solve new and difficult problems, we should take advantage of emergence to get closer to this goal.     

Survivable telecommunications network design under different types of failures

Telecommunications networks are expected to provide near-instantaneous restoration in the event that some network elements fail. Models for designing survivable networks are very complex and difficult to solve optimally. In this paper, we provide simple heuristics that augment existing network resources to ensure restoration under several scenarios of a single failure. The goal is to demonstrate that effective, though not necessarily optimal, survivable designs can be achieved by augmenting capacities along prudently selected variants of spanning tree and ring structures, without resorting to complex mathematical programming methods. The first model considers line restoration (reroutes around the failed link) under a partial link failure. We propose a heuristic that augments capacities of selected network links by forming a "virtual" spanning tree of restoration capacity. The second model provides line restoration under a complete link failure. We propose a heuristic that ensures survivability by repeatedly constructing spanning trees to various subnetworks. The third model provides path restoration (end-to-end reroutes) under a node failure. We propose a heuristic that repeatedly constructs restoration rings that cover a subset of source-destination nodes that carry traffic through intermediate nodes.     

A comparison of grammatical genetic programming grammars for controlling femtocell network coverage

We study grammars used in grammatical genetic programming (GP) which create algorithms that control the base station pilot power in a femtocell network. The overall goal of evolving algorithms for femtocells is to create a continuous online evolution of the femtocell pilot power control algorithm in order to optimize their coverage. We compare the performance of different grammars and analyse the femtocell simulation model using the grammatical genetic programming method called grammatical evolution. The grammars consist of conditional statements or mathematical functions as are used in symbolic regression applications of GP, as well as a hybrid containing both kinds of statements. To benchmark and gain further information about our femtocell network simulation model we also perform random sampling and limited enumeration of femtocell pilot power settings. The symbolic regression based grammars require the most configuration of the evolutionary algorithm and more fitness evaluations, whereas the conditional statement grammar requires more domain knowledge to set the parameters. The content of the resulting femtocell algorithms shows that the evolutionary computation (EC) methods are exploiting the assumptions in the model. The ability of EC to exploit bias in both the fitness function and the underlying model is vital for identifying the current system and improves the model and the EC method. Finally, the results show that the best fitness and engineering performances for the grammars are similar over both test and training scenarios. In addition, the evolved solutions’ performance is superior to those designed by humans.     

Automatic generation of visual programming environments

We have developed the visual language compiler-compiler (VLCC) system to automatically generate visual programming environments. VLCC is a grammar based system that can support implementation of any visual language by assisting the language designer in defining the language's graphical objects, syntax, and semantics. The final result of the generation process includes an integrated environment with a visual editor and a compiler for the defined visual language. In VLCC, graphical tools define visual languages to create both graphical objects and composition rules. Visual editors enable language designers to directly and visually manipulate the syntax of these languages. To capture the widest range of visual languages, the VLCC system can be configured for a specific language class. Different language classes can be characterized depending on their graphical objects' structure and on the way they can be composed. Also, box and arrow diagrams are defined for primitive objects with attaching points and for composition rules to join boxes and arrows at those attaching points. After choosing the visual language type to create, the designer can concentrate on language definition details. VLCC uses the positional grammar model as its underlying grammar formalism