Depth-control strategies for crossover in tree-based genetic programming

The standard subtree crossover operator in the tree-based genetic programming (GP) has been considered as problematic. In order to improve the standard subtree crossover, controlling depth of crossover points becomes a research topic. However, the existence of many different and inconsistent crossover depth-control schemes and the possibility of many other depth-control schemes make the identification of good depth-control schemes a challenging problem. This paper aims to investigate general heuristics for making good depth-control schemes for crossover in tree-based GP. It analyses the patterns of depth of crossover points in good predecessor programs of five GP systems that use the standard subtree crossover and four approximations of the optimal crossover operator on three problems in different domains. The analysis results show that an effective depth-control scheme is problem-dependent and evolutionary stage-dependent, and that good crossover events have a strong preference for roots and (less strongly) bottoms of parent program trees. The results also show that some ranges of depths between the roots and the bottoms are also preferred, suggesting that unequal-depth-selection-probability strategies are better than equal-depth-selection-probability strategies.     

Dynamic page based crossover in linear genetic programming

Page-based linear genetic programming (GP) is proposed in which individuals are described in terms of a number of pages. Pages are expressed in terms of a fixed number of instructions, which is constant for all individuals in the population. Pairwise crossover results in the swapping of single pages, and thus, individuals are of a fixed number of instructions. Head-to-head comparison with Tree-structured GP and block-based linear GP indicates that the page-based approach evolves succinct solutions without penalizing generalization ability.     

Prediction of the Unified Parkinson’s Disease Rating Scale assessment using a genetic programming system with geometric semantic genetic operators

Unified Parkinson’s Disease Rating Scale (UPDRS) assessment is the most used scale for tracking Parkinson’s disease symptom progression. Nowadays, the tracking process requires a patient to undergo invasive and time-consuming specialized examinations in hospital clinics, under the supervision of trained medical staff. Thus, the process is costly and logistically inconvenient for both patients and clinicians. For this reason, new powerful computational tools, aimed at making the process more automatic, cheaper and less invasive, are becoming more and more a necessity. The purpose of this paper is to investigate the use of an innovative intelligent system based on genetic programming for the prediction of UPDRS assessment, using only data derived from simple, self-administered and non-invasive speech tests. The system we propose is called geometric semantic genetic programming and it is based on recently defined geometric semantic genetic operators. Experimental results, achieved using the largest database of Parkinson’s disease speech in existence (approximately 6000 recordings from 42 Parkinson’s disease patients, recruited in a six-month, multi-centre trial), show the appropriateness of the proposed system for the prediction of UPDRS assessment. In particular, the results obtained with geometric semantic genetic programming are significantly better than the ones produced by standard genetic programming and other state of the art machine learning methods both on training and unseen test data.     

A survey of semantic methods in genetic programming

Several methods to incorporate semantic awareness in genetic programming have been proposed in the last few years. These methods cover fundamental parts of the evolutionary process: from the population initialization, through different ways of modifying or extending the existing genetic operators, to formal methods, until the definition of completely new genetic operators. The objectives are also distinct: from the maintenance of semantic diversity to the study of semantic locality; from the use of semantics for constructing solutions which obey certain constraints to the exploitation of the geometry of the semantic topological space aimed at defining easy-to-search fitness landscapes. All these approaches have shown, in different ways and amounts, that incorporating semantic awareness may help improving the power of genetic programming. This survey analyzes and discusses the state of the art in the field, organizing the existing methods into different categories. It restricts itself to studies where semantics is intended as the set of output values of a program on the training data, a definition that is common to a rather large set of recent contributions. It does not discuss methods for incorporating semantic information into grammar-based genetic programming or approaches based on formal methods. The objective is keeping the community updated on this interesting research track, hoping to motivate new and stimulating contributions.     

Schema theory for genetic programming with one-point crossover and point mutation

We review the main results obtained in the theory of schemata in genetic programming (GP), emphasizing their strengths and weaknesses. Then we propose a new, simpler definition of the concept of schema for GP, which is closer to the original concept of schema in genetic algorithms (GAs). Along with a new form of crossover, one-point crossover, and point mutation, this concept of schema has been used to derive an improved schema theorem for GP that describes the propagation of schemata from one generation to the next. We discuss this result and show that our schema theorem is the natural counterpart for GP of the schema theorem for GAs, to which it asymptotically converges.     

A new crossover operator in genetic programming for object classification.

The crossover operator has been considered "the centre of the storm" in genetic programming (GP). However, many existing GP approaches to object recognition suggest that the standard GP crossover is not sufficiently powerful in producing good child programs due to the totally random choice of the crossover points. To deal with this problem, this paper introduces an approach with a new crossover operator in GP for object recognition, particularly object classification. In this approach, a local hill-climbing search is used in constructing good building blocks, a weight called looseness is introduced to identify the good building blocks in individual programs, and the looseness values are used as heuristics in choosing appropriate crossover points to preserve good building blocks. This approach is examined and compared with the standard crossover operator and the headless chicken crossover (HCC) method on a sequence of object classification problems. The results suggest that this approach outperforms the HCC, the standard crossover, and the standard crossover operator with hill climbing on all of these problems in terms of the classification accuracy. Although this approach spends a bit longer time than the standard crossover operator, it significantly improves the system efficiency over the HCC method.     

General schema theory for genetic programming with subtree-swapping crossover: Part II

This paper is the second part of a two-part paper which introduces a general schema theory for genetic programming (GP) with subtree-swapping crossover (Part I (Poli and McPhee, 2003)). Like other recent GP schema theory results, the theory gives an exact formulation (rather than a lower bound) for the expected number of instances of a schema at the next generation. The theory is based on a Cartesian node reference system, introduced in Part I, and on the notion of a variable-arity hyperschema, introduced here, which generalises previous definitions of a schema. The theory includes two main theorems describing the propagation of GP schemata: a microscopic and a macroscopic schema theorem. The microscopic version is applicable to crossover operators which replace a subtree in one parent with a subtree from the other parent to produce the offspring. Therefore, this theorem is applicable to Koza's GP crossover with and without uniform selection of the crossover points, as well as one-point crossover, size-fair crossover, strongly-typed GP crossover, context-preserving crossover and many others. The macroscopic version is applicable to crossover operators in which the probability of selecting any two crossover points in the parents depends only on the parents' size and shape. In the paper we provide examples, we show how the theory can be specialised to specific crossover operators and we illustrate how it can be used to derive other general results. These include an exact definition of effective fitness and a size-evolution equation for GP with subtree-swapping crossover.     

General schema theory for genetic programming with subtree-swapping crossover: part I

This is the first part of a two-part paper which introduces a general schema theory for genetic programming (GP) with subtree-swapping crossover. The theory is based on a Cartesian node reference system which makes it possible to describe programs as functions over the space N(2) and allows one to model the process of selection of the crossover points of subtree-swapping crossovers as a probability distribution over N(4). In Part I, we present these notions and models and show how they can be used to calculate useful quantities. In Part II we will show how this machinery, when integrated with other definitions, such as that of variable-arity hyperschema, can be used to construct a general and exact schema theory for the most commonly used types of GP.     

Semantically-based crossover in genetic programming: application to real-valued symbolic regression

We investigate the effects of semantically-based crossover operators in genetic programming, applied to real-valued symbolic regression problems. We propose two new relations derived from the semantic distance between subtrees, known as semantic equivalence and semantic similarity. These relations are used to guide variants of the crossover operator, resulting in two new crossover operators—semantics aware crossover (SAC) and semantic similarity-based crossover (SSC). SAC, was introduced and previously studied, is added here for the purpose of comparison and analysis. SSC extends SAC by more closely controlling the semantic distance between subtrees to which crossover may be applied. The new operators were tested on some real-valued symbolic regression problems and compared with standard crossover (SC), context aware crossover (CAC), Soft Brood Selection (SBS), and No Same Mate (NSM) selection. The experimental results show on the problems examined that, with computational effort measured by the number of function node evaluations, only SSC and SBS were significantly better than SC, and SSC was often better than SBS. Further experiments were also conducted to analyse the perfomance sensitivity to the parameter settings for SSC. This analysis leads to a conclusion that SSC is more constructive and has higher locality than SAC, NSM and SC; we believe these are the main reasons for the improved performance of SSC.     

Inter-deriving semantic artifacts for object-oriented programming

We present a new abstract machine for Abadi and Cardelli's untyped non-imperative calculus of objects. This abstract machine mechanically corresponds to both the reduction semantics (i.e., small-step operational semantics) and the natural semantics (i.e., big-step operational semantics) specified in Abadi and Cardelli's monograph. To move closer to actual implementations, which use environments rather than actual substitutions, we then represent methods as closures and we present three new semantic artifacts for a version of Abadi and Cardelli's calculus with explicit substitutions: a reduction semantics, an environment-based abstract machine, and a natural semantics (i.e., an interpreter) with environments. These three new semantic artifacts mechanically correspond to each other, and the two abstract machines are bisimilar. Their significance lies in the fact that they have not been designed from scratch and then proved correct; instead, they have been inter-derived.To illustrate the inter-derivation and to make this article stand-alone, we also comprehensively treat the example of negational normalization over Boolean formulas, in appendix.     

Locally geometric semantic crossover: a study on the roles of semantics and homology in recombination operators

This study presents an extensive account of Locally Geometric Semantic Crossover (LGX), a semantically-aware recombination operator for genetic programming (GP). LGX is designed to exploit the semantic properties of programs and subprograms, in particular the geometry of semantic space that results from distance-based fitness functions used predominantly in GP. When applied to a pair of parents, LGX picks in them at random a structurally common (homologous) locus, calculates the semantics of subprograms located at that locus, finds a procedure that is semantically medial with respect to these subprograms, and replaces them with that procedure. The library of procedures is prepared prior to the evolutionary run and indexed by a multidimensional structure (kd-tree) allowing for efficient search. The paper presents the rationale for LGX design and an extensive computational experiment concerning performance, computational cost, impact on program size, and capability of generalization. LGX is compared with six other operators, including conventional tree-swapping crossover, semantic-aware operators proposed in previous studies, and control methods designed to verify the importance of homology and geometry of the semantic space. The overall conclusion is that LGX, thanks to combination of the semantically medial operation with homology, improves the efficiency of evolutionary search, lowers the variance of performance, and tends to be more resistant to overfitting.     

Discovering approximate expressions of GPS geometric dilution of precision using genetic programming

Global Positioning System (GPS) has been used extensively in various fields. Geometric Dilution of Precision (GDOP) is an indicator showing how well the constellation of GPS satellites is organized geometrically, so as a reliability indicator presenting the GPS positioning accuracy. Traditional methods for calculating GPS GDOP need to solve the measurement equations where involve complicated matrix transformation and inversion. Some studies rephrase the calculation of GPS GDOP a regression problem and employ “black-boxed” machine learning methods for problem solving. However, the regression models obtained from such methods lack of expressivity for describing the relationships among variables. Making the structures of GDOP expressions visible is valuable because they can be further studied or tailored for specific GPS applications. This study employs the technique of genetic programming (GP) for the regression of GPS GDOP. The performance of GP working with various operators and parameter settings is studied and discussed. The experimental results show that GP generates precise models with better expressivity for GPS GDOP than other methods.     

Self-tuning geometric semantic Genetic Programming

The process of tuning the parameters that characterize evolutionary algorithms is difficult and can be time consuming. This paper presents a self-tuning algorithm for dynamically updating the crossover and mutation probabilities during a run of genetic programming. The genetic operators that are considered in this work are the geometric semantic genetic operators introduced by Moraglio et al. Differently from other existing self-tuning algorithms, the proposed one works by assigning a (different) crossover and mutation probability to each individual of the population. The experimental results we present show the appropriateness of the proposed self-tuning algorithm: on seven different test problems, the proposed algorithm finds solutions of a quality that is better than, or comparable to, the one achieved using the best known values for the geometric semantic crossover and mutation rates for the same problems. Also, we study how the mutation and crossover probabilities change during the execution of the proposed self-tuning algorithm, pointing out an interesting insight: mutation is basically the only operator used in the exploration phase, while crossover is used for exploitation, further improving good quality solutions.     

Prediction of high performance concrete strength using Genetic Programming with geometric semantic genetic operators

Concrete is a composite construction material made primarily with aggregate, cement, and water. In addition to the basic ingredients used in conventional concrete, high-performance concrete incorporates supplementary cementitious materials, such as fly ash and blast furnace slag, and chemical admixture, such as superplasticizer. Hence, high-performance concrete is a highly complex material and modeling its behavior represents a difficult task. In this paper, we propose an intelligent system based on Genetic Programming for the prediction of high-performance concrete strength. The system we propose is called Geometric Semantic Genetic Programming, and it is based on recently defined geometric semantic genetic operators for Genetic Programming. Experimental results show the suitability of the proposed system for the prediction of concrete strength. In particular, the new method provides significantly better results than the ones produced by standard Genetic Programming and other machine learning methods, both on training and on out-of-sample data.     

Progress properties and fitness bounds for geometric semantic search operators

Metrics are essential for geometric semantic genetic programming. On one hand, they structure the semantic space and govern the behavior of geometric search operators; on the other, they determine how fitness is calculated. The interactions between these two types of metrics are an important aspect that to date was largely neglected. In this paper, we investigate these interactions and analyze their consequences. We provide a systematic theoretical analysis of the properties of abstract geometric semantic search operators under Minkowski metrics of arbitrary order. For nine combinations of popular metrics (city-block, Euclidean, and Chebyshev) used in fitness functions and of search operators, we derive pessimistic bounds on fitness change. We also define three types of progress properties (weak, potential, and strong) and verify them for operators under those metrics. The analysis allows us to determine the combinations of metrics that are most attractive in terms of progress properties and deterioration bounds.     

Applying genetic programming and ant colony optimisation to improve the geometric design of a reflector

The lighting performance of an LED (light-emitting diode) flash is significantly influenced by the geometric form of a reflector. Previously, design engineers have usually determined the geometric design of a reflector according to the principles of optics and their own experience. Some real reflectors have then been created to verify the feasibility and performance of a certain geometric design. This, however, is a costly and time-consuming procedure. Furthermore, the geometric design of a reflector cannot be proven to be actually optimal. This study proposes a systematic approach based on genetic programming (GP) and ant colony optimisation (ACO), called the GP–ACO procedure, to improve the geometric design of a reflector. A case study is used to demonstrate the feasibility and effectiveness of the proposed optimisation procedure. The results show that all the crucial quality characteristics of an LED flash fulfil the required specifications; thus, the optimal geometric parameter settings of the reflector obtained can be directly applied to mass production. Consequently, the proposed GP–ACO procedure can be considered an effective method for resolving general multi-response parameter design problems.     

Semantic schema theory for genetic programming

Schema theory is the most well-known model of evolutionary algorithms. Imitating from genetic algorithms (GA), nearly all schemata defined for genetic programming (GP) refer to a set of points in the search space that share some syntactic characteristics. In GP, syntactically similar individuals do not necessarily have similar semantics. The instances of a syntactic schema do not behave similarly, hence the corresponding schema theory becomes unreliable. Therefore, these theories have been rarely used to improve the performance of GP. The main objective of this study is to propose a schema theory which could be a more realistic model for GP and could be potentially employed for improving GP in practice. To achieve this aim, the concept of semantic schema is introduced. This schema partitions the search space according to semantics of trees, regardless of their syntactic variety. We interpret the semantics of a tree in terms of the mutual information between its output and the target. The semantic schema is characterized by a set of semantic building blocks and their joint probability distribution. After introducing the semantic building blocks, an algorithm for finding them in a given population is presented. An extraction method that looks for the most significant schema of the population is provided. Moreover, an exact microscopic schema theorem is suggested that predicts the expected number of schema samples in the next generation. Experimental results demonstrate the capability of the proposed schema definition in representing the semantics of the schema instances. It is also revealed that the semantic schema theorem estimation is more realistic than previously defined schemata.     

Initialization method for grammar-guided genetic programming

This paper proposes a new tree-generation algorithm for grammar-guided genetic programming that includes a parameter to control the maximum size of the trees to be generated. An important feature of this algorithm is that the initial populations generated are adequately distributed in terms of tree size and distribution within the search space. Consequently, genetic programming systems starting from the initial populations generated by the proposed method have a higher convergence speed. Two different problems have been chosen to carry out the experiments: a laboratory test involving searching for arithmetical equalities and the real-world task of breast cancer prognosis. In both problems, comparisons have been made to another five important initialization methods.