Grammatical evolution as a hyper-heuristic to evolve deterministic real-valued optimization algorithms

Hyper-heuristic methodologies have been extensively and successfully used to generate combinatorial optimization heuristics. On the other hand, there have been almost no attempts to build a hyper-heuristic to evolve an algorithm for solving real-valued optimization problems. In our previous research, we succeeded to evolve a Nelder–Mead-like real function minimization heuristic using genetic programming and the primitives extracted from the original Nelder–Mead algorithm. The resulting heuristic was better than the original Nelder–Mead method in the number of solved test problems but it was slower in that it needed considerably more cost function evaluations to solve the problems also solved by the original method. In this paper we exploit grammatical evolution as a hyper-heuristic to evolve heuristics that outperform the original Nelder–Mead method in all aspects. However, the main goal of the paper is not to build yet another real function optimization algorithm but to shed some light on the influence of different factors on the behavior of the evolution process as well as on the quality of the obtained heuristics. In particular, we investigate through extensive evolution runs the influence of the shape and dimensionality of the training function, and the impact of the size limit set to the evolving algorithms. At the end of this research we succeeded to evolve a number of heuristics that solved more test problems and in fewer cost function evaluations than the original Nelder–Mead method. Our solvers are also highly competitive with the improvements made to the original method based on rigorous mathematical convergence proofs found in the literature. Even more importantly, we identified some directions in which to continue the work in order to be able to construct a productive hyper-heuristic capable of evolving real function optimization heuristics that would outperform a human designer in all aspects.