Maze exploration behaviors using an integrated evolutionary robotics environment

This paper presents results generated with a new evolutionary robotics (ER) simulation environment and its complementary real mobile robot colony research test-bed. Neural controllers producing mobile robot maze searching and exploration behaviors using binary tactile sensors as inputs were evolved in a simulated environment and subsequently transferred to and tested on real robots in a physical environment. There has been a considerable amount of proof-of-concept and demonstration research done in the field of ER control in recent years, most of which has focused on elementary behaviors such as object avoidance and homing. Artificial neural networks (ANN) are the most commonly used evolvable controller paradigm found in current ER literature. Much of the research reported to date has been restricted to the implementation of very simple behaviors using small ANN controllers. In order to move beyond the proof-of-concept stage our ER research was designed to train larger more complicated ANN controllers, and to implement those controllers on real robots quickly and efficiently. To achieve this a physical robot test-bed that includes a colony of eight real robots with advanced computing and communication abilities was designed and built. The real robot platform has been coupled to a simulation environment that facilitates the direct wireless transfer of evolved neural controllers from simulation to real robots (and vice versa). We believe that it is the simultaneous development of ER computing systems in both the simulated and the physical worlds that will produce advances in mobile robot colony research. Our simulation and training environment development focuses on the definition and training of our new class of ANNs, networks that include multiple hidden layers, and time-delayed and recurrent connections. Our physical mobile robot design focuses on maximizing computing and communications power while minimizing robot size, weight, and energy usage. The simulation and ANN-evolution environment was developed using MATLAB. To allow for efficient control software portability our physical evolutionary robots (EvBots) are equipped with a PC-104-based computer running a custom distribution of Linux and connected to the Internet via a wireless network connection. In addition to other high-level computing applications, the mobile robots run a condensed version of MATLAB, enabling ANN controllers evolved in simulation to be transferred directly onto physical robots without any alteration to the code. This is the first paper in a series to be published cataloging our results in this field.     

Fitness functions in evolutionary robotics: A survey and analysis

This paper surveys fitness functions used in the field of evolutionary robotics (ER). Evolutionary robotics is a field of research that applies artificial evolution to generate control systems for autonomous robots. During evolution, robots attempt to perform a given task in a given environment. The controllers in the better performing robots are selected, altered and propagated to perform the task again in an iterative process that mimics some aspects of natural evolution. A key component of this process–one might argue, the key component–is the measurement of fitness in the evolving controllers. ER is one of a host of machine learning methods that rely on interaction with, and feedback from, a complex dynamic environment to drive synthesis of controllers for autonomous agents. These methods have the potential to lead to the development of robots that can adapt to uncharacterized environments and which may be able to perform tasks that human designers do not completely understand. In order to achieve this, issues regarding fitness evaluation must be addressed. In this paper we survey current ER research and focus on work that involved real robots. The surveyed research is organized according to the degree of a priori knowledge used to formulate the various fitness functions employed during evolution. The underlying motivation for this is to identify methods that allow the development of the greatest degree of novel control, while requiring the minimum amount of a priori task knowledge from the designer.     

Evolvable Hardware in Evolutionary Robotics

In recent decades the research on Evolutionary Robotics (ER) has developed rapidly. This direction is primarily concerned with the use of evolutionary computing techniques in the design of intelligent and adaptive controllers for robots. Meanwhile, much attention has been paid to a new set of integrated circuits named Evolvable Hardware (EHW), which is capable of reconfiguring its architectures unlimited time based on artificial evolution techniques. This paper surveys the application of evolvable hardware in evolutionary robotics. The evolvable hardware is an emerging research field concerning the development of evolvable robot controllers at the hardware level to adapt to dynamic changes in environments. The context of evolvable hardware and evolutionary robotics is reviewed, and a few representative experiments in the field of robotic hardware evolution are presented. As an alternative to conventional robotic controller designs, the potentialities and limitations of the EHW-based robotic system are discussed and summarized.     

Evolutionary robotics: the Sussex approach

We give an overview of evolutionary robotics research at Sussex over the last five years. We explain and justify our distinctive approaches to (artificial) evolution, and to the nature of robot control systems that are evolved. Results are presented from research with evolved controllers for autonomous mobile robots, simulated robots, co-evolved animats, real robots with software controllers, and a real robot with a controller directly evolved in hardware.     

Evolutionary design and behavior analysis of neuromodulatory neural networks for mobile robots control

Evolutionary Robotics (ER) is one of promising approaches to design robot controllers which essentially have complicated and/or complex properties. In most ER research, the sensory–motor mappings of robots are represented as artificial neural networks, and their connection weights (and sometimes the structure of the networks) can be optimized in the parameter spaces by using evolutionary computation. However, generally, the evolved neural controllers could be fragile in unexperienced environments, especially in real worlds, because the evolutionary optimization processes would be executed in idealized simulators. This is known as the gap problem between the simulated and real worlds. To overcome this, the author focused on evolving an on-line learning ability instead of weight parameters in a simulated environment. According to recent biological findings, actually, the kinds of on-line adaptation abilities can be found in real nervous systems of insects and crustaceans, and it is also known that a variety of neuromodulators (NMs) play crucial roles to regulate the network characteristics (i.e. activating/blocking/changing of synaptic connections). Based on this, a neuromodulatory neural network model was proposed and it was utilized as a mobile robot controller. In the paper, the detail behavior analysis of the evolved neuromodulatory neural network is also discussed.     

Evolving neural controllers that elicit nontrivial behaviors using a developmental process

Recently, evolutionary robotics (ER) has been attracting a lot of attention in the field of robotics artificial life and so on. ER approaches are expected to provide feasible methods to design controllers for autonomous mobile robots with less human intervention. However, most of the conventional studies in ER have been aiming at obtaining very simple (trivial) behaviors such as obstacle avoiding, wall following, and target approaching. To make the ER approach more fruitful, we should pay close attention to obtaining nontrivial behaviors. Based on these considerations, in this paper we propose a method for obtaining nontrivial behaviors using a developmental process with a carefully arranged grafting method. To verify the validity, we apply our idea to the construction of neural controllers that can cope with rough terrain. This work was presented, in part, at the Third International Symptosium on Artificial Life and Robotics, Oita, Japan, January 19–21, 1998     

Adaptive robotics in entertainment

Techniques developed in the fields of evolutionary computation, adaptive systems, agents, and artificial neural networks can be used in entertainment robotics in order to provide easy access to the robot technology. We have developed a number of user-guided approaches based on the techniques from these research fields. These techniques include user-guided behaviour-based systems, user-guided evolutionary robotics, user-guided co-evolutionary robotics, and morphological development. All these techniques are applied to allow children to develop their own robot behaviours in a very easy and fast manner. Here, I show examples with development of Khepera robots and LEGO MINDSTORMS robots, including the World Cup’98 stadium, the Co-evolutionary Robot Soccer Show, the Toybot Soccer Player, the LEGO Interactive Football, and RoboCup Junior Rescue.     

Encouraging behavioral diversity in evolutionary robotics: an empirical study

Evolutionary robotics (ER) aims at automatically designing robots or controllers of robots without having to describe their inner workings. To reach this goal, ER researchers primarily employ phenotypes that can lead to an infinite number of robot behaviors and fitness functions that only reward the achievement of the task-and not how to achieve it. These choices make ER particularly prone to premature convergence. To tackle this problem, several papers recently proposed to explicitly encourage the diversity of the robot behaviors, rather than the diversity of the genotypes as in classic evolutionary optimization. Such an approach avoids the need to compute distances between structures and the pitfalls of the noninjectivity of the phenotype/behavior relation; however, it also introduces new questions: how to compare behavior? should this comparison be task specific? and what is the best way to encourage diversity in this context? In this paper, we review the main published approaches to behavioral diversity and benchmark them in a common framework. We compare each approach on three different tasks and two different genotypes. The results show that fostering behavioral diversity substantially improves the evolutionary process in the investigated experiments, regardless of genotype or task. Among the benchmarked approaches, multi-objective methods were the most efficient and the generic, Hamming-based, behavioral distance was at least as efficient as task specific behavioral metrics.     

Artificial neural network based robot control: An overview

The current thrust of research in robotics is to build robots which can operate in dynamic and/or partially known environments. The ability of learning endows the robot with a form of autonomous intelligence to handle such situations. This paper focuses on the intersection of the fields of robot control and learning methods as represented by artificial neural networks. An in-depth overview of the application of neural networks to the problem of robot control is presented. Some typical neural network architectures are discussed first. The important issues involved in the study of robotics are then highlighted. This paper concentrates on the neural network applications to the motion control of robots involved in both non-contact and contact tasks. The current state of research in this area is surveyed and the strengths and weakness of the present approaches are emphasized. The paper concludes by indentifying areas which need future research work.     

A framework for embodied evolution with pre-evaluation applied to a biped robot

Embodied evolution (EE) is a methodology in evolutionary robotics in which, without simulations on a host computer, real robots evolve on the basis of their interactions with the actual environment. However, when adopting EE, we had to accept robot behavior with a low fitness, especially in the early generations. This article introduces pre-evaluation into the EE framework for a biped robot in order to restrain the behavior of a robot of which the fitness is estimated to be low, especially falling down onto the ground. We provide a comparative discussion on the conventional simulate-and-transfer method, the original EE method, and the proposed one in terms of calculation time, cost of fitness evaluation, and cost of simulation or modeling based on the evaluation experiments. We believe that the EE framework with pre-evaluation is applicable to a wide variety of optimization tasks in which the cost or the risk of fitness evaluation is not negligible.     

Simulating Robots Without Conventional Physics: A Neural Network Approach

The construction of physics-based simulators for use in Evolutionary Robotics (ER) can be complex and time-consuming. Alternative simulation schemes construct robotic simulators from empirically-collected data. Such empirical simulators, however, also have associated challenges. This paper therefore investigates the potential use of Artificial Neural Networks, henceforth simply referred to as Neural Networks (NNs), as alternative robotic simulators. In contrast to physics models, NN-based simulators can be constructed without requiring an explicit mathematical model of the system being modeled, which can simplify simulator development. The generalization abilities of NNs, along with NNs’ noise tolerance, suggest that NNs could be well-suited to application in robotics simulation. Investigating whether NNs can be effectively used as robotic simulators in ER is thus the endeavour of this work. Two robot morphologies were selected on which the NN simulators created in this work were based, namely a differentially steered robot and an inverted pendulum robot. Accuracy tests indicated that NN simulators created for these robots generally trained well and could generalize well on data not presented during simulator construction. In order to validate the feasibility of the created NN simulators in the ER process, these simulators were subsequently used to evolve controllers in simulation, similar to controllers developed in related studies. Encouraging results were obtained, with the newly-evolved controllers allowing experimental robots to exhibit obstacle avoidance, light-approaching behaviour and inverted pendulum stabilization. It was thus clearly established that NN-based robotic simulators can be successfully employed as alternative simulation schemes in the ER process.     

Design of a fuzzy controller in mobile robotics using genetic algorithms

The design of fuzzy controllers for the implementation of behaviors in mobile robotics is a complex and highly time-consuming task. The use of machine learning techniques, such as evolutionary algorithms or artificial neural networks for the learning of these controllers allows to automate the design process. In this paper, the automated design of a fuzzy controller using genetic algorithms for the implementation of the wall-following behavior in a mobile robot is described. The algorithm is based on the Iterative Rule Learning (IRL) approach, and a parameter (δ) is defined with the aim of selecting the relation between the number of rules and the quality and accuracy of the controller. The designer has to define the universe of discourse and the precision of each variable, and also the scoring function. No restrictions are placed neither in the number of linguistic labels nor in the values that define the membership functions.     

Human breeders for evolving robots

In this article we describe a new approach in evolutionary robotics according to which human breeders are involved in the evolutionary process. While traditionally robots are selected to reproduce automatically according to a fitness formula, which is a quantitative and strictly defined measure, human breeders can operate selection based on qualitative criteria, and rewarding behaviors that can slip between the meshes woven by the fitness formula. In authors’ opinion this may bring advantages to the evolutionary robotics methodology, allowing the production of robots that display more, and more multiform, behaviors. In order to illustrate this approach, the software Breedbot was developed in which human breeders can intervene in evolving robots, complementing the automatic evaluation. After describing the software, some results on sample evolutionary processes are reported showing that the joint use of human and artificial selection on an exploration task generates robots with a higher performance and in a shorter time compared with the exclusive action of each breeding method. Future work will explore this hypothesis further. This work was presented in part at the First European Workshop on Artificial Life and Robotics, Vienna, Austria, July 12–13, 2007     

Evolutionary learning of a fuzzy controller for wall-following behavior in mobile robotics

The design of fuzzy controllers for the implementation of behaviors in mobile robotics is a complex and highly time-consuming task. The use of machine learning techniques such as evolutionary algorithms or artificial neural networks for the learning of these controllers allows to automate the design process. In this paper, the automated design of a fuzzy controller using genetic algorithms for the implementation of the wall-following behavior in a mobile robot is described. The algorithm is based on the iterative rule learning approach, and is characterized by three main points. First, learning has no restrictions neither in the number of membership functions, nor in their values. In the second place, the training set is composed of a set of examples uniformly distributed along the universe of discourse of the variables. This warrantees that the quality of the learned behavior does not depend on the environment, and also that the robot will be capable to face different situations. Finally, the trade off between the number of rules and the quality/accuracy of the controller can be adjusted selecting the value of a parameter. Once the knowledge base has been learned, a process for its reduction and tuning is applied, increasing the cooperation between rules and reducing its number.     

Further explorations in evolutionary humanoid robotics

The field of evolutionary humanoid robotics is a branch of evolutionary robotics specifically dealing with the application of evolutionary principles to humanoid robot design. Previous studies demonstrated the possible future potential of this approach by evolving walking behaviors for simulated humanoid robots with up to 20 degrees of freedom. In this paper we examine further the evolutionary process by looking at the changes in diversity over time. We then investigate the effect of the immobilization of an individual joint or joints in the robot. The latter study may be of potential future use in prosthetic design. We also explore the possibility of the evolution of humanoid robots which can cope with different environmental conditions. These include reduced ground friction (ice) and modified gravitation (moon walking). We present initial results on the implementation of our simulated humanoid robots in hardware using the Bioloid robotic platform, using a model of this robot in order to evolve the desired motion patterns, for subsequent transfer to the real robot. We finish the article with a summary and brief discussion of future work. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

Educational training software for modelling and synthesis of controllers for robotic manipulators and robotized manufacturing cells

A general software system aimed at computer-aided design of controllers for robots and robotized technological systems is described in this paper. The software system includes modules for the synthesis of various levels of robot controller as well as controllers of complex robotized technological systems. The software includes simulation of robotic systems within manufacturing cells using various types of models: complete dynamic models, kinematic models and simple models in the form of finite automata. Using these modelsvarious algorithms for all controls levels in robot controllers may be synthesized taking into account the actual interaction between the robot and its environment. The software system enables the solution of the important problem of the interaction between higher and lower levels of controllers. Finally, a general purpose controller as a target system for the proposed software is described. The controller is designed as an open system allowing the user to apply various control laws and to run in conjunction with an actual robot. The general software system together with the controller represents a powerful educational tool in modern robotics.     

Emergent behaviors of a fuzzy sensory-motor controller evolved bygenetic algorithm

Recently, there has been extensive work on the construction of fuzzy controllers for mobile robots by a genetic algorithm (GA); therefore, we can realize evolutionary optimization as a promising method for developing fuzzy controllers. However, much investigation on the evolutionary fuzzy controller remains because most of the previous works have not seriously attempted to analyze the fuzzy controller obtained by evolution. This paper develops a fuzzy logic controller for a mobile robot with a GA in simulation environments and analyzes the behaviors of the controller with a state transition diagram of the internal model. Experimental results show that appropriate control mechanisms of the fuzzy controller are obtained by evolution. The controller has evolved wen enough to smoothly drive the robot in different environments. The robot produces emergent behaviors by the interaction of several fuzzy rules obtained.     

Concurrent controller and Simulator Neural Network development for a differentially-steered robot in Evolutionary Robotics

Evolutionary Robotics (ER) strives for the automatic creation of robotic controllers and morphologies. The ER process is normally performed in simulation in order to reduce the time required and robot wear. Simulator development is a time consuming process which requires expert knowledge and must traditionally be completed before the ER process can commence. Traditional simulators have limited accuracy, can be computationally expensive and typically do not account for minor operational differences between physical robots.This research proposes the automatic creation of simulators concurrently with the normal ER process. The simulator is derived from an Artificial Neural Network (ANN) to remove the need for formulating an analytical model for the robot. The ANN simulator is improved concurrently with the ER process through real-world controller evaluations which continuously generate behavioural data. Simultaneously, the ER process is informed by the improving simulator to evolve better controllers which are periodically evaluated in the real-world. Hence, the concurrent processes provide further targeted behavioural data for simulator improvement.The concurrent and real-time creation of both controllers and ANN-based simulators is successfully demonstrated for a differentially-steered mobile robot. Various parameter settings in the proposed algorithm are investigated to determine factors pertinent to the success of the proposed approach.     

EDHMoR: Evolutionary designer of heterogeneous modular robots

This paper is devoted to the problem of automatically designing feasible and manufacturable robots made up of heterogeneous modules. Specifically, the coevolution of morphology and control in robots is analyzed and a particular strategy to address this problem is contemplated. To this end, the main issues of this approach such as encoding, evaluation or transfer to reality are studied through the use of heterogeneous modular structures with distributed control. We also propose a constructive evolutionary algorithm based on tree-like representations of the morphology that can intrinsically provide for a type of generative evolutionary approach. The algorithm introduces some new elements to smooth the search space and make finding solutions much easier. The evaluation of the individuals is carried out in simulations and then transferred to real robots assembled from the modules considered. To this end, the extension of the principles proposed by classical authors in traditional evolutionary robotics to brain–body evolution regarding how simulations should be set up so that robust behaviors that can be transferred to reality are obtained is considered here. All these issues are analyzed by means of an evolutionary design system called EDHMoR (Evolutionary Designer of Heterogeneous Modular Robots) that contains all the elements involved in this process. To show practical evidences of the conclusions that have been extracted with this work, two benchmark problems in modular robotics are considered and EDHMoR is tested over them. The first one is focused on solving a linear robot motion mission and the second one on a static task of the robot that does not require displacements.     

The evolution of robotics research

This article surveys traditional research topics in industrial robotics and mobile robotics and then expands on new trends in robotics research that focus more on the interaction between human and robot. The new trends in robotics research have been denominated service robotics because of their general goal of getting robots closer to human social needs, and this article surveys research on service robotics such as medical robotics, rehabilitation robotics, underwater robotics, field robotics, construction robotics and humanoid robotics. The aim of this article is to provide an overview of the evolution of research topics in robotics from classical motion control for industrial robots to modern intelligent control techniques and social learning paradigms, among other aspects