Maze exploration behaviors using an integrated evolutionary robotics environment

This paper presents results generated with a new evolutionary robotics (ER) simulation environment and its complementary real mobile robot colony research test-bed. Neural controllers producing mobile robot maze searching and exploration behaviors using binary tactile sensors as inputs were evolved in a simulated environment and subsequently transferred to and tested on real robots in a physical environment. There has been a considerable amount of proof-of-concept and demonstration research done in the field of ER control in recent years, most of which has focused on elementary behaviors such as object avoidance and homing. Artificial neural networks (ANN) are the most commonly used evolvable controller paradigm found in current ER literature. Much of the research reported to date has been restricted to the implementation of very simple behaviors using small ANN controllers. In order to move beyond the proof-of-concept stage our ER research was designed to train larger more complicated ANN controllers, and to implement those controllers on real robots quickly and efficiently. To achieve this a physical robot test-bed that includes a colony of eight real robots with advanced computing and communication abilities was designed and built. The real robot platform has been coupled to a simulation environment that facilitates the direct wireless transfer of evolved neural controllers from simulation to real robots (and vice versa). We believe that it is the simultaneous development of ER computing systems in both the simulated and the physical worlds that will produce advances in mobile robot colony research. Our simulation and training environment development focuses on the definition and training of our new class of ANNs, networks that include multiple hidden layers, and time-delayed and recurrent connections. Our physical mobile robot design focuses on maximizing computing and communications power while minimizing robot size, weight, and energy usage. The simulation and ANN-evolution environment was developed using MATLAB. To allow for efficient control software portability our physical evolutionary robots (EvBots) are equipped with a PC-104-based computer running a custom distribution of Linux and connected to the Internet via a wireless network connection. In addition to other high-level computing applications, the mobile robots run a condensed version of MATLAB, enabling ANN controllers evolved in simulation to be transferred directly onto physical robots without any alteration to the code. This is the first paper in a series to be published cataloging our results in this field.     

Humanoid robot arm performance optimization using multi objective evolutionary algorithm

As humanoid robots are expected to operate in human environments they are expected to perform a wide range of tasks. Therefore, the robot arm motion must be generated based on the specific task. In this paper we propose an optimal arm motion generation satisfying multiple criteria. In our method, we evolved neural controllers that generate the humanoid robot arm motion satisfying three different criteria; minimum time, minimum distance and minimum acceleration. The robot hand is required to move from the initial to the final goal position. In order to compare the performance, single objective GA is also considered as an optimization tool. Selected neural controllers from the Pareto solution are implemented and their performance is evaluated. Experimental investigation shows that the evolved neural controllers performed well in the real hardware of the mobile humanoid robot platform.     

Active vision and receptive field development in evolutionary robots

In this paper, we describe the artificial evolution of adaptive neural controllers for an outdoor mobile robot equipped with a mobile camera. The robot can dynamically select the gazing direction by moving the body and/or the camera. The neural control system, which maps visual information to motor commands, is evolved online by means of a genetic algorithm, but the synaptic connections (receptive fields) from visual photoreceptors to internal neurons can also be modified by Hebbian plasticity while the robot moves in the environment. We show that robots evolved in physics-based simulations with Hebbian visual plasticity display more robust adaptive behavior when transferred to real outdoor environments as compared to robots evolved without visual plasticity. We also show that the formation of visual receptive fields is significantly and consistently affected by active vision as compared to the formation of receptive fields with grid sample images in the environment of the robot. Finally, we show that the interplay between active vision and receptive field formation amounts to the selection and exploitation of a small and constant subset of visual features available to the robot.     

Evolutionary design and behavior analysis of neuromodulatory neural networks for mobile robots control

Evolutionary Robotics (ER) is one of promising approaches to design robot controllers which essentially have complicated and/or complex properties. In most ER research, the sensory–motor mappings of robots are represented as artificial neural networks, and their connection weights (and sometimes the structure of the networks) can be optimized in the parameter spaces by using evolutionary computation. However, generally, the evolved neural controllers could be fragile in unexperienced environments, especially in real worlds, because the evolutionary optimization processes would be executed in idealized simulators. This is known as the gap problem between the simulated and real worlds. To overcome this, the author focused on evolving an on-line learning ability instead of weight parameters in a simulated environment. According to recent biological findings, actually, the kinds of on-line adaptation abilities can be found in real nervous systems of insects and crustaceans, and it is also known that a variety of neuromodulators (NMs) play crucial roles to regulate the network characteristics (i.e. activating/blocking/changing of synaptic connections). Based on this, a neuromodulatory neural network model was proposed and it was utilized as a mobile robot controller. In the paper, the detail behavior analysis of the evolved neuromodulatory neural network is also discussed.     

Using direct competition to select for competent controllers in evolutionary robotics

Evolutionary robotics (ER) is a field of research that applies artificial evolution toward the automatic design and synthesis of intelligent robot controllers. The preceding decade saw numerous advances in evolutionary robotics hardware and software systems. However, the sophistication of resulting robot controllers has remained nearly static over this period of time. Here, we make the case that current methods of controller fitness evaluation are primary factors limiting the further development of ER. To address this, we define a form of fitness evaluation that relies on intra-population competition. In this research, complex neural networks were trained to control robots playing a competitive team game. To limit the amount of human bias or know-how injected into the evolving controllers, selection was based on whether controllers won or lost games. The robots relied on video sensing of their environment, and the neural networks required on the order of 150 inputs. This represents an order of magnitude increase in sensor complexity compared to other research in this field. Evolved controllers were tested extensively in real fully-autonomous robots and in simulation. Results and experiments are presented to characterize the training process and the acquisition of controller competency under different evolutionary conditions.     

Aibo and Webots: Simulation,wireless remote control and controller transfer

This article introduces a new software tool that provides an accurate simulation of Sony Aibo robots and the capability to transfer controller programs from the simulation to the real robot. Five components are described: (1) a simulated physics-based model of the Sony Aibo ERS-210(A) and ERS-7 quadruped robots; (2) a graphical user interface for controlling the simulated and real robots; (3) a wireless communication protocol for controlling the robot from within Webots; (4) software components on the robot that enable remote control; and (5) a method for cross-compiling Webots robot controllers. The complete system has been calibrated and proof tested. It enables simultaneous control of both a simulated and a real Aibo robot and provides the user with a platform for convenient robot programming without any knowledge of the underlying robot firmware.     

Development of a flexible autonomous robot without sensors or controllers

Recently, various robots with many degrees of freedom, such as rescue robots and domestic robots, have been developed and used in practical applications. It is difficult to control such robots autonomously in real environments, because in order to control the many degrees of freedom, we have to observe many states, calculate huge amounts of information, and operate many actuators. In this study, we consider a flexible robot without sensors or controllers that can determine the inclination of a slope and climb up the slope. In order to demonstrate the effectiveness of the proposed framework, we have developed a prototype robot and conducted experiments. The result indicates that the robot could determine the inclination and climb up a gentle slope autonomously. Thus, we have realized an autonomous robot that has no explicit sensors or controllers.     

Evolution of fuzzy behaviors for multi-robotic system

In a multi-robotic system, robots interact with each other in a dynamically changing environment. The robots need to be intelligent both at the individual and group levels. In this paper, the evolution of a fuzzy behavior-based architecture is discussed. The behavior-based architecture decomposes the complicated interactions of multiple robots into modular behaviors at different complexity levels. The fuzzy logic approach brings in human-like reasoning to the behavior construction, selection and coordination. Various behaviors in the fuzzy behavior-based architecture are evolved by genetic algorithm (GA). At the lowest level of the architecture hierarchy, the evolved fuzzy controllers enhanced the smoothness and accuracy of the primitive robot actions. At a higher level, the individual robot behaviors have become more skillful after the evolution. At the topmost level, the evolved group behaviors have resulted in aggressive competition strategy. The simulation and real-world experimentation on a robot-soccer system justify the effectiveness of the approach.     

Design and control of tensegrity robots for locomotion

The static properties of tensegrity structures have been widely appreciated in civil engineering as the basis of extremely lightweight yet strong mechanical structures. However, the dynamic properties and their potential utility in the design of robots have been relatively unexplored. This paper introduces robots based on tensegrity structures, which demonstrate that the dynamics of such structures can be utilized for locomotion. Two tensegrity robots are presented: TR3, based on a triangular tensegrity prism with three struts, and TR4, based on a quadrilateral tensegrity prism with four struts. For each of these robots, simulation models are designed, and automatic design of controllers for forward locomotion are performed in simulation using evolutionary algorithms. The evolved controllers are shown to be able to produce static and dynamic gaits in both robots. A real-world tensegrity robot is then developed based on one of the simulation models as a proof of concept. The results demonstrate that tensegrity structures can provide the basis for lightweight, strong, and fault-tolerant robots with a potential for a variety of locomotor gaits.     

Global behavior via cooperative local control

The purpose of this paper is twofold. First, we outline important issues in designing real-time controllers for robots with numerous sensors, actuators, and behaviors. We address these issues by implementing a behavior based controller on a sophisticated autonomous robot. Hence, this work provides a point of reference for the scalability, ease of design, and effectiveness of the behavior based control for complex robots. Second, we explore the viability of using cooperation among local controllers to achieve coherent global behavior. Our approach is to decompose a difficult control task for a complex robot into a multitude of simpler control tasks for robotic subsystems. We illustrate and examine the effectiveness of this approach via rough terrain locomotion using an autonomous hexapod robot. Traversing rough terrain is a good task to test the viability of this approach because it requires a considerable amount of leg coordination. We found that implementing a complicated global control task with cooperating local controllers can effectively control complex robots.Support for this research was provided in part by a NASA Graduate Student Researcher Program Fellowship administered through the Jet Propulsion Laboratory, by Jet Propulsion Laboratory grant 959333, and by the Advanced Research Projects Agency under Office of Naval Research contract N00014-91-J-4038.     

Combining Simulation and Reality in Evolutionary Robotics

Evolutionary Robotics (ER) is a promising methodology, intended for the autonomous development of robots, in which their behaviors are obtained as a consequence of the structural coupling between robot and environment. It is essential that there be a great amount of interaction to generate complex behaviors. Thus, nowadays, it is common to use simulation to speed up the learning process; however simulations are achieved from arbitrary off-line designs, rather than from the result of embodied cognitive processes. According to the reality gap problem, controllers evolved in simulation usually do not allow the same behavior to arise once transferred to the real robot. Some preliminary approaches for combining simulation and reality exist in the ER literature; nonetheless, there is no satisfactory solution available. In this work we discuss recent advances in neuroscience as a motivation for the use of environmentally adapted simulations, which can be achieved through the co-evolution of robot behavior and simulator. We present an algorithm in which only the differences between the behavior fitness obtained in reality versus that obtained in simulations are used as feedback for adapting a simulation. The proposed algorithm is experimentally validated by showing the successful development and continuous transference to reality of two complex low-level behaviors with Sony AIBO¹ robots: gait optimization and ball-kicking behavior. ¹AIBO is a trademark of Sony Corporation     

Evolving reliable and robust controllers for real robots by genetic programming

 Using Genetic Programming (GP)-based approaches to evolve robot controllers has the advantage of operating variable-size genotype. This is an important feature for evolving robot control systems as it allows complete freedom for the control architecture in respect to the task complexity which is difficult to predict. However, GP-based work in evolving controllers has been questioned in the verification of the performance on real robots, the generalisation of defining primitives, and the computational cost needed. In this paper, we present our GP framework in which a special representation of the robot controller is designed; this representation can capture well the characteristic of a behaviour controller so that our system can efficiently evolve desired robot behaviours by a relatively low computational cost. This system has been successfully used to evolve reliable and robust controllers working on a real robot, for a variety of tasks.     

Emergent behaviors of a fuzzy sensory-motor controller evolved bygenetic algorithm

Recently, there has been extensive work on the construction of fuzzy controllers for mobile robots by a genetic algorithm (GA); therefore, we can realize evolutionary optimization as a promising method for developing fuzzy controllers. However, much investigation on the evolutionary fuzzy controller remains because most of the previous works have not seriously attempted to analyze the fuzzy controller obtained by evolution. This paper develops a fuzzy logic controller for a mobile robot with a GA in simulation environments and analyzes the behaviors of the controller with a state transition diagram of the internal model. Experimental results show that appropriate control mechanisms of the fuzzy controller are obtained by evolution. The controller has evolved wen enough to smoothly drive the robot in different environments. The robot produces emergent behaviors by the interaction of several fuzzy rules obtained.     

Artificial evolution and real robots

Artificial evolution as a design methodology allows the relaxation of many of the constraints that have held back conventional methods. It does not require a complete prior analysis and decomposition of the task to be tackled, as human designers require. However, this freedom comes at some cost; there are a whole new set of issues relating to evolution that must be considered. Standard genetic algorithms may not be appropriate for incremental evolution of robot controllers. Species adaptation genetic algorithms, (SAGA) have been developed to meet these special needs. The main cost of an evolutionary approach is the large number of trials that are required. Simulations-especially those involving vision in complex environments, or modeling detailed semiconductor physics—may not be adequate or practical. Examples of evolved robots will be discussed, including a specialized piece of equipment which allows a robot to be tested using simple vision in real time, and what is believed to be the first successful example of an evolved hardware controller for a robot. Presented at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

Abstraction and control for Groups of robots

This paper addresses the general problem of controlling a large number of robots required to move as a group. We propose an abstraction based on the definition of a map from the configuration space Q of the robots to a lower dimensional manifold A, whose dimension is independent of the number of robots. In this paper, we focus on planar fully actuated robots. We require that the manifold A has a product structure A=G/spl times/S, where G is a Lie group, which captures the position and orientation of the ensemble in the chosen world coordinate frame, and S is a shape manifold, which is an intrinsic characterization of the team describing the "shape" as the area spanned by the robots. We design decoupled controllers for the group and shape variables. We derive controllers for individual robots that guarantee the desired behavior on A. These controllers can be realized by feedback that depends only on the current state of the robot and the state of the manifold A. This has the practical advantage of reducing the communication and sensing that is required and limiting the complexity of individual robot controllers, even for large numbers of robots.     

Design of a Mobile Robot Controller using Esterel Tools

Robot controllers are often programmed using either standard sequential programming languages or a robot-specific language, which are then compiled to assembly language specific to the robot. Modern real-time programming languages, on the other hand, are more appropriate to program robots, as they better fit the real-time reactive model of robots. This paper reports on a project to program a non-trivial robot, the Rug Warrior, in the Artificial Intelligence Laboratory of UNSW, using Esterel, which is a real-time programming language. The approach is illustrated by simulation of a colony of Siberian ants using a group of Rug Warriors.     

Towards principled experimental study of autonomous mobile robots

We review the current state of research in autonomous mobile robots and conclude that there is an inadequate basis for predicting the reliability and behavior of robots operating in unengineered environments. We present a new approach to the study of autonomous mobile robot performance based on formal statistical analysis of independently reproducible experiments conducted on real robots. Simulators serve as models rather than experimental surrogates. We demonstrate three new results: 1) Two commonly used performance metrics (time and distance) are not as well correlated as is often tacitly assumed. 2) The probability distributions of these performance metrics are exponential rather than normal, and 3) a modular, object-oriented simulation accurately predicts the behavior of the real robot in a statistically significant manner.     

Self-Organizing Sync in a Robotic Swarm: A Dynamical System View

Self-organized synchronization is a common phenomenon observed in many natural and artificial systems: simple coupling rules at the level of the individual components of the system result in an overall coherent behavior. Owing to these properties, synchronization appears particularly interesting for swarm robotics systems, as it allows robust temporal coordination of the group while minimizing the complexity of the individual controllers. The goal of the experiments presented in this paper is the study of self-organizing synchronization for robots that present an individual periodic behavior. In order to design the robot controllers, we make use of artificial evolution, which proves to be capable of synthesizing minimal synchronization strategies based on the dynamical coupling between robots and environment. The obtained results are analyzed under a dynamical system perspective, which allows us to uncover the evolved mechanisms and to predict the scalability properties of the self-organizing synchronization with respect to varying group size.      

Adaptation technique for integrating genetic programming and reinforcement learning for real robots

We propose an integrated technique of genetic programming (GP) and reinforcement learning (RL) to enable a real robot to adapt its actions to a real environment. Our technique does not require a precise simulator because learning is achieved through the real robot. In addition, our technique makes it possible for real robots to learn effective actions. Based on this proposed technique, we acquire common programs, using GP, which are applicable to various types of robots. Through this acquired program, we execute RL in a real robot. With our method, the robot can adapt to its own operational characteristics and learn effective actions. In this paper, we show experimental results from two different robots: a four-legged robot "AIBO" and a humanoid robot "HOAP-1." We present results showing that both effectively solved the box-moving task; the end result demonstrates that our proposed technique performs better than the traditional Q-learning method.     

Development of membrane controllers for mobile robots

The main contribution of this paper is the introduction of the new concept of membrane controller based on the structure and functioning of a deterministic numerical P system. The procedure for developing a membrane controller and for using it to control a mobile robot is explained and several test cases are given in which membrane controllers are used to control both simulated and real mobile robots and to generate various desired behaviours (obstacle avoidance, wall following, and follow the leader). The experiments reported in this paper validate the concept and prove that the performance of a membrane controller is comparable to or better than that of other controllers (such as fuzzy logic controllers).