Using direct competition to select for competent controllers in evolutionary robotics

Evolutionary robotics (ER) is a field of research that applies artificial evolution toward the automatic design and synthesis of intelligent robot controllers. The preceding decade saw numerous advances in evolutionary robotics hardware and software systems. However, the sophistication of resulting robot controllers has remained nearly static over this period of time. Here, we make the case that current methods of controller fitness evaluation are primary factors limiting the further development of ER. To address this, we define a form of fitness evaluation that relies on intra-population competition. In this research, complex neural networks were trained to control robots playing a competitive team game. To limit the amount of human bias or know-how injected into the evolving controllers, selection was based on whether controllers won or lost games. The robots relied on video sensing of their environment, and the neural networks required on the order of 150 inputs. This represents an order of magnitude increase in sensor complexity compared to other research in this field. Evolved controllers were tested extensively in real fully-autonomous robots and in simulation. Results and experiments are presented to characterize the training process and the acquisition of controller competency under different evolutionary conditions.     

Evolvable hardware: using evolutionary computation to design and optimize hardware systems

Evolvable hardware lies at the intersection of evolutionary computation and physical design. Through the use of evolutionary computation methods, the field seeks to develop a variety of technologies that enable automatic design, adaptation, and reconfiguration of electrical and mechanical hardware systems in ways that outperform conventional techniques. This article surveys evolvable hardware with emphasis on some of the latest developments, many of which deliver performance exceeding traditional methods. As such, the field of evolvable hardware is just now starting to emerge from the research laboratory and into mainstream hardware applications.     

Evolutionary robotics: the Sussex approach

We give an overview of evolutionary robotics research at Sussex over the last five years. We explain and justify our distinctive approaches to (artificial) evolution, and to the nature of robot control systems that are evolved. Results are presented from research with evolved controllers for autonomous mobile robots, simulated robots, co-evolved animats, real robots with software controllers, and a real robot with a controller directly evolved in hardware.     

硬件进化中演化算法的研究及应用

详细介绍了硬件进化的概念,硬件进化的原理与实现思想,遗传算法与蚁群算法动态融合的基本原理,融合后算法中遗传算法及蚁群算法规则.融合过程中遗传算法与蚁群算法动态衔接问题以及融合后的算法在硬件进化中的应用过程.最后,分析了通过该算法进化后硬件的进化应用前景.     

Evolving autonomous specialization in congested path formation task of robotic swarms

Redundancy in the number of robots is a fundamental feature of robotic swarms to confer robustness, flexibility, and scalability. However, robots tend to interfere with each other in a case, where multiple robots gather in a spatially limited environment. The aim of this paper is to understand how a robotic swarm develops an effective strategy to manage congestion. The controllers of the robots are obtained by an evolutionary robotics approach. The strategy of managing congestion is observed in the process of generating a collective path of robots visiting two landmarks alternately. The robotic swarm exhibits autonomous specialization that the robots traveling inside the path activate the LEDs, while the robots in the outer side deactivate them. We found that the congestion is regulated in an emergent way of autonomous specialization by the result of an artificial evolution.     

Fitness functions in evolutionary robotics: A survey and analysis

This paper surveys fitness functions used in the field of evolutionary robotics (ER). Evolutionary robotics is a field of research that applies artificial evolution to generate control systems for autonomous robots. During evolution, robots attempt to perform a given task in a given environment. The controllers in the better performing robots are selected, altered and propagated to perform the task again in an iterative process that mimics some aspects of natural evolution. A key component of this process–one might argue, the key component–is the measurement of fitness in the evolving controllers. ER is one of a host of machine learning methods that rely on interaction with, and feedback from, a complex dynamic environment to drive synthesis of controllers for autonomous agents. These methods have the potential to lead to the development of robots that can adapt to uncharacterized environments and which may be able to perform tasks that human designers do not completely understand. In order to achieve this, issues regarding fitness evaluation must be addressed. In this paper we survey current ER research and focus on work that involved real robots. The surveyed research is organized according to the degree of a priori knowledge used to formulate the various fitness functions employed during evolution. The underlying motivation for this is to identify methods that allow the development of the greatest degree of novel control, while requiring the minimum amount of a priori task knowledge from the designer.     

Further explorations in evolutionary humanoid robotics

The field of evolutionary humanoid robotics is a branch of evolutionary robotics specifically dealing with the application of evolutionary principles to humanoid robot design. Previous studies demonstrated the possible future potential of this approach by evolving walking behaviors for simulated humanoid robots with up to 20 degrees of freedom. In this paper we examine further the evolutionary process by looking at the changes in diversity over time. We then investigate the effect of the immobilization of an individual joint or joints in the robot. The latter study may be of potential future use in prosthetic design. We also explore the possibility of the evolution of humanoid robots which can cope with different environmental conditions. These include reduced ground friction (ice) and modified gravitation (moon walking). We present initial results on the implementation of our simulated humanoid robots in hardware using the Bioloid robotic platform, using a model of this robot in order to evolve the desired motion patterns, for subsequent transfer to the real robot. We finish the article with a summary and brief discussion of future work. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

Maze exploration behaviors using an integrated evolutionary robotics environment

This paper presents results generated with a new evolutionary robotics (ER) simulation environment and its complementary real mobile robot colony research test-bed. Neural controllers producing mobile robot maze searching and exploration behaviors using binary tactile sensors as inputs were evolved in a simulated environment and subsequently transferred to and tested on real robots in a physical environment. There has been a considerable amount of proof-of-concept and demonstration research done in the field of ER control in recent years, most of which has focused on elementary behaviors such as object avoidance and homing. Artificial neural networks (ANN) are the most commonly used evolvable controller paradigm found in current ER literature. Much of the research reported to date has been restricted to the implementation of very simple behaviors using small ANN controllers. In order to move beyond the proof-of-concept stage our ER research was designed to train larger more complicated ANN controllers, and to implement those controllers on real robots quickly and efficiently. To achieve this a physical robot test-bed that includes a colony of eight real robots with advanced computing and communication abilities was designed and built. The real robot platform has been coupled to a simulation environment that facilitates the direct wireless transfer of evolved neural controllers from simulation to real robots (and vice versa). We believe that it is the simultaneous development of ER computing systems in both the simulated and the physical worlds that will produce advances in mobile robot colony research. Our simulation and training environment development focuses on the definition and training of our new class of ANNs, networks that include multiple hidden layers, and time-delayed and recurrent connections. Our physical mobile robot design focuses on maximizing computing and communications power while minimizing robot size, weight, and energy usage. The simulation and ANN-evolution environment was developed using MATLAB. To allow for efficient control software portability our physical evolutionary robots (EvBots) are equipped with a PC-104-based computer running a custom distribution of Linux and connected to the Internet via a wireless network connection. In addition to other high-level computing applications, the mobile robots run a condensed version of MATLAB, enabling ANN controllers evolved in simulation to be transferred directly onto physical robots without any alteration to the code. This is the first paper in a series to be published cataloging our results in this field.     

基于EHW的可进化传感器的研究

为克服常规传感器对信号采集和处理的局限,探求了一种具有智能传感并自适应于外部环境的机理,将常规传感器和可进化硬件相结合,提出了一种可进化传感器的基本框架,介绍了基于遗传算法和可进化硬件原理的具有自适应能力和容错特性的可进化传感器的初步研究。     

基于动态可重构FPGA的自演化硬件概述

演化硬件研究如何利用遗传算法进行硬件自动设计,或者设计随外界环境变化而自适应地改变自身结构的硬件,在电子设计自动化、自主移动机器人控制器、无线传感器网络节点等领域都有潜在的应用价值. 自演化硬件是在硬件内部完成遗传操作和适应度计算,利用支持动态部分可重构的FPGA芯片上的微处理器核实现遗传算法,模拟生物群体演化过程搜索可能的电路设计并配置片上的可重构逻辑,找到最优或较优的设计结果,从而实现自适应硬件. 当电路发生故障时,自演化硬件自动搜索新的配置,利用片上冗余资源取代故障区域,从而实现自修复硬件. 介绍了基于动态部分可重构FPGA的自演化硬件的基本思想、体系结构以及研究现状,总结并提出了亟待解决的关键技术,指出高效的电路染色体编码表示与可重构逻辑配置位串之间的映射方式是当前研究的重点之一.     

基于免疫原理的逻辑电路设计算法

硬件进化是基于进化计算和可重构硬件的新兴研究领域。逻辑电路的进化设计是硬件进化的主要研究方向之一。文章将生物免疫系统的进化非选择机制引入到逻辑电路设计中,提出了相应的逻辑电路设计算法,并给出了该文算法和进化算法的对比实验结果,结果表明该文算法更加有效。     

Surgical robotics: Reviewing the past,analysing the present,imagining the future

This paper presents an overview of the surgical robotics field, highlighting significant milestones and grouping the various propositions into cohorts. The review does not aim to be exhaustive but rather to highlight how surgical robotics is acting as an enabling technology for minimally invasive surgery. As such, there is a focus on robotic surgical solutions which are commercially available; research efforts which have not gained regulatory approval or entered clinical use are mostly omitted. The practice of robotic surgery is currently largely dominated by the da Vinci system of Intuitive Surgical (Sunnyvale, CA, USA) but other commercial players have now entered the market with surgical robotic products or are appearing in the horizon with medium and long term propositions. Surgical robotics is currently a vibrant research topic and new research directions may lead to the development of very different robotic surgical devices in the future—small, special purpose, lower cost, possibly disposable robots rather than the current large, versatile and capital expensive systems. As the trend towards minimally invasive surgery (MIS) increases, surgery becomes more technically demanding for surgeons and more challenging for medical device technologists and it is clear that surgical robotics has now an established foothold in medicine as an enabling technology of MIS.     

Closed-loop evolutionary multiobjective optimization

Artificial evolution has been used for more than 50 years as a method of optimization in engineering, operations research and computational intelligence. In closed-loop evolution (a term used by the statistician, George Box) or, equivalently, evolutionary experimentation (Ingo Rechenberg?s terminology), the ?phenotypes? are evaluated in the real world by conducting a physical experiment, whilst selection and breeding is simulated. Well-known early work on artificial evolution?design engineering problems in fluid dynamics, and chemical plant process optimization? was carried out in this experimental mode. More recently, the closed-loop approach has been successfully used in much evolvable hardware and evolutionary robotics research, and in some microbiology and biochemistry applications. In this article, several further new targets for closed-loop evolutionary and multiobjective optimization are considered. Four case studies from my own collaborative work are described: (i) instrument optimization in analytical biochemistry; (ii) finding effective drug combinations in vitro; (iii) onchip synthetic biomolecule design; and (iv) improving chocolate production processes. Accurate simulation in these applications is not possible due to complexity or a lack of adequate analytical models. In these and other applications discussed, optimizing experimentally brings with it several challenges: noise; nuisance factors; ephemeral resource constraints; expensive evaluations, and evaluations that must be done in (large) batches. Evolutionary algorithms (EAs) are largely equal to these vagaries, whilst modern multiobjective EAs also enable tradeoffs among conflicting optimization goals to be explored. Nevertheless, principles from other disciplines, such as statistics, Design of Experiments, machine learning and global optimization are also relevant to aspects of the closed-loop problem, and may inspire futher development of multiobjective EAs.     

REVISITING OVERLOOKED FOUNDATIONS OF EVOLUTIONARY COMPUTATION: PART II

Evolutionary computation is a rapidly expanding field of research with a long history. Much of that history remains unknown to most practitioners and researchers. This two-part article offers a review of selected foundational efforts in evolutionary computation, with a focus on those that have not received commensurate attention. Part I presented an initial overview of the essential components of evolutionary algorithms followed by a review of early research in artificial life and modeling genetic systems. Here, Part II reviews seminal results in evolving programs and evolvable hardware. Comments on theoretical developments and future developments conclude Part II.     

Artificial life and embodied robotics: current issues and future challenges

In this article we explore some of the issues currently facing researchers in the interface between the twin fields of Artificial Life and Robotics, and the challenges and potential synergy of these two areas in the creation of future robotic life forms. There are three strands of research which we feel will be of key importance in the possible development of future embodied artificial life forms. These are the areas of evolutionary robotics and evolutionary humanoid robotics in particular, probabilistic robotics for deliberation, and robot benchmarking with associated metrics and standards. We briefly explore each of these areas in turn, focusing on our current research in each field and what we see as the potential issues and challenges for the future. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

REVISITING OVERLOOKED FOUNDATIONS OF EVOLUTIONARY COMPUTATION: PART I

Evolutionary computation is a rapidly expanding field of research with a long history. Much of that history remains unknown to most practitioners and researchers. This two-part article offers a review of selected foundational efforts in evolutionary computation, with focus on those that have not received commensurate attention. Part I presents a brief initial overview of the essential components of evolutionary algorithms followed by a review of early research in artificial life. Part II reviews seminal results in evolving programs and evolvable hardware. Comments on theoretical developments and future developments conclude Part II.     

The evolution of robotics research

This article surveys traditional research topics in industrial robotics and mobile robotics and then expands on new trends in robotics research that focus more on the interaction between human and robot. The new trends in robotics research have been denominated service robotics because of their general goal of getting robots closer to human social needs, and this article surveys research on service robotics such as medical robotics, rehabilitation robotics, underwater robotics, field robotics, construction robotics and humanoid robotics. The aim of this article is to provide an overview of the evolution of research topics in robotics from classical motion control for industrial robots to modern intelligent control techniques and social learning paradigms, among other aspects     

A family of compact genetic algorithms for intrinsic evolvable hardware

For many evolvable hardware applications, small size and power efficiency are critical design considerations. One manner in which significant memory, and thus, power and space savings can be realized in a hardware-based evolutionary algorithm is to represent populations of candidate solutions as probability vectors rather than as sets of bit strings. The compact genetic algorithm (CGA) is a probability vector-based evolutionary algorithm that can be efficiently and elegantly implemented in digital hardware. Unfortunately, the CGA is a very weak, first order, evolutionary algorithm that is unlikely to possess sufficient search power to enable intrinsic evolvable hardware applications. In this paper, we further develop a number of modifications to the basic CGA that significantly improve its search efficacy without substantially increasing the size and complexity of its hardware implementation. The paper provides both benchmark results demonstrating increased efficacy and a conceptual data path/microcontroller design suitable for implementation in digital hardware. Following, it demonstrates efficient implementation by making a head-to-head comparison of field programmable gate array implementations of both the classic CGA and a member of our family of modifications. The paper concludes with a discussion of future research, including several additional extensions that we expect will further increase search efficacy without increasing implementation cost.     

基于函数级FPGA原型的硬件内部进化

电路进化设计是现阶段可进化硬件（EHW）研究的重点内容，针对制约进化设计能力的主要“瓶颈”，该文提出并讨论了一种简洁高效的内部进化方法，包括基于函数变换的染色体高效编码方案，与之配套的函数级FPGA原型和进化实验平台以及在线评估与遗传数自适应方法等，交通灯控制器，4位可级联比较器等相对复杂且具应用价值的电路的成功进化，证明该方法适用于组合，时序电路的进化设计，并可显著地减少运算量，提高进化设计的速度和规模。     

Neuroevolution of Inverted Pendulum Control: A Comparative Study of Simulation Techniques

The inverted pendulum control problem is a classical benchmark in control theory. Amongst the approaches to developing control programs for an inverted pendulum, the evolution of Artificial Neural Network (ANN) based controllers has received some attention. The authors have previously shown that Evolutionary Robotics (ER) can successfully be used to evolve inverted pendulum stabilization controllers in simulation and that these controllers can transfer successfully from simulation to real-world robotic hardware. During this process, use was made of robotic simulators constructed from empirically-collected data and based on ANNs. The current work aims to compare this method of simulator construction with the more traditional method of building robotic simulators based on physics equations governing the robotic system under consideration. In order to compare ANN-based and physics-based simulators in the evolution of inverted pendulum controllers, a real-world wheeled inverted pendulum robot was considered. Simulators based on ANNs as well as on a system of ordinary differential equations describing the dynamics of the robot were developed. These two simulation techniques were then compared by using each in the simulation-based evolution of controllers. During the evolution process, the effects of injecting different levels of noise into the simulation was furthermore studied. Encouraging results were obtained, with controllers evolved using ANN-based simulators and realistic levels of noise outperforming those evolved using the physics-based simulators.