Evolving team behaviors with specialization

This article evaluates Collective Neuro-Evolution (CONE), a cooperative co-evolutionary method for solving collective behavior tasks and increasing task performance via facilitating behavioral specialization in agent teams. Specialization is used as a problem solving mechanism, and its emergence is guided and regulated by CONE. CONE is comparatively evaluated with related methods in a simulated evolutionary robotics pursuit-evasion task. This task required multiple pursuer robots to cooperatively capture evader robots. Results indicate that CONE is appropriate for evolving specialized behaviors. The interaction of specialized behaviors produces behavioral heterogeneity in teams and collective prey capture behaviors that yield significantly higher performances compared to related methods.     

Self-generating algorithm of evaluation for cooperative behavior

We propose a self-generating algorithm of behavioral evaluation that is important for a learning function in order to develop appropriate cooperative behavior among robots depending on the situation. The behavioral evaluation is composed of rewards and a consumption of energy. Rewards are provided by an operator when the robots share tasks appropriately, and the consumption of energy is measured during the execution of the tasks. Each robot estimates rules of behavior selection by using the evaluation generated, and learns to select an appropriate behavior when it meets the same situation. As a result, the robots may be able to share tasks efficiently even if the purpose of their task is changed by an operator in the middle of execution, because the evaluation is modified depending on the situation. We performed simulations to study the effectiveness of the proposed algorithm. In the simulations, we applied the algorithm to three robots, each with three behaviors. We confirmed that each robot can generate an appropriate behavioral evaluation based on rewards from an operator, and therefore they develop cooperative behaviors such as task sharing. This work was presented, in part, at the Second International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1997     

Optimized Stochastic Policies for Task Allocation in Swarms of Robots

We present a scalable approach to dynamically allocating a swarm of homogeneous robots to multiple tasks, which are to be performed in parallel, following a desired distribution. We employ a decentralized strategy that requires no communication among robots. It is based on the development of a continuous abstraction of the swarm obtained by modeling population fractions and defining the task allocation problem as the selection of rates of robot ingress and egress to and from each task. These rates are used to determine probabilities that define stochastic control policies for individual robots, which, in turn, produce the desired collective behavior. We address the problem of computing rates to achieve fast redistribution of the swarm subject to constraint(s) on switching between tasks at equilibrium. We present several formulations of this optimization problem that vary in the precedence constraints between tasks and in their dependence on the initial robot distribution. We use each formulation to optimize the rates for a scenario with four tasks and compare the resulting control policies using a simulation in which 250 robots redistribute themselves among four buildings to survey the perimeters.      

带偏向性轮盘赌的多算子协同粒子群优化算法

针对粒子群优化算法在处理高维、大规模、多变量耦合、多模态、多极值属性优化问题时易早熟收敛等性能和技术瓶颈,基于粒子群优化算法行为学习算子和3种不同学习偏好的差分变异算子,建立带偏向性轮盘赌的多算子选择与融合机制,提出一种带偏向性轮盘赌的多算子协同粒子群优化算法MOCPSO.MOCPSO针对迭代粒子群榜样粒子集,首先通过对迭代种群及其榜样粒子集优劣分组,同时采用轮盘赌分别为每组榜样粒子集选配不同学习偏好的变异算子,并为每组榜样粒子适配差分基向量和最优基向量,预学习并优化迭代种群及其榜样粒子,以权衡算法的全局探索和局部开发;然后通过合并所有子种群,并结合粒子群优化算法行为学习算子,指导迭代种群状态更新,以提高算法的全局收敛性;最后结合精英学习策略,对群体历史最优进行高斯扰动,以提高算法的局部逃生能力,保障算法收敛的多样性.实验结果表明,MOCPSO算法与5种先进的同类型群智能算法在求解CEC2014基准测试问题上具备竞争力,且有更强的优化特性.     

From Fireflies to Fault-Tolerant Swarms of Robots

One of the essential benefits of swarm robotic systems is redundancy. In case one robot breaks down, another robot can take steps to repair the failed robot or take over the failed robot's task. Although fault tolerance and robustness to individual failures have often been central arguments in favor of swarm robotic systems, few studies have been dedicated to the subject. In this paper, we take inspiration from the synchronized flashing behavior observed in some species of fireflies. We derive a completely decentralized algorithm to detect non-operational robots in a swarm robotic system. Each robot flashes by lighting up its on-board light-emitting diodes (LEDs), and neighboring robots are driven to flash in synchrony. Since robots that are suffering catastrophic failures do not flash periodically, they can be detected by operational robots. We explore the performance of the proposed algorithm both on a real-world swarm robotic system and in simulation. We show that failed robots are detected correctly and in a timely manner, and we show that a system composed of robots with simulated self-repair capabilities can survive relatively high failure rates.      

Comparative Experimental Studies on Spatial Memory and Learning in Rats and Robots

The study of behavioral and neurophysiological mechanisms involved in rat spatial cognition provides a basis for the development of computational models and robotic experimentation of goal-oriented learning tasks. These models and robotics architectures offer neurobiologists and neuroethologists alternative platforms to study, analyze and predict spatial cognition based behaviors. In this paper we present a comparative analysis of spatial cognition in rats and robots by contrasting similar goal-oriented tasks in a cyclical maze, where studies in rat spatial cognition are used to develop computational system-level models of hippocampus and striatum integrating kinesthetic and visual information to produce a cognitive map of the environment and drive robot experimentation. During training, Hebbian learning and reinforcement learning, in the form of Actor-Critic architecture, enable robots to learn the optimal route leading to a goal from a designated fixed location in the maze. During testing, robots exploit maximum expectations of reward stored within the previously acquired cognitive map to reach the goal from different starting positions. A detailed discussion of comparative experiments in rats and robots is presented contrasting learning latency while characterizing behavioral procedures during navigation such as errors associated with the selection of a non-optimal route, body rotations, normalized length of the traveled path, and hesitations. Additionally, we present results from evaluating neural activity in rats through detection of the immediate early gene Arc to verify the engagement of hippocampus and striatum in information processing while solving the cyclical maze task, such as robots use our corresponding models of those neural structures.     

Interactive Teaching for Vision-Based Mobile Robots: A Sensory-Motor Approach

For the last decade, we have been developing a vision-based architecture for mobile robot navigation. Using our bio-inspired model of navigation, robots can perform sensory-motor tasks in real time in unknown indoor as well as outdoor environments. We address here the problem of autonomous incremental learning of a sensory-motor task, demonstrated by an operator guiding a robot. The proposed system allows for semisupervision of task learning and is able to adapt the environmental partitioning to the complexity of the desired behavior. A real dialogue based on actions emerges from the interactive teaching. The interaction leads the robot to autonomously build a precise sensory-motor dynamics that approximates the behavior of the teacher. The usability of the system is highlighted by experiments on real robots, in both indoor and outdoor environments. Accuracy measures are also proposed in order to evaluate the learned behavior as compared to the expected behavioral attractor. These measures, used first in a real experiment and then in a simulated experiment, demonstrate how a real interaction between the teacher and the robot influences the learning process.     

Self-modeling in humanoid soccer robots

In this paper we discuss the applicability, potential benefits, open problems and expected contributions that an emerging set of self-modeling techniques might bring on the development of humanoid soccer robots. The idea is that robots might continuously generate, validate and adjust physical models of their sensorimotor interaction with the world. These models are exploited for adapting behavior in simulation, enhancing the learning skills of a robot with the regular transference of controllers developed in simulation to reality. Moreover, these simulations can be used to aid the execution of complex sensorimotor tasks, speed up adaptation and enhance task planning. We present experiments on the generation of behaviors for humanoid soccer robots using the Back-to-Reality algorithm. General motivations are presented, alternative algorithms are discussed and, most importantly, directions of research are proposed.     

Cognitive Architecture for Robust Adaptive Control of Robots in a Team

The objective of this paper is to present a cognitive architecture thatutilizes three different methodologies for adaptive, robust control ofrobots behaving intelligently in a team. The robots interact within a worldof objects, and obstacles, performing tasks robustly, while improving theirperformance through learning. The adaptive control of the robots has beenachieved by a novel control system. The Tropism-based cognitive architecturefor the individual behavior of robots in a colony is demonstrated throughexperimental investigation of the robot colony. This architecture is basedon representation of the likes and dislikes of the robots. It is shown thatthe novel architecture is not only robust, but also provides the robots withintelligent adaptive behavior. This objective is achieved by utilization ofthree different techniques of neural networks, machine learning, and geneticalgorithms. Each of these methodologies are applied to the tropismarchitecture, resulting in improvements in the task performance of the robotteam, demonstrating the adaptability and robustness of the proposed controlsystem.     

Swarm-like Methodologies for Executing Tasks with Deadlines

Very few studies have been carried out to test multi-robot task allocation swarm algorithms in real time systems, where each task must be executed before a deadline. This paper presents a comparative study of several swarm-like algorithms and auction based methods for this kind of scenarios. Moreover, a new paradigm called pseudo-probabilistic swarm-like, is proposed, which merges characteristics of deterministic and probabilistic classical swarm approaches. Despite that this new paradigm can not be classified as swarming, it is closely related with swarm methods. Pseudo-probabilistic swarm-like algorithms can reduce the interference between robots and are particularly suitable for real time environments. This work presents two pseudo-probabilistic swarm-like algorithms: distance pseudo-probabilistic and robot pseudo-probabilistic. The experimental results show that the pseudo-probabilistic swarm-like methods significantly improve the number of finished tasks before a deadline, compared to classical swarm algorithms. Furthermore, a very simple but effective learning algorithm has been implemented to fit the parameters of these new methods. To verify the results a foraging task has been used under different configurations.     

Effects of automation and task load on task switching during human supervision of multiple semi-autonomous robots in a dynamic environment

The present study assessed the impact of task load and level of automation (LOA) on task switching in participants supervising a team of four or eight semi-autonomous robots in a simulated ‘capture the flag’ game. Participants were faster to perform the same task than when they chose to switch between different task actions. They also took longer to switch between different tasks when supervising the robots at a high compared to a low LOA. Task load, as manipulated by the number of robots to be supervised, did not influence switch costs. The results suggest that the design of future unmanned vehicle (UV) systems should take into account not simply how many UVs an operator can supervise, but also the impact of LOA and task operations on task switching during supervision of multiple UVs.

The findings of this study are relevant for the ergonomics practice of UV systems. This research extends the cognitive theory of task switching to inform the design of UV systems and results show that switching between UVs is an important factor to consider.     

基于蚁群-遗传融合框架的仓储群机器人任务分配

将智能仓储中的自主移动群机器人订单任务分配,建模成群机器人协同调度的多目标优化问题,将成员机器人完成拣货任务的路径代价和时间代价作为优化目标.设计了蚁群-遗传算法融合框架并在其中求解.该框架中,蚁群算法作为副算法,用于初始种群优化;遗传算法改进后作为主算法.具体地,在遗传算法轮盘赌选择算子后引入精英保留策略,并在遗传操作中加入逆转算子.针对不同数量的订单任务,使用不同规模的群机器人系统进行了任务分配仿真实验.结果表明,在本文所提的融合框架中求解,较分别使用蚁群算法或遗传算法单独求解,性能上具有明显优势,能够发挥蚁群算法鲁棒性好和遗传算法全局搜索能力强的特点,提高智能仓储系统的整体运行效率.     

Encouraging behavioral diversity in evolutionary robotics: an empirical study

Evolutionary robotics (ER) aims at automatically designing robots or controllers of robots without having to describe their inner workings. To reach this goal, ER researchers primarily employ phenotypes that can lead to an infinite number of robot behaviors and fitness functions that only reward the achievement of the task-and not how to achieve it. These choices make ER particularly prone to premature convergence. To tackle this problem, several papers recently proposed to explicitly encourage the diversity of the robot behaviors, rather than the diversity of the genotypes as in classic evolutionary optimization. Such an approach avoids the need to compute distances between structures and the pitfalls of the noninjectivity of the phenotype/behavior relation; however, it also introduces new questions: how to compare behavior? should this comparison be task specific? and what is the best way to encourage diversity in this context? In this paper, we review the main published approaches to behavioral diversity and benchmark them in a common framework. We compare each approach on three different tasks and two different genotypes. The results show that fostering behavioral diversity substantially improves the evolutionary process in the investigated experiments, regardless of genotype or task. Among the benchmarked approaches, multi-objective methods were the most efficient and the generic, Hamming-based, behavioral distance was at least as efficient as task specific behavioral metrics.     

A Study on Acquiring Underlying Behavioral Criteria for Manipulator Motion by Focusing on Learning Efficiency

Conventional humanoid robotic behaviors are directly programmed depending on the programmer's personal experience. With this method, the behaviors usually appear unnatural. It is believed that a humanoid robot can acquire new adaptive behaviors from a human, if the robot has the criteria underlying such behaviors. The aim of this paper is to establish a method of acquiring human behavioral criteria. The advantage of acquiring behavioral criteria is that the humanoid robots can then autonomously produce behaviors for similar tasks with the same behavioral criteria but without transforming data obtained from morphologically different humans every time for every task. In this paper, a manipulator robot learns a model behavior, and another robot is created to perform the model behavior instead of being performed by a person. The model robot is presented some behavioral criteria, but the learning manipulator robot does not know them and tries to infer them. In addition, because of the difference between human and robot bodies, the body sizes of the learning robot and the model robot are also made different. The method of obtaining behavioral criteria is realized by comparing the efficiencies with which the learning robot learns the model behaviors. Results from the simulation have demonstrated that the proposed method is effective for obtaining behavioral criteria. The proposed method, the details regarding the simulation, and the results are presented in this paper.     

Digital twin-driven deep reinforcement learning for adaptive task allocation in robotic construction

In order to accomplish diverse tasks successfully in a dynamic (i.e., changing over time) construction environment, robots should be able to prioritize assigned tasks to optimize their performance in a given state. Recently, a deep reinforcement learning (DRL) approach has shown potential for addressing such adaptive task allocation. It remains unanswered, however, whether or not DRL can address adaptive task allocation problems in dynamic robotic construction environments. In this paper, we developed and tested a digital twin-driven DRL learning method to explore the potential of DRL for adaptive task allocation in robotic construction environments. Specifically, the digital twin synthesizes sensory data from physical assets and is used to simulate a variety of dynamic robotic construction site conditions within which a DRL agent can interact. As a result, the agent can learn an adaptive task allocation strategy that increases project performance. We tested this method with a case project in which a virtual robotic construction project (i.e., interlocking concrete bricks are delivered and assembled by robots) was digitally twinned for DRL training and testing. Results indicated that the DRL model’s task allocation approach reduced construction time by 36% in three dynamic testing environments when compared to a rule-based imperative model. The proposed DRL learning method promises to be an effective tool for adaptive task allocation in dynamic robotic construction environments. Such an adaptive task allocation method can help construction robots cope with uncertainties and can ultimately improve construction project performance by efficiently prioritizing assigned tasks.     

A PATH PLANNING METHOD BASED ON CELLULAR AUTOMATA FOR COOPERATIVE ROBOTS

A Cellular Automaton-based technique suitable for solving the path planning problem in a distributed robot team is outlined. Real-time path planning is a challenging task that has many applications in the fields of artificial intelligence, moving robots, virtual reality, and agent behavior simulation. The problem refers to finding a collision-free path for autonomous robots between two specified positions in a configuration area. The complexity of the problem increases in systems of multiple robots. More specifically, some distance should be covered by each robot in an unknown environment, avoiding obstacles found on its route to the destination. On the other hand, all robots must adjust their actions in order to keep their initial team formation immutable. Two different formations were tested in order to study the efficiency and the flexibility of the proposed method. Using different formations, the proposed technique could find applications to image processing tasks, swarm intelligence, etc. Furthermore, the presented Cellular Automaton (CA) method was implemented and tested in a real system using three autonomous mobile minirobots called E-pucks. Experimental results indicate that accurate collision-free paths could be created with low computational cost. Additionally, cooperation tasks could be achieved using minimal hardware resources, even in systems with low-cost robots.     

A Framework for Augmented Reality using Non-Central Catadioptric Cameras

This paper proposes a strategy for a group of swarm robots to self-assemble into a single articulated(legged) structure in response to terrain difficulties during autonomous exploration. These articulated structures will have several articulated legs or backbones, so they are well suited to walk on difficult terrains like animals. There are three tasks in this strategy: exploration, self-assembly and locomotion. We propose a formation self-assembly method to improve self-assembly efficiency. At the beginning, a swarm of robots explore the environment using their sensors and decide whether to self-assemble and select a target configuration from a library to form some robotic structures to finish a task. Then, the swarm of robots will execute a self-assembling task to construct the corresponding configuration of an articulated robot. For the locomotion, with joint actuation from the connected robots, the articulated robot generates locomotive motions. Based on Sambot that are designed to unite swarm mobile and self-reconfigurable robots, we demonstrate the feasibility for a varying number of swarm robots to self-assemble into snake-like and multi-legged robotic structures. Then, the effectiveness and scalability of the strategy are discussed with two groups of experiments and it proves the formation self-assembly is more efficient in the end.     

Ad hoc teamwork by learning teammates’ task

This paper addresses the problem of ad hoc teamwork, where a learning agent engages in a cooperative task with other (unknown) agents. The agent must effectively coordinate with the other agents towards completion of the intended task, not relying on any pre-defined coordination strategy. We contribute a new perspective on the ad hoc teamwork problem and propose that, in general, the learning agent should not only identify (and coordinate with) the teammates’ strategy but also identify the task to be completed. In our approach to the ad hoc teamwork problem, we represent tasks as fully cooperative matrix games. Relying exclusively on observations of the behavior of the teammates, the learning agent must identify the task at hand (namely, the corresponding payoff function) from a set of possible tasks and adapt to the teammates’ behavior. Teammates are assumed to follow a bounded-rationality best-response model and thus also adapt their behavior to that of the learning agent. We formalize the ad hoc teamwork problem as a sequential decision problem and propose two novel approaches to address it. In particular, we propose (i) the use of an online learning approach that considers the different tasks depending on their ability to predict the behavior of the teammate; and (ii) a decision-theoretic approach that models the ad hoc teamwork problem as a partially observable Markov decision problem. We provide theoretical bounds of the performance of both approaches and evaluate their performance in several domains of different complexity.     

Factors influencing analysis of complex cognitive tasks: a framework and example from industrial process control

We propose that considering four categories of task factors can facilitate knowledge elicitation efforts in the analysis of complex cognitive tasks: materials, strategies, knowledge characteristics, and goals. A study was conducted to examine the effects of altering aspects of two of these task categories on problem-solving behavior across skill levels: materials and goals. Two versions of an applied engineering problem were presented to expert, intermediate, and novice participants. Participants were to minimize the cost of running a steam generation facility by adjusting steam generation levels and flows. One version was cast in the form of a dynamic, computer-based simulation that provided immediate feedback on flows, costs, and constraint violations, thus incorporating key variable dynamics of the problem context. The other version was cast as a static computer-based model, with no dynamic components, cost feedback, or constraint checking. Experts performed better than the other groups across material conditions, and, when required, the presentation of the goal assisted the experts more than the other groups. The static group generated richer protocols than the dynamic group, but the dynamic group solved the problem in significantly less time. Little effect of feedback was found for intermediates, and none for novices. We conclude that demonstrating differences in performance in this task requires different materials than explicating underlying knowledge that leads to performance. We also conclude that substantial knowledge is required to exploit the information yielded by the dynamic form of the task or the explicit solution goal. This simple model can help to identify the contextual factors that influence elicitation and specification of knowledge, which is essential in the engineering of joint cognitive systems.     

A reinforcement learning approach to fail-safe design for multiple space robots—cooperation mechanism without communication and negotiation schemes

This paper explores a fail-safe design for multiple space robots, which enables robots to complete given tasks even when they can no longer be controlled due to a communication accident or negotiation problem. As the first step towards this goal, we propose new reinforcement learning methods that help robots avoid deadlock situations in addition to improving the degree of task completion without communications via ground stations or negotiations with other robots. Through intensive simulations on a truss construction task, we found that our reinforcement learning methods have great potential to contribute towards fail-safe design for multiple space robots in the above case. Furthermore, the simulations revealed the following detailed implications: (i) the first several planned behaviors must not be reinforced with negative rewards even in deadlock situations in order to derive cooperation among multiple robots, (ii) a certain amount of positive rewards added into negative rewards in deadlock situations contributes to reducing the computational cost of finding behavior plans for task completion, and (iii) an appropriate balance between positive and negative rewards in deadlock situations is indispensable for finding good behavior plans at a small computational cost.