Experiences Acquired in the Design of RoboCup Teams: A Comparison of Two Fielded Teams

Increasingly, multi-agent systems are being designed for a variety of complex, dynamic domains. Effective agent interactions in such domains raise some of the most fundamental research challenges for agent-based systems, in teamwork, multi-agent learning and agent modelling. The RoboCup research initiative, particularly the simulation league, has been proposed to pursue such multi-agent research challenges, using the common testbed of simulation soccer. Despite the significant popularity of RoboCup within the research community, general lessons have not often been extracted from participation in RoboCup. This is what we attempt to do here. We have fielded two teams, ISIS97 and ISIS98, in RoboCup competitions. These teams have been in the top four teams in these competitions. We compare the teams, and attempt to analyze and generalize the lessons learned. This analysis reveals several surprises, pointing out lessons for teamwork and for multi-agent learning.     

Towards efficient multiagent task allocation in the RoboCup Rescue: a biologically-inspired approach

This paper addresses team formation in the RoboCup Rescue centered on task allocation. We follow a previous approach that is based on so-called extreme teams, which have four key characteristics: agents act in domains that are dynamic; agents may perform multiple tasks; agents have overlapping functionality regarding the execution of each task but differing levels of capability; and some tasks may depict constraints such as simultaneous execution. So far these four characteristics have not been fully tested in domains such as the RoboCup Rescue. We use a swarm intelligence based approach, address all characteristics, and compare it to other two GAP-based algorithms. Experiments where computational effort, communication load, and the score obtained in the RoboCup Rescue aremeasured, show that our approach outperforms the others.     

Genetic Team Composition and Level of Selection in the Evolution of Cooperation

In cooperative multiagent systems, agents interact to solve tasks. Global dynamics of multiagent teams result from local agent interactions, and are complex and difficult to predict. Evolutionary computation has proven a promising approach to the design of such teams. The majority of current studies use teams composed of agents with identical control rules (“genetically homogeneous teams”) and select behavior at the team level (“team-level selection”). Here we extend current approaches to include four combinations of genetic team composition and level of selection. We compare the performance of genetically homogeneous teams evolved with individual-level selection, genetically homogeneous teams evolved with team-level selection, genetically heterogeneous teams evolved with individual-level selection, and genetically heterogeneous teams evolved with team-level selection. We use a simulated foraging task to show that the optimal combination depends on the amount of cooperation required by the task. Accordingly, we distinguish between three types of cooperative tasks and suggest guidelines for the optimal choice of genetic team composition and level of selection.      

国际机器人足球比赛及其相关技术

本文在简要介绍两个相关的国际组织RoboCup联合会和FIRA的基础上,重点论述了RoboCup的 比赛环境;同时详细阐述了目前各国参加RoboCup比赛球队的系统结构及其相关技术.对提高 我国相关领域的研究水平,迅速组织我们自己的机器人足球队参加国际比赛并取得好名次具 有指导意义．     

Building agent teams using an explicit teamwork model and learning

Multi-agent collaboration or teamwork and learning are two critical research challenges in a large number of multi-agent applications. These research challenges are highlighted in RoboCup, an international project focused on robotic and synthetic soccer as a common testbed for research in multi-agent systems. This article describes our approach to address these challenges, based on a team of soccer-playing agents built for the simulation league of RoboCup—the most popular of the RoboCup leagues so far.To address the challenge of teamwork, we investigate a novel approach based on the (re)use of a domain-independent, explicit model of teamwork, an explicitly represented hierarchy of team plans and goals, and a team organization hierarchy based on roles and role-relationships. This general approach to teamwork, shown to be applicable in other domains beyond RoboCup, both reduces development time and improves teamwork flexibility. We also demonstrate the application of off-line and on-line learning to improve and specialize agents' individual skills in RoboCup. These capabilities enabled our soccer-playing team, ISIS, to successfully participate in the first international RoboCup soccer tournament (RoboCup'97) held in Nagoya, Japan, in August 1997. ISIS won the third-place prize in over 30 teams that participated in the simulation league.     

Exploiting Spatial Locality and Heterogeneity of Agents for Search and Rescue Teamwork*

We propose coordination mechanisms for multiple heterogeneous physical agents that operate in city‐scale disaster scenarios, where they need to find and rescue people and extinguish fires. Large‐scale disasters are characterized by limited and unreliable communications; dangerous events that may disable agents; uncertainty about the location, duration, and type of tasks; and stringent temporal constraints on task completion times. In our approach, agents form teams with other agents that are in the same geographical area. Our algorithms either yield stable teams formed up front and never change, fluid teams where agents can change teams as need arises, or teams that restrict the types of agents that can belong to the same team. We compare our teaming algorithms against a baseline algorithm in which agents operate independently of others and two state‐of‐the‐art coordination mechanisms. Our algorithms are tested in city‐scale disaster simulations using the RoboCup Rescue simulator. Our experiments with different city maps show that, in general, forming teams leads to increased task completion and, specifically, that our teaming method that restricts the types of agents in a team outperforms the other methods.     

RoboCup: robot world cup

RoboCup is an attempt to foster intelligent robotics research by providing a standard problem where a wide range of technologies can be integrated and examined. The First Robot World Cup Soccer Games and Conferences (RoboCup-97) was held during IJCAI-97, Nagoya, with over 40 teams participating from throughout the world. RoboCup soccer is a task for a team of fast-moving robots in a dynamic, noisy environment. In order for a robot team to actually perform a soccer game, various technologies must be incorporated including: design principles of autonomous agents, multi-agent collaboration, strategy acquisition, real-time reasoning, robotics, and sensor-fusion. This article describes technical challenges involved in RoboCup, its official rules, a report of RoboCup-97, and future perspectives     

机器学习在RoboCup中的应用研究

RoboCup is a particularly good domain for studying multi-agent systems.A wide variety of MAS issues can be studied in robotic soccer,in which the theory,algorithm and architecture of agent system can be evaluated.Because of the inherent complexity of MAS,there are many interests in using machine learning techniques to handle it.This paper investigates and discusses the machine-learning techniques used in RoboCup.The background is firstly presented and the application of machine learning in RoboCup is lately demonstrated with some top simulation teams.The machine-learning system in NDSocTeam is also introduced.Finally some open issues in this field are pointed out.     

RoboCup Rescue as multiagent task allocation among teams: experiments with task interdependencies

This paper addresses distributed task allocation among teams of agents in a RoboCup Rescue scenario. We are primarily concerned with testing different mechanisms that formalize issues underlying implicit coordination among teams of agents. These mechanisms are developed, implemented, and evaluated using two algorithms: Swarm-GAP and LA-DCOP. The latter bases task allocation on a comparison between an agent’s capability to perform a task and the capability demanded by this task. Swarm-GAP is a probabilistic approach in which an agent selects a task using a model inspired by task allocation among social insects. Both algorithms were also compared to another one that allocates tasks in a greedy way. Departing from previous works that tackle task allocation in the rescue scenario only among fire brigades, here we consider the various actors in the RoboCup Rescue, a step forward in the direction of realizing the concept of extreme teams. Tasks are allocated to teams of agents without explicit negotiation and using only local information. Our results show that the performance of Swarm-GAP and LA-DCOP are similar and that they outperform a greedy strategy. Also, it is possible to see that using more sophisticated mechanisms for task selection does pay off in terms of score.     

Using agent- and organisation-oriented programming to develop a team of agents for a competitive game

The multi-agent programming contest uses a cow-herding scenario where two teams of cooperative agents compete for resources against each other. We developed such a team of agents using two well-known platforms, one based on a logic-based agent-oriented programming language, called Jason, and the other based on an organisational model, called $\mathcal{M}$ oise. While there is significant research on both agent programming and agent organisations, this was one of the first applications of a combined approach where we can program deliberative agents and organise them using a sophisticated organisational model. In this paper, we describe and discuss our contribution to the multi-agent contest using this combination of agent and organisation programming.     

Quantifying the impact of communication on performance in multi-agent teams

In this work, we relate the extent and quality of inter-agent communication and the overall performance in teams of multiple agents. Specifically, we examine the RoboCup Soccer Simulation 2D League, and carry out multiple simulation experiments against two evenly matched teams. For each simulated run (a 2D soccer simulation game), we generate the communication efficiencies (i.e., communications sent/communications received) for each agent pair. Applying linear regression and principal component analyses, we then correlate these efficiencies with measures of performance (i.e., goals scored and goals conceded), enabling the construction of inter-agent communication networks. Analysis of these networks highlights the microscopic player-to-player and macroscopic role-to-role communications correlated with performance. The approach determines the salient pathways within inter-agent communications which globally affect the coordination and the overall performance in multi-agent teams.     

On the use of human instruction for improving the behavior of RoboCup soccer agents

In this article, we propose a behavior generation approach with human instruction to improve the strategy of the RoboCup soccer 3D simulation team. Many teams implement their strategies based on the programmer’s own knowledge about soccer. That is, the programmers have to write action rules that cover any situation on the soccer field. Although it is clear that this is not the best approach, there are only a few researchers who have tackled this problem. Here, we solve the problem using human instruction to improve the manually implemented behavior of soccer robots. It is shown that the team performance is improved by the rules generated by this approach.     

RoboCup 3D Server分析

RoboCup仿真比赛是MAS系统的理想测试平台,而3D仿真是RoboCup研究的热点。由于目前缺乏相关的资料,给参赛队增加了不少困难。本文将分析RoboCup 3D Sewer的一些关键技术,以期能让读者对这一新技术有一个全面的了解,缩短球队的开发过程。     

Multi-goal Q-learning of cooperative teams

This paper studies a multi-goal Q-learning algorithm of cooperative teams. Member of the cooperative teams is simulated by an agent. In the virtual cooperative team, agents adapt its knowledge according to cooperative principles. The multi-goal Q-learning algorithm is approached to the multiple learning goals. In the virtual team, agents learn what knowledge to adopt and how much to learn (choosing learning radius). The learning radius is interpreted in Section 3.1. Five basic experiments are manipulated proving the validity of the multi-goal Q-learning algorithm. It is found that the learning algorithm causes agents to converge to optimal actions, based on agents’ continually updated cognitive maps of how actions influence learning goals. It is also proved that the learning algorithm is beneficial to the multiple goals. Furthermore, the paper analyzes how sensitive the learning performance is affected by the parameter values of the learning algorithm.     

VisCoMET: Visually Analyzing Team Collaboration in Medical Emergency Trainings

Handling emergencies requires efficient and effective collaboration of medical professionals. To analyze their performance, in an application study, we have developed VisCoMET, a visual analytics approach displaying interactions of healthcare personnel in a triage training of a mass casualty incident. The application scenario stems from social interaction research, where the collaboration of teams is studied from different perspectives. We integrate recorded annotations from multiple sources, such as recorded videos of the sessions, transcribed communication, and eye-tracking information. For each session, an information-rich timeline visualizes events across these different channels, specifically highlighting interactions between the team members. We provide algorithmic support to identify frequent event patterns and to search for user-defined event sequences. Comparing different teams, an overview visualization aggregates each training session in a visual glyph as a node, connected to similar sessions through edges. An application example shows the usage of the approach in the comparative analysis of triage training sessions, where multiple teams encountered the same scene, and highlights discovered insights. The approach was evaluated through feedback from visualization and social interaction experts. The results show that the approach supports reflecting on teams' performance by exploratory analysis of collaboration behavior while particularly enabling the comparison of triage training sessions.     

Human-agent teamwork in dynamic environments

Teamwork between humans and computer agents has become increasingly prevalent. This paper presents a behavioral study of fairness and trust in a heterogeneous setting comprising both computer agents and human participants. It investigates people’s choice of teammates and their commitment to their teams in a dynamic environment in which actions occur at a fast pace and decisions are made within tightly constrained time frames, under conditions of uncertainty and partial information. In this setting, participants could form teams by negotiating over the division of a reward for the successful completion of a group task. Participants could also choose to defect from their existing teams in order to join or create other teams. Results show that when people form teams, they offer significantly less reward to agents than they offer to people. The most significant factor affecting people’s decisions whether to defect from their existing teams is the extent to which they had successful previous interactions with other team members. Also, there is no significant difference in people’s rate of defection from agent-led teams as compared to their defection from human-led teams. These results are significant for agent designers and behavioral researchers who study human-agent interactions.     

Using team cognitive work analysis to reveal healthcare team interactions in a birthing unit

Cognitive work analysis (CWA) as an analytical approach for examining complex sociotechnical systems has shown success in modelling the work of single operators. The CWA approach incorporates social and team interactions, but a more explicit analysis of team aspects can reveal more information for systems design. In this paper, Team CWA is explored to understand teamwork within a birthing unit at a hospital. Team CWA models are derived from theories and models of teamwork and leverage the existing CWA approaches to analyse team interactions. Team CWA is explained and contrasted with prior approaches to CWA. Team CWA does not replace CWA, but supplements traditional CWA to more easily reveal team information. As a result, Team CWA may be a useful approach to enhance CWA in complex environments where effective teamwork is required.

Practitioner Summary: This paper looks at ways of analysing cognitive work in healthcare teams. Team Cognitive Work Analysis, when used to supplement traditional Cognitive Work Analysis, revealed more team information than traditional Cognitive Work Analysis. Team Cognitive Work Analysis should be considered when studying teams.     

Cooperative behavior acquisition for mobile robots in dynamically changing real worlds via vision-based reinforcement learning and development

In this paper, we first discuss the meaning of physical embodiment and the complexity of the environment in the context of multi-agent learning. We then propose a vision-based reinforcement learning method that acquires cooperative behaviors in a dynamic environment. We use the robot soccer game initiated by RoboCup (Kitano et al., 1997) to illustrate the effectiveness of our method. Each agent works with other team members to achieve a common goal against opponents. Our method estimates the relationships between a learner's behaviors and those of other agents in the environment through interactions (observations and actions) using a technique from system identification. In order to identify the model of each agent, Akaike's Information Criterion is applied to the results of Canonical Variate Analysis to clarify the relationship between the observed data in terms of actions and future observations. Next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior policy. The proposed method is applied to a soccer playing situation. The method successfully models a rolling ball and other moving agents and acquires the learner's behaviors. Computer simulations and real experiments are shown and a discussion is given.     

Sharing in teams of heterogeneous,collaborative learning agents

This paper is focused on the effects of sharing knowledge and collaboration of multiple heterogeneous, intelligent agents (hardware or software) which work together to learn a task. As each agent employs a different machine learning technique, the system consists of multiple knowledge sources and their respective heterogeneous knowledge representations. Collaboration between agents involves sharing knowledge to both speed up team learning, as well as refine the team's overall performance and group behavior. Experiments have been performed that vary the team composition in terms of machine learning algorithms, learning strategies employed by the agents, and sharing frequency for a predator‐prey cooperative pursuit task. For lifelong learning, heterogeneous learning teams were more successful than homogeneous learning counterparts. Interestingly, sharing increased the learning rate, but sharing with higher frequency showed diminishing results. Lastly, knowledge conflicts are reduced over time the more sharing takes place. These results support further investigation of the merits of heterogeneous learning. © 2008 Wiley Periodicals, Inc.     

The Impact of Project Planning Team Experience on Software Project Cost Estimates

Data from 135 teams that have participated in a software project planning exercise are analyzed to determine the relationship between team experience and each teams estimate of total project cost. The analysis shows that cost estimates are dependent upon two kinds of team experience: (1) the average experience for the members of each team and (2) whether or not any members of the team have similar project experience. It is shown that if no members of a planning team have had similar project experience then the estimate of cost is correlated with average team experience, with teams having greater average team experience producing higher total cost estimates. If at least one member of the planning team has had similar project experience then there is a weaker relationship between average team experience and cost, and cost estimates produced by those teams with similar project experience are close to those produced by teams with the greatest average team experience. A qualitative examination of the project plans produced by all teams indicates that the primary reasons that teams with less experience of either type produce lower cost estimates are that they have failed to include some tasks that are included by more experienced teams, and that they have estimated shorter task durations than have the more experienced teams.