Layered approach to learning client behaviors in the robocup soccer server

In the past few years, multiagent systems (MAS) have emerged as an active subfield of artificial intelligence (AI). Because of the inherent complexity of MAS, there is much interest in using machine learning (ML) techniques to help build multiagent systems. Robotic soccer is a particularly good domain for studying MAS and multiagent learning. Our approach to using ML as a tool for building Soccer Server clients involves layering increasingly complex learned behaviors. In this article, we describe two levels of learned behaviors. First, the clients learn a low-level individual skill that allows them to control the ball effectively. Then, using this learned skill, they learn a higher level skill that involves multiple players. For both skills, we describe the learning method in detail and report on our extensive empirical testing. We also verify empirically that the learned skills are applicable to game situations.     

The RoboCup Soccer Server and CMUnited Clients: Implemented Infrastructure for MAS Research

The RoboCup Soccer Server and associated client code is a growing body of software infrastructure that enables a wide variety of multiagent systems research. The Soccer Server is a multiagent environment that supports 22 independent agents interacting in a complex, real-time environment. AI researchers have been using the Soccer Server to pursue research in a wide variety of areas, including real-time multiagent planning, real-time communication methods, collaborative sensing, and multiagent learning. This article describes the current Soccer Server and the champion CMUnited soccer-playing agents, both of which are publically available and used by a growing research community. It also describes the ongoing development of FUSS, a new, flexible simulation environment for multiagent research in a variety of multiagent domains.     

From roles to teamwork: A framework and architecture

Teamwork requires organization, strategies, and coordination. The design of a multiagent system should support these conceptual properties for constructing effective teams. The advantage of a teamwork approach is the reduction in complexity of the task through distribution of responsibilities, resulting in better utilization of resources, robust behaviors, and a greater variety of behaviors against competitors. In this article a framework for building teams of responsible agents using roles, responsibilities, and strategies is described. Its application to the domain of soccer is used to design a high-performance team of soccer agents. The architecture for these agents utilizes a reactive planning system with support for teamwork. The team of soccer agents will be tested in a series of competitions against other teams in the real-time soccer simulator proposed for Robocup-97, which provides an uncertain, resource bounded world.     

RoboCup: robot world cup

RoboCup is an attempt to foster intelligent robotics research by providing a standard problem where a wide range of technologies can be integrated and examined. The First Robot World Cup Soccer Games and Conferences (RoboCup-97) was held during IJCAI-97, Nagoya, with over 40 teams participating from throughout the world. RoboCup soccer is a task for a team of fast-moving robots in a dynamic, noisy environment. In order for a robot team to actually perform a soccer game, various technologies must be incorporated including: design principles of autonomous agents, multi-agent collaboration, strategy acquisition, real-time reasoning, robotics, and sensor-fusion. This article describes technical challenges involved in RoboCup, its official rules, a report of RoboCup-97, and future perspectives     

Using the cassiopeia method to design a robot soccer team

We have defined the Cassiopeia method, whose specificity is to focus the analysis and design of a multiagent system on the notion of organization. This article reports the use of this methodological framework for designing and implementing the organization of a robot soccer team in the context of a research project on collective robotics. We show why we chose this application, and we discuss its interest and inherent difficulties, in order to clearly express the needs for a design methodology dedicated to distributed artificial intelligence.     

The Open International Soccer Database for machine learning

How well can machine learning predict the outcome of a soccer game, given the most commonly and freely available match data? To help answer this question and to facilitate machine learning research in soccer, we have developed the Open International Soccer Database. Version v1.0 of the Database contains essential information from 216,743 league soccer matches from 52 leagues in 35 countries. The earliest entries in the Database are from the year 2000, which is when football leagues generally adopted the “three points for a win” rule. To demonstrate the use of the Database for machine learning research, we organized the 2017 Soccer Prediction Challenge. One of the goals of the Challenge was to estimate where the limits of predictability lie, given the type of match data contained in the Database. Another goal of the Challenge was to pose a real-world machine learning problem with a fixed time line and a genuine prediction task: to develop a predictive model from the Database and then to predict the outcome of the 206 future soccer matches taking place from 31 March 2017 to the end of the regular season. The Open International Soccer Database is released as an open science project, providing a valuable resource for soccer analysts and a unique benchmark for advanced machine learning methods. Here, we describe the Database and the 2017 Soccer Prediction Challenge and its results.

     

An automatic approach to extract goal plans from soccer simulated matches

Soccer is a competitive and collective sport in which teammates try to combine the execution of basic actions (cooperative behavior) to lead their team to more advantageous situations. The ability to recognize, extract and reproduce such behaviors can prove useful to improve the performance of a team in future matches. This work describes a methodology for achieving just that makes use of a plan definition language to abstract the representation of relevant behaviors in order to promote their reuse. Experiments were conducted based on a set of game log files generated by the Soccer Server simulator which supports the RoboCup 2D simulated robotic soccer league. The effectiveness of the proposed approach was verified by focusing primarily on the analysis of behaviors which started from set-pieces and led to the scoring of goals while the ball possession was kept. One of the results obtained showed that a significant part of the total goals scored was based on this type of behaviors, demonstrating the potential of conducting this analysis. Other results allowed us to assess the complexity of these behaviors and infer meaningful guidelines to consider when defining plans from scratch. Some possible extensions to this work include assessing which plans have the ability to maximize the creation of goal opportunities by countering the opponent’s team strategy and how the effectiveness of plans can be improved using optimization techniques.     

Implementing agent teams in dynamic multiagent environments

Teamwork is becoming increasingly critical in multiagent environments ranging from virtual environments for training and education, to information integration on the internet, to potential multirobotic space missions. Teamwork in such complex, dynamic environments is more than a simple union of simultaneous individual activity, even if supplemented with preplanned coordination. Indeed, in these dynamic environments, unanticipated events can easily cause a breakdown in such preplanned coordination. The central hypothesis in this article is that for effective teamwork, agents should be provided explicit representation of team goals and plans, as well as an explicit representation of a model of teamwork to support the execution of team plans. In our work, this model of teamwork takes the form of a set of domain independent rules that clearly outline an agent's commitments and responsibilities as a participant in team activities, and thus guide the agent's social activities while executing team plans. This article describes two implementations of agent-teams based on the above principles, one for a realworld helicopter combat simulation, and one for the RoboCup soccer simulation. The article also provides a preliminary comparison of the two agent-teams to illustrate some of the strengths and weaknesses of RoboCup as a common test bed for multiagent systems.     

基于人工神经网络的多机器人协作学习研究

机器人足球比赛是一个有趣并且复杂的新兴的人工智能研究领域，它是一个典型的多智能体系统。文中主要研究机器人足球比赛中的协作行为的学习问题，采用人工神经网络算法实现了两个足球机器人的传球学习，实验结果表明了该方法的有效性。最后讨论了对BP算法的诸多改进方法。     

足球视频镜头分类与球场区域检测

目的 足球视频镜头和球场区域是足球视频事件检测的必要条件,对于足球视频语义分析具有重要作用。针对现有镜头分类方法的不足,提出识别足球视频镜头类型的波动检测法。方法 该方法使用一个滑动窗口在视频帧图像中滑动,记录滑动窗口内球场像素比例在远镜头阈值上下的波动次数,根据波动次数判断镜头类型。对于足球场地区域分类,提出使用视频图像中球场区域的左上角和右上角点的位置关系识别球场区域类型的方法,该方法使用高斯混合模型识别出球场,根据球场在帧图像中左右边界坐标的高低判断球场区域类型,方法简单高效。结果 本文提出的两种方法与现有的分类方法相比,在准确率和召回率方面具有较大提高,检测效率高,可以满足实时性要求。结论 本文方法解决了传统滑动窗口法无法正确识别球场倾斜角度过大的帧图像,降低了传统球场区域检测方法依赖球场线检测而导致的准确率不高的问题。     

Zeus: A toolkit for building distributed multiagent systems

The multiagent systems approach of knowledge- level cooperation between autonomous agents promises significant benefits to distributed systems engineering, such as enhanced interoperability, scalability, and reconfigurability. However, thus far, because of the innate difficulty of constructing multiagent systems, this promise has been largely unrealized. Hence there is an emerging desire among agent developers to move away from developing point solutions to point problems in favor of developing methodologies and toolkits for building distributed multiagent systems. This philosophy led to the development of the ZEUS Agent Building Toolkit, which facilitates the rapid development of collaborative agent applications through the provision of a library of agent- level components and an environment to support the agent-building process. The ZEUS toolkit is a synthesis of established agent technologies with some novel solutions to provide an integrated collaborative agent-building environment.     

Incorporating domain knowledge in machine learning for soccer outcome prediction

The task of the 2017 Soccer Prediction Challenge was to use machine learning to predict the outcome of future soccer matches based on a data set describing the match outcomes of 216,743 past soccer matches. One of the goals of the Challenge was to gauge where the limits of predictability lie with this type of commonly available data. Another goal was to pose a real-world machine learning challenge with a fixed time line, involving the prediction of real future events. Here, we present two novel ideas for integrating soccer domain knowledge into the modeling process. Based on these ideas, we developed two new feature engineering methods for match outcome prediction, which we denote as recency feature extraction and rating feature learning. Using these methods, we constructed two learning sets from the Challenge data. The top-ranking model of the 2017 Soccer Prediction Challenge was our k-nearest neighbor model trained on the rating feature learning set. In further experiments, we could slightly improve on this performance with an ensemble of extreme gradient boosted trees (XGBoost). Our study suggests that a key factor in soccer match outcome prediction lies in the successful incorporation of domain knowledge into the machine learning modeling process.

     

Alcod idss: Assisting the Australian stock market surveillance team's review process

ALCOD is a cooperative multiagent intelligent decision support system to assist stock market surveillance teams to classify alerted noncompliant events transacted on the exchange. ALCOD facilitates the review of the classifications. The system combines heuristic approxi mate and causal reasoning and is centered around a relational database which is used as a control blackboard.     

Approximately adaptive neural cooperative control for nonlinear multiagent systems with performance guarantee

This paper studies the cooperative control problem for a class of multiagent dynamical systems with partially unknown nonlinear system dynamics. In particular, the control objective is to solve the state consensus problem for multiagent systems based on the minimisation of certain cost functions for individual agents. Under the assumption that there exist admissible cooperative controls for such class of multiagent systems, the formulated problem is solved through finding the optimal cooperative control using the approximate dynamic programming and reinforcement learning approach. With the aid of neural network parameterisation and online adaptive learning, our method renders a practically implementable approximately adaptive neural cooperative control for multiagent systems. Specifically, based on the Bellman's principle of optimality, the Hamilton–Jacobi–Bellman (HJB) equation for multiagent systems is first derived. We then propose an approximately adaptive policy iteration algorithm for multiagent cooperative control based on neural network approximation of the value functions. The convergence of the proposed algorithm is rigorously proved using the contraction mapping method. The simulation results are included to validate the effectiveness of the proposed algorithm.     

集控式足球机器人系统关键技术研究

足球机器人比赛是机器人研究的一个新热点,它为人工智能理论和算法的研究提供了一个实验平台,其研究的领域涵盖了人工智能、自动控制、机器人视觉、无线通信、机器学习和多智能体合作与协调等。集控式足球机器人系统通常可以划分为4个子系统,即视觉、决策、通信和车型机器人。结合研究经验,介绍了集控式足球机器人各个子系统的关键技术。     

A statistical property of multiagent learning based on Markov decision process

We exhibit an important property called the asymptotic equipartition property (AEP) on empirical sequences in an ergodic multiagent Markov decision process (MDP). Using the AEP which facilitates the analysis of multiagent learning, we give a statistical property of multiagent learning, such as reinforcement learning (RL), near the end of the learning process. We examine the effect of the conditions among the agents on the achievement of a cooperative policy in three different cases: blind, visible, and communicable. Also, we derive a bound on the speed with which the empirical sequence converges to the best sequence in probability, so that the multiagent learning yields the best cooperative result.     

MetrIntPair—A Novel Accurate Metric for the Comparison of Two Cooperative Multiagent Systems Intelligence Based on Paired Intelligence Measurements

In this paper, we propose a novel metric called MetrIntPair (Metric for Pairwise Intelligence Comparison of Agent‐Based Systems) for comparison of two cooperative multiagent systems problem‐solving intelligence. MetrIntPair is able to make an accurate comparison by taking into consideration the variability in intelligence in problem‐solving. The metric could treat the outlier intelligence indicators, intelligence measures that are statistically different from those others. For evaluation of the proposed metric, we realized a case study for two cooperative multiagent systems applied for solving a class of NP‐hard problems. The results of the case study proved that the small difference in the measured intelligence of the multiagent systems is the consequence of the variability. There is no statistical difference between the intelligence quotients/level of the multiagent systems. Both multiagent systems should be classified in the same intelligence class.     

Voting almost maximizes social welfare despite limited communication

In cooperative multiagent systems an alternative that maximizes the social welfare—the sum of utilities—can only be selected if each agent reports its full utility function. This may be infeasible in environments where communication is restricted. Employing a voting rule to choose an alternative greatly reduces the communication burden, but leads to a possible gap between the social welfare of the optimal alternative and the social welfare of the one that is ultimately elected. Procaccia and Rosenschein (2006) [13] have introduced the concept of distortion to quantify this gap.In this paper, we present the notion of embeddings into voting rules: functions that receive an agent?s utility function and return the agent?s vote. We establish that very low distortion can be obtained using randomized embeddings, especially when the number of agents is large compared to the number of alternatives. We investigate our ideas in the context of three prominent voting rules with low communication costs: Plurality, Approval, and Veto. Our results arguably provide a compelling reason for employing voting in cooperative multiagent systems.     

Cooperation without deliberation: A minimal behavior-based approach to multi-robot teams

While terminology and some concepts of behavior-based robotics have become widespread, the central ideas are often lost as researchers try to scale behavior to higher levels of complexity. “Hybrid systems” with model-based strategies that plan in terms of behaviors rather than simple actions have become common for higher-level behavior. We claim that a strict behavior-based approach can scale to higher levels of complexity than many robotics researchers assume, and that the resulting systems are in many cases more efficient and robust than those that rely on “classical AI” deliberative approaches. Our focus is on systems of cooperative autonomous robots in dynamic environments. We will discuss both claims that deliberation and explicit communication are necessary to cooperation and systems that cooperate only through environmental interaction. In this context we introduce three design principles for complex cooperative behavior—minimalism, statelessness and tolerance—and present a RoboCup soccer system that matches the sophistication of many deliberative soccer systems while exceeding their robustness, through the use of strict behavior-based techniques with no explicit communication.     

The robocup physical agent challenge: Phase i

Traditional AI research has not given due attention to the important role that physical bodies play for agents as their interactions produce complex emergent behaviors to achieve goals in the dynamic real world. The RoboCup Physical Agent Challenge provides a good test bed for studying how physical bodies play a significant role in realizing intelligent behaviors using the RoboCup framework (Kitano et al., 1995). In order for the robots to play a soccer game reasonably well, a wide range of technologies needs to be integrated, and a number of technical breakthroughs must be made. In this article, we present three challenging tasks as the RoboCup Physical Agent Challenge Phase I: (1) moving the ball to the specified area (shooting, passing, and dribbling) with no, with stationary, or with moving obstacles; (2) catching the ball from an opponent or a teammate (receiving, goal keeping, and intercepting); and (3) passing the ball between two players. The first two tasks are concerned with single-agent skills, while the third is related to a simple cooperative behavior. Motivation for these challenges and evaluation methodology is given.