Multiagent Systems: A Survey from a Machine Learning Perspective

Distributed Artificial Intelligence (DAI) has existed as a subfield of AI for less than two decades. DAI is concerned with systems that consist of multiple independent entities that interact in a domain. Traditionally, DAI has been divided into two sub-disciplines: Distributed Problem Solving (DPS) focuses on the information management aspects of systems with several components working together towards a common goal; Multiagent Systems (MAS) deals with behavior management in collections of several independent entities, or agents. This survey of MAS is intended to serve as an introduction to the field and as an organizational framework. A series of general multiagent scenarios are presented. For each scenario, the issues that arise are described along with a sampling of the techniques that exist to deal with them. The presented techniques are not exhaustive, but they highlight how multiagent systems can be and have been used to build complex systems. When options exist, the techniques presented are biased towards machine learning approaches. Additional opportunities for applying machine learning to MAS are highlighted and robotic soccer is presented as an appropriate test bed for MAS. This survey does not focus exclusively on robotic systems. However, we believe that much of the prior research in non-robotic MAS is relevant to robotic MAS, and we explicitly discuss several robotic MAS, including all of those presented in this issue.     

A Virtual World Grammar for automatic generation of virtual worlds

Hybrid systems such as those that combine 3D virtual worlds and organization based multiagent systems add new visual and communication features for multiuser applications. The design of such hybrid and dynamic systems is a challenging task. In this paper, we propose a system that can automatically generate a 3D virtual world (VW) from an organization based multiagent system (MAS) specification that establishes the activities participants can engage on. Both shape grammar and virtual world paradigms inspired us to propose a Virtual World Grammar (VWG) to support the generation process of a virtual world design. A VWG includes semantic information about both MAS specification and shape grammar elements. This information, along with heuristics and validations, guides the VW generation producing functional designs. To support the definition and execution of a Virtual World Grammar, we have developed a so named Virtual World Builder Toolkit (VWBT). We illustrate this process by generating a 3D visualization of a virtual institution from its specification.     

A modern approach to multiagent development

Multiagent systems (MAS) development frameworks aim at facilitating the development and administration of agent-based applications. Currently relevant tools, such as JADE, offer huge possibilities but they are generally linked to a specific technology (commonly Java). This fact may limit some application domains when deploying MAS, such as low efficiency or programming language restrictions. To contribute to the evolution of multiagent development tools and to overcome these constraints, we introduce a multiagent platform based on the FIPA standards and built on top of a modern object-oriented middleware. Experimental results prove the scalability and the short response-time of the proposal and justify the design and development of modern tools to contribute the multiagent technology.     

面向自动文摘的多Agent系统中的协调算法研究

为了解决自动文摘系统所面临的领域通用性和文摘质量的矛盾,提出了建造面向自动文摘的多Aent系统（MAS／ABS)的方案。建造这样的系统,有两个关键问题：建立什么样的系统模型和选择什么样的协调算法。给出了在Internet环境下面向自动文摘的多Agent系统模型,提出了3种协调算法。在仿真的基础上分析了系统性能,对3种协调算法进行了比较研究,并得到了在一定负载下面向各个领域合适的文摘Agent数目。     

Agent Mining: The Synergy of Agents and Data Mining

Autonomous agents and multiagent systems (or agents) and data mining and knowledge discovery (or data mining) are two of the most active areas in information technology. Ongoing research has revealed a number of intrinsic challenges and problems facing each area, which can't be addressed solely within the confines of the respective discipline. A profound insight of bringing these two communities together has unveiled a tremendous potential for new opportunities and wider applications through the synergy of agents and data mining. With increasing interest in this synergy, agent mining is emerging as a new research field studying the interaction and integration of agents and data mining. In this paper, we give an overall perspective of the driving forces, theoretical underpinnings, main research issues, and application domains of this field, while addressing the state-of-the-art of agent mining research and development. Our review is divided into three key research topics: agent-driven data mining, data mining-driven agents, and joint issues in the synergy of agents and data mining. This new and promising field exhibits a great potential for groundbreaking work from foundational, technological and practical perspectives.     

Colonies of learning automata

Originally, learning automata (LAs) were introduced to describe human behavior from both a biological and psychological point of view. In this paper, we show that a set of interconnected LAs is also able to describe the behavior of an ant colony, capable of finding the shortest path from their nest to food sources and back. The field of ant colony optimization (ACO) models ant colony behavior using artificial ant algorithms. These algorithms find applications in a whole range of optimization problems and have been experimentally proved to work very well. It turns out that a known model of interconnected LA, used to control Markovian decision problems (MDPs) in a decentralized fashion, matches perfectly with these ant algorithms. The field of LAs can thus both impart in the understanding of why ant algorithms work so well and may also become an important theoretical tool for learning in multiagent systems (MAS) in general. To illustrate this, we give an example of how LAs can be used directly in common Markov game problems.     

Distributed nonlinear control algorithms for network consensus

In this paper, we develop a thermodynamic framework for addressing consensus problems for nonlinear multiagent dynamical systems with fixed and switching topologies. Specifically, we present distributed nonlinear static and dynamic controller architectures for multiagent coordination. The proposed controller architectures are predicated on system thermodynamic notions resulting in controller architectures involving the exchange of information between agents that guarantee that the closed-loop dynamical network is consistent with basic thermodynamic principles.     

Coordinating multiagent applications on the WWW: a referencearchitecture

The original Web did not support multiuser, interactive applications. This shortcoming is being studied, and several approaches have been proposed to use the Web as a platform for programming Internet applications. However, most existing approaches are oriented to centralized applications at servers, or local programs within clients. To overcome this deficit, we introduce PageSpace, that is a reference architecture for designing interactive multiagent applications. We describe how we control agents in PageSpace, using variants of the coordination language Linda to guide their interactions. Coordination technology is integrated with the standard Web technology and the programming language Java. Several kinds of agents live in the PageSpace: user interface agents, personal homeagents, agents that implement applications, and agents which interoperate with legacy systems. Within our architecture, it is possible to support fault-tolerance and mobile agents as well     

Design and implementation of a multiagent stock trading system

Stock trading is one of the key items in an economy and estimating its behavior and taking the best decision in it are among the most challenging issues. Solutions based on intelligent agent systems are proposed to cope with those challenges. Agents in a multiagent system (MAS) can share a common goal or they can pursue their own interests. That nature of MASs exactly fits the requirements of a free market economy. Although existing studies include noteworthy proposals on agent‐based market simulation and researchers discuss theoretical design issues of agent‐based stock exchange systems, unfortunately only a very few of the studies consider exact development and implementation of multiagent stock trading systems within the software engineering perspective and guides to the software engineers for constructing such software systems starting from scratch. To fill this gap, in this paper, we discuss the development of a multiagent‐based stock trading system by taking into consideration software design according to a well‐defined agent oriented software engineering methodology and implementation with a widely‐used MAS software development framework. Each participant in the system is first designed as belief–desire–intention agents with their facts, goals, and plans, and then belief–desire–intention reasoning and behavioral structure of the designed agents are implemented. Lessons learned during design and development within the software engineering perspective and evaluation of the implemented multiagent stock exchange system are also reported. Copyright © 2011 John Wiley & Sons, Ltd.     

Environment as a first class abstraction in multiagent systems

The current practice in multiagent systems typically associates the environment with resources that are external to agents and their communication infrastructure. Advanced uses of the environment include infrastructures for indirect coordination, such as digital pheromones, or support for governed interaction in electronic institutions. Yet, in general, the notion of environment is not well defined. Functionalities of the environment are often dealt with implicitly or in an ad hoc manner. This is not only poor engineering practice, it also hinders engineers to exploit the full potential of the environment in multiagent systems. In this paper, we put forward the environment as an explicit part of multiagent systems.We give a definition stating that the environment in a multiagent system is a first-class abstraction with dual roles: (1) the environment provides the surrounding conditions for agents to exist, which implies that the environment is an essential part of every multiagent system, and (2) the environment provides an exploitable design abstraction for building multiagent system applications. We discuss the responsibilities of such an environment in multiagent systems and we present a reference model for the environment that can serve as a basis for environment engineering. To illustrate the power of the environment as a design abstraction, we show how the environment is successfully exploited in a real world application. Considering the environment as a first-class abstraction in multiagent systems opens up new horizons for research and development in multiagent systems.     

多Agent系统中的角色继承

继承用来描述一般与特殊关系,多Agent系统中引入继承有助于分析多Agent系统的结构层次和软件重用。本文讨论了开发多Agent系统时分析和描述继承的重要性和多Agent系统中继承的特点,通过一个角色模型对继承机制进行了语义定义,提出了一种在需求分析阶段对继承进行建模和分析的方法。     

Layered approach to learning client behaviors in the robocup soccer server

In the past few years, multiagent systems (MAS) have emerged as an active subfield of artificial intelligence (AI). Because of the inherent complexity of MAS, there is much interest in using machine learning (ML) techniques to help build multiagent systems. Robotic soccer is a particularly good domain for studying MAS and multiagent learning. Our approach to using ML as a tool for building Soccer Server clients involves layering increasingly complex learned behaviors. In this article, we describe two levels of learned behaviors. First, the clients learn a low-level individual skill that allows them to control the ball effectively. Then, using this learned skill, they learn a higher level skill that involves multiple players. For both skills, we describe the learning method in detail and report on our extensive empirical testing. We also verify empirically that the learned skills are applicable to game situations.     

Using bayesian networks to model agent relationships

An agent-society of the future is envisioned to be as complex as a human society. Just like human societies, such multiagent systems (MAS) deserve an in-depth study of the dynamics, relationships, and interactions of the constituent agents. An agent in a MAS may have only approximate a priori estimates of the trustworthiness of another agent. But it can learn from interactions with other agents, resulting in more accurate models of these agents and their dependencies together with the influences of other environmental factors. Such models are proposed to be represented as Bayesian or belief networks. An objective mechanism is presented to enable an agent elicit crucial information from the environment regarding the true nature of the other agents. This mechanism allows the modeling agent to choose actions that will produce guaranteed minimal improvement of the model accuracy. The working of the proposed maxim in entropy procedure is demonstrated in a multiagent scenario.     

Cohesive Behaviors of Multiagent Systems With Information Flow Constraints

Bacteria, bees, and birds often work together in groups to find food. A group of mobile wheeled robots can be designed to coordinate their activities to achieve a goal. Networked cooperative uninhabited air vehicles (UAVs) are being developed for commercial and military applications. In order for such multiagent systems to succeed it is often critical that they can both maintain cohesive behaviors and appropriately respond to environmental stimuli. In this paper, we characterize cohesiveness of discrete-time multiagent systems as a boundedness or stability property of the agents' position trajectories and use a Lyapunov approach to develop conditions under which local agent actions will lead to cohesive group behaviors even in the presence of i) an interagent "sensing topology" that constrains information flow, where by "information flow," we mean the sensing of positions and velocities of agents, ii) a random but bounded delay and "noise" in sensing other agents' positions and velocities, and iii) noise in sensing a resource profile that represents an environmental stimulus and quantifies the goal of the multiagent system. Simulations are used to illustrate the ideas for multivehicle systems and to make connections to synchronization of coupled oscillators     

SOFTWARE AND PERFORMANCE MEASURES FOR EVALUATING MULTI-AGENT FRAMEWORKS

ABSTRACT

Nowadays, multi-agent frameworks allow to implement very complex systems by means of agent technology. However, this complexity makes it more difficult to evaluate software and runtime characteristics of multiagent systems (MAS). Our aim is to define and study some quantitative measures to measure MAS aspects like development time, reusability, scalability, etc. These measures could be used by engineers to guide the selection among several MAS frameworks. Our study has been carried out in several MAS frameworks like JADE, JATLite, SkeletonAgent, and ZEUS, which have been used to build a MAS application for news retrieval.     

Fuzzy OLAP association rules mining-based modular reinforcement learning approach for multiagent systems.

Multiagent systems and data mining have recently attracted considerable attention in the field of computing. Reinforcement learning is the most commonly used learning process for multiagent systems. However, it still has some drawbacks, including modeling other learning agents present in the domain as part of the state of the environment, and some states are experienced much less than others, or some state-action pairs are never visited during the learning phase. Further, before completing the learning process, an agent cannot exhibit a certain behavior in some states that may be experienced sufficiently. In this study, we propose a novel multiagent learning approach to handle these problems. Our approach is based on utilizing the mining process for modular cooperative learning systems. It incorporates fuzziness and online analytical processing (OLAP) based mining to effectively process the information reported by agents. First, we describe a fuzzy data cube OLAP architecture which facilitates effective storage and processing of the state information reported by agents. This way, the action of the other agent, not even in the visual environment. of the agent under consideration, can simply be predicted by extracting online association rules, a well-known data mining technique, from the constructed data cube. Second, we present a new action selection model, which is also based on association rules mining. Finally, we generalize not sufficiently experienced states, by mining multilevel association rules from the proposed fuzzy data cube. Experimental results obtained on two different versions of a well-known pursuit domain show the robustness and effectiveness of the proposed fuzzy OLAP mining based modular learning approach. Finally, we tested the scalability of the approach presented in this paper and compared it with our previous work on modular-fuzzy Q-learning and ordinary Q-learning.     

Self-organization and multiagent systems: II. Applications and the development technology

Major trends in the development of modern information technologies are to a large extend determined by important practical problems that arise in economics, ecology, safety of society and individuals, and in other fields. Even though these problems seem to be quite different and the requirements for their software implementation are also different, they have many common features, which imply the most stringent requirements for modern information technologies. These features were analyzed in the first part of the present paper. That analysis showed that the new requirements for the model and software implementation of such problems are best met by the multiagent self-organizing system model. In this paper, we consider examples of using this model in various applications and describe their architectures and software implementation; in particular, multiagent self-organization models as applied for flood forecasting and planning and operational enterprise management are described. New capabilities of multiagent self-organizing systems are demonstrated using a self-learning system for detecting intrusions into computer networks as an example. Here, the problem of self-configuration of an overlay network is actually solved. The capabilities of a multiagent self-organizing system in large-scale control in real time are demonstrated using adaptive traffic control in large cities. For the software implementation of multiagent self-organizing systems, special development tools that are different from the existing ones are needed because the conventional top-down development paradigm is inappropriate for self-organizing architectures. The cause is that the global behavior of a multiagent self-organizing system emerges due to local interactions; therefore, it cannot be predicted in advance. For that reason, the bottom-up development model is more appropriate for such systems. In this paper, we give a brief review of the models and approaches proposed for this purpose. One of the promising approaches based on the use of the so-called self-organization design patterns is described in more detail. Results of using the multiagent self-organization model are discussed and prospects of its practical application are estimated.     

MAP: Design and implementation of a mobile agents' platform

The recent development of telecommunication networks has contributed to the success of applications such as information retrieval and electronic commerce, as well as all the services that take advantage of communication in distributed systems. In this area, the emerging technology of mobile agents aroused considerable interest. Mobile agents are applications that can move through the network for carrying out a given task on behalf of the user. In this work we present a platform called MAP (Mobile Agents Platform) for the development and the management of mobile agents. The language used both for developing the platform and for carrying out the agents is Java. The platform gives the user all the basic tools needed for creating some applications based on the use of agents. It enables us to create, run, suspend, resume, deactivate, reactivate local agents, to stop their execution, to make them communicate each other and migrate.     

A platform-independent metamodel for multiagent systems

Various agent-oriented methodologies and metamodels exist to design and develop multiagent systems (MAS) in an abstract manner. Frequently, these frameworks specialise on particular parts of the MAS and only few works have been invested to derive a common standardisation. This limits the impact of agent-related systems in commercial applications. In this paper, we present a metamodel for agent systems that abstracts from existing agent-oriented methodologies, programming languages, and platforms and could thus be considered as platform-independent. This metamodel defines the abstract syntax of a proposed domain-specific modelling language for MAS that is currently under development and provides furthermore the base to generate code out of the generated designs. This is done by applying the principles of model-driven development (MDD) and providing two model transformations that allow transforming the generated models into textual code that can be executed with JACK and JADE.