基于政策自组织系统的软件agent体系结构及其实现

当前自组织系统软件工程面临的一个重要挑战,就是如何设计适宜的个体交互行为来满足自组织系统的宏观涌现需求。针对此问题,提出了一种基于政策的自组织多agent系统的开发方法,此方法通过政策调节引导agent的行为,以期在系统层面得到用户所需求的宏观涌现结果。开发这类系统的核心问题是如何构造系统中的软件agent,使得agent能够感知、理解系统政策,并在遵循政策的前提下实现行为的自主决策。提出了一种基于政策自组织多agent系统的软件agent体系结构,并基于该体系结构设计了运行机制及行为决策算法。通过软件方式实现了一个基于政策的自组织多agent系统开发平台原型,并通过案例实现说明了体系结构、运行机制的有效性。     

An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games

In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective. Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED) from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms. The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically, we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED. Author funded by a doctoral grant of the institute for advancement of scientific technological research in Flanders (IWT).     

MaNEA: A distributed architecture for enforcing norms in open MAS

Norms have been promoted as a coordination mechanism for controlling agent behaviours in open MAS. Thus, agent platforms must provide normative support, allowing both norm-aware and non-norm-aware agents to take part in MAS that are controlled by norms. In this paper, the most relevant proposals on the definition of norm enforcement mechanisms are analyzed. These proposals present several drawbacks that make them unsuitable for open MAS. In response to these problems, this paper describes a new Norm-Enforcing Architecture aimed at controlling norms in open MAS. Specifically, this architecture supports the creation and deletion of norms on-line as well as the dynamic activation and expiration of instances. Finally, it can dynamically adapt to different scale MAS. The efficiency of this architecture has been experimentally evaluated and the results are shown in this paper.     

The influence of social norms and social consciousness on intention reconciliation

Research on resource-bounded agents has established that rational agents need to be able to revise their commitments in light of new opportunities. In the context of collaborative activities, rational agents must be able to reconcile their intentions to do team-related actions with other, conflicting intentions. The SPIRE experimental system allows the process of intention reconciliation in team contexts to be simulated and studied. Initial work with SPIRE examined the impact of environmental factors and agent utility functions on individual and group outcomes in the context of one set of social norms governing collaboration. This paper extends those results by further studying the effect of environmental factors and the agents' level of social consciousness and by comparing the impact of two different types of social norms on agent behavior and outcomes. The results show that the choice of social norms influences the accuracy of the agents' responses to varying environmental factors, as well as the effectiveness of social consciousness and other aspects of agents' utility functions. In experiments using heterogeneous groups of agents, both sets of norms were susceptible to the free-rider effect. However, the gains of the less responsible agents were minimal, suggesting that agent designers would have little incentive to design agents that deviate from the standard level of responsibility to the group.     

Applications and environments for multi-agent systems

This paper addresses multi agent system (MAS) environments from an application perspective. It presents a structured view on environment-centric MAS applications. This comprises three base configurations, which MAS applications may apply directly or combine into a composite configuration. For each configuration, the paper presents key issues, requirements and opportunities (e.g. time management issues, real-world augmentation opportunities and state snapshot requirements). Thus, the paper delineates what environment technology may implement to serve MAS applications. Sample applications illustrate the configurations. Next, electronic institutions provide an example of an environment technology, addressing norms and laws in an agent society, already achieving some maturity. In comparison, application-domain specific environment technologies still are in embryonic stages.     

Debugging complex software systems by means of pathfinder networks

This paper introduces a new methodology based on the use of Pathfinder networks (PFNETs) for the debugging of multi-agent systems (MASs). This methodology is specifically designed to develop a forensic analysis (i.e. a debugging process performed on previously recorded data of the MAS run) of MASs showing complex tissues of relationships between agents (i.e. a high complexity in their social level). Like previous works in the field of forensic analysis of MASs, our approach is performed by considering displays of the system activity which aim to be understandable by human beings. These displays allow us to understand the social behavior of the system, discover emergent behaviors, and debug possible undesirable behaviors. However, it is well known that the visualization of information in a humanly comprehensible way becomes a complex task when large amounts of information have to be represented, as is the case of the social behavior of large-scale MASs. Our methodology tackles this problem through the use of PFNETs, which are considered to reduce the data complexity in order to obtain simple representations that show only the most important global interactions in the system. In addition, the proposed methodology is customizable thanks to the use of two thresholds allowing the user to define the desired specificity level in the display. The proposal is illustrated with a detailed case study considering a complex customer-seller MAS.     

AGENT COORDINATION CONTEXTS IN A MAS COORDINATION INFRASTRUCTURE

In order to harness complexity in multi-agent systems (MAS), first-class entities that mediate interaction between agents and environment are required, which can encapsulate control over MAS behavior and evolution. To this end, MAS infrastructures should provide mediating artifacts, both enabling and constraining agent interactions, and possibly representing admissible agent perceptions and actions over the environment.

Along this line, in this paper, we take the notion of agent coordination context (ACC) as a means to model agent-environment interactions, and show how it can be embedded within a MAS infrastructure in terms of model and runtime structures. Then, we take the TuCSoN coordination infrastructure as a reference, and extend it with the ACC abstraction to integrate the support for coordination with organization and security.     

Predicting the Expected Behavior of Agents that Learn About Agents: The CLRI Framework

We describe a framework and equations used to model and predict the behavior of multi-agent systems (MASs) with learning agents. A difference equation is used for calculating the progression of an agent's error in its decision function, thereby telling us how the agent is expected to fare in the MAS. The equation relies on parameters which capture the agent's learning abilities, such as its change rate, learning rate and retention rate, as well as relevant aspects of the MAS such as the impact that agents have on each other. We validate the framework with experimental results using reinforcement learning agents in a market system, as well as with other experimental results gathered from the AI literature. Finally, we use PAC-theory to show how to calculate bounds on the values of the learning parameters.     

多Agent系统分布式问题求解的代数模型方法(Ⅰ): 社会行为、社会局势和社会动力学

该组论文提出一种新的代数模型方法，用于多Agent系统超分布超并行社会智能问题求解，该方法通过社会动力学和社会智能，统一地处理各种复杂的并行的社会行为，用于求解用常规方法难以处理的许多社会交互问题，本文是组合论文中第一篇，提出多Agent系统分布式问题求解的代数模型结构，讨论多Agent系统中典型社会行为模式及其性质，建立形式化描述，同时也论述了代数模型中的社会局势和社会动力学。     

Analysis of emergent symmetry breaking in collective decision making

We investigate a simulated multi-agent system (MAS) that collectively decides to aggregate at an area of high utility. The agents’ control algorithm is based on random agent–agent encounters and is inspired by the aggregation behavior of honeybees. In this article, we define symmetry breaking, several symmetry breaking measures, and report the phenomenon of emergent symmetry breaking within our observed system. The ability of the MAS to successfully break the symmetry depends significantly on a local-neighborhood-based threshold of the agents’ control algorithm that determines at which number of neighbors the agents stop. This dependency is analyzed and two macroscopic features are determined that significantly influence the symmetry breaking behavior. In addition, we investigate the connection between the ability of the MAS to break symmetries and the ability to stay flexible in a dynamic environment.     

Competing agents in agent-mediated institutions

Social processes and agent interaction always take place in a specific context. A school of thought in social studies analyses them in the framework of institutions. We present in this paper the notion ofagentmediated institutions and show how it is relevant for multi-agent systems (MAS) in general and, more specifically, for MAS that include human agents and software agents involved in socioeconomic interactions. We show how the social interactions of human and software agents taking place in the Cohabited Mixed-Reality Information Spaces (COMRIS) project can be described as such an institution, the Conference Centre institution.     

Agents in object‐oriented software engineering

Software engineers of multi‐agent systems (MASs) are faced with different concerns such as autonomy, adaptation, interaction, collaboration, learning, and mobility, which are essentially different from classical concerns addressed in object‐oriented software engineering. MAS developers, however, have relied mostly on object‐oriented design techniques and programming languages, such as Java. This often leads to a poor separation of MAS concerns and in turn to the production of MASs that are difficult to maintain and reuse. This paper discusses software engineering approaches for MASs, and presents a new method for integrating agents into object‐oriented software engineering from an early stage of design. The proposed approach encourages the separate handling of MAS concerns, and provides a disciplined scheme for their composition. Our proposal explores the benefits of aspect‐oriented software development for the incorporation of agents into object‐oriented systems. We also illustrate our aspect‐oriented approach through the Portalware multi‐agent system, a Web‐based environment for the development of e‐commerce portals. Copyright © 2004 John Wiley & Sons, Ltd.     

Predicting Interactions Between Agents in Agent-Based Modeling and Simulation of Sociotechnical Systems

Agent-based modeling and simulation are a valuable research tools for the analysis of dynamic and emergent phenomena of large-scale complex sociotechnical systems. The dynamic behavior of such systems includes both the individual behavior of heterogeneous agents within the system and the emergent behavior arising from interactions between agents; both must be accurately modeled and efficiently executed in simulations. This paper provides a timing and prediction mechanism for the accurate modeling of interactions among agents, correspondingly increasing the computational efficiency of agent-based simulations. A method for assessing the accuracy of interaction prediction methods is described based on signal detection theory. An intelligent interaction timing agent framework that uses a neural network to predict the timing of interactions between heterogeneous agents is presented; this framework dramatically improves the accuracy of interaction timing without requiring detailed scenario-specific modeling efforts for each simulation configuration.      

Norm reasoning services

Norms are used in open Multi-Agent Systems as a formal specification of deontic statements aimed at regulating the actions of agents and the interactions among them. In this paper, we propose a set of services facilitating the development of both non-normative and normative agents for norm-governed MAS. Specifically, we propose to provide agents with norm reasoning services. These services will help agent designers/developers to programme agents that consider norm reasoning without having to implement the needed mechanisms to reason about norms by themselves. This article shows how these services perform as well as the results of the experiments that we conducted to evaluate their performance.     

Flexible coordinator design for modeling resource sharing in multi-agent systems

One approach to modeling multi-agent systems (MASs) is to employ a method that defines components which describe the local behavior of individual agents, as well as a special component, called a coordinator. The coordinator component coordinates the resource sharing behavior among the agents. The agent models define a set of local plans, and the combination of local plans and a coordinator defines a system’s global plan. Although earlier work has provided the base functionality needed to synthesize inter-agent resource sharing behavior for a global, conflict-free MAS environment, the lack of coordination flexibility limits the modeling capability at both the local plan level and the global plan level. In this paper, we describe a flexible design method that supports a range of coordinator components. The method defines four levels of coordination and an associated four-step coordinator generation process, which allows for the design of coordinators with increasing capabilities for handling complexity associated with resource coordination. Colored Petri net based simulation is used to analyze various properties that derive from different coordinators and synthesis of a reduced coordinator component is discussed for cases that involve homogeneous agents.     

Cooperative Multi-Agent Learning: The State of the Art

Cooperative multi-agent systems (MAS) are ones in which several agents attempt, through their interaction, to jointly solve tasks or to maximize utility. Due to the interactions among the agents, multi-agent problem complexity can rise rapidly with the number of agents or their behavioral sophistication. The challenge this presents to the task of programming solutions to MAS problems has spawned increasing interest in machine learning techniques to automate the search and optimization process. We provide a broad survey of the cooperative multi-agent learning literature. Previous surveys of this area have largely focused on issues common to specific subareas (for example, reinforcement learning, RL or robotics). In this survey we attempt to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics. We find that this broad view leads to a division of the work into two categories, each with its own special issues: applying a single learner to discover joint solutions to multi-agent problems (team learning), or using multiple simultaneous learners, often one per agent (concurrent learning). Additionally, we discuss direct and indirect communication in connection with learning, plus open issues in task decomposition, scalability, and adaptive dynamics. We conclude with a presentation of multi-agent learning problem domains, and a list of multi-agent learning resources.     

Adaptive selection of reactive/deliberate planning for a dynamic environment

This paper proposes and evaluates a new real-time reactive planning approach for a dynamic environment. In addition to having the features of conventional real-time reactive planning, which can react in a dynamic environment, our planning can perform deliberate planning appropriately. The proposed planning uses three kinds of agents: behavior agents that control simple behavior, planning agents that make plans to achieve their own goals, and behavior-selection agents that intermediate between behavior agents and planning agents. They coordinate a plan in an emergent way for the planning system as a whole. We confirmed the effectiveness of our planning by means of a simulation. Furthermore, we implemented an active-vision system and used it to verify the real-world efficiency of our planning.     

IODA: an interaction-oriented approach for multi-agent based simulations

Multi-Agent Systems (MAS) design methodologies and Integrated Development Environments exhibit many interesting properties that also support simulation design. Yet, in their current form, they are not appropriate enough to model Multi-Agent Based Simulations (MABS). Indeed, their design is focused on the functionalities to be achieved by the MAS and the allocation of these functionalities among software agents. In that context, the most important point of design is the organization of the agents and how they communicate with each other. On the opposite, MABS aim at studying emergent phenomena, the origin of which lies in the interactions between entities and their interaction with the environment. In that context, the interactions are not limited to exchanging messages but can also be fundamental physical interactions or any other actions involving simultaneously the environment and one or several agents. To deal with this issue, this paper presents the core notions of the Interaction-Oriented Design of Agent simulations (IODA) approach to simulation design. It includes a design methodology, a model, an architecture and also JEDI, a simple implementation of IODA concepts for reactive agents. First of all, our approach focuses on the design of an agent-independent specification of behaviors, called interactions. These interactions are not limited to the analysis phase of simulation: they are made concrete both in the model and at the implementation stage. In addition, no distinction is made between agents and objects: all entities of the simulation are agents. Owing to this principle, designing which interactions occur between agents, as well as how agents act, is achieved by means of an intuitive plug-and-play process, where interaction abilities are distributed among the agents. Besides, the guidelines provided by IODA are not limited to the specification of the model as they help the designer from the very beginning towards a concrete implementation of the simulation.     

Design and implementation of a multiagent stock trading system

Stock trading is one of the key items in an economy and estimating its behavior and taking the best decision in it are among the most challenging issues. Solutions based on intelligent agent systems are proposed to cope with those challenges. Agents in a multiagent system (MAS) can share a common goal or they can pursue their own interests. That nature of MASs exactly fits the requirements of a free market economy. Although existing studies include noteworthy proposals on agent‐based market simulation and researchers discuss theoretical design issues of agent‐based stock exchange systems, unfortunately only a very few of the studies consider exact development and implementation of multiagent stock trading systems within the software engineering perspective and guides to the software engineers for constructing such software systems starting from scratch. To fill this gap, in this paper, we discuss the development of a multiagent‐based stock trading system by taking into consideration software design according to a well‐defined agent oriented software engineering methodology and implementation with a widely‐used MAS software development framework. Each participant in the system is first designed as belief–desire–intention agents with their facts, goals, and plans, and then belief–desire–intention reasoning and behavioral structure of the designed agents are implemented. Lessons learned during design and development within the software engineering perspective and evaluation of the implemented multiagent stock exchange system are also reported. Copyright © 2011 John Wiley & Sons, Ltd.     

Decentralised event-triggered cooperative control with limited communication

This note studies event-triggered control of Multi-Agent Systems (MAS) with first-order integrator dynamics. It extends previous work on event-triggered consensus by considering limited communication capabilities through strict peer-to-peer non-continuous information exchange. The approach provides both a decentralised control law and a decentralised communication policy. Communication events require no global information and are based only on local state errors; agents do not require a global sampling period or synchronous broadcasting as in sampled-data approaches. The proposed decentralised event-triggered control technique guarantees that the inter-event times for each agent are strictly positive. Finally, the ideas in this note are used to consider the practical scenario where agents are able to exchange only quantised measurements of their states.