Off-line synthesis of evolutionarily stable normative systems

Within the area of multi-agent systems, normative systems are a widely used framework for the coordination of interdependent activities. A crucial problem associated with normative systems is that of synthesising norms that will effectively accomplish a coordination task and that the agents will comply with. Many works in the literature focus on the on-line synthesis of a single, evolutionarily stable norm (convention) whose compliance forms a rational choice for the agents and that effectively coordinates them in one particular coordination situation that needs to be identified and modelled as a game in advance. In this work, we introduce a framework for the automatic off-line synthesis of evolutionarily stable normative systems that coordinate the agents in multiple interdependent coordination situations that cannot be easily identified in advance nor resolved separately. Our framework roots in evolutionary game theory. It considers multi-agent systems in which the potential conflict situations can be automatically enumerated by employing MAS simulations along with basic domain information. Our framework simulates an evolutionary process whereby successful norms prosper and spread within the agent population, while unsuccessful norms are discarded. The outputs of such a natural selection process are sets of codependent norms that, together, effectively coordinate the agents in multiple interdependent situations and are evolutionarily stable. We empirically show the effectiveness of our approach through empirical evaluation in a simulated traffic domain.     

Anticipation based on constraint processing in a multi-agent context

Anticipation is a general concept used and applied in various domains. Many studies in the field of artificial intelligence have investigated the capacity for anticipation. In this article, we focus on the use of anticipation in multi-agent coordination, particularly preventive anticipation which consists of anticipating undesirable future situations in order to avoid them. We propose to use constraint processing to formalize preventive anticipation in the context of multi-agent coordination. The resulting algorithm allows any action that may induce an undesirable future state to be detected upstream of any multi-agent coordination process. Our proposed method is instantiated in a road traffic simulation tool. For the specific question of simulating traffic at road junctions, our results show that taking anticipation into account allows globally realistic behaviors to be reproduced without provoking gridlock between the simulated vehicles.     

A multiagent systems with Petri Net approach for simulation of urban traffic networks

This paper presents a novel model framework for complex urban traffic systems based on the interconnection of a dynamical multi-agent system in a macroscopic level. The agents describe all the types of street segments, intersections, sources and sinks of cars, modelling the behavior of the flow of vehicles through them as simple differential equations. These agents include the phenomena of changes in the flow rate due to congestions, traffic signals and the density of the vehicles. Traffic signal changes are obtained by the evolution of Petri Nets, in order to represent a more real behavior. Therefore, a complex network can be constructed by the interconnection of the agents, in continuous time, and the Petri Nets, in a discrete-event behavior, becoming a hybrid and scalable system. In order to analyze the performance of the approach, a real set of streets and intersections in Montevideo City is studied. Also, the approach is compared with a simulation realized in the software TSIS-CORSIM, which contains real data of density of vehicles. The multi-agent system achieves comparable results, taking into account the differences in the level of details respect to TSIS-CORSIM. Thus, the results can represent the most important issues of vehicular traffic with less computational resources.     

Dialogue games that agents play within a society

Human societies have long used the capability of argumentation and dialogue to overcome and resolve conflicts that may arise within their communities. Today, there is an increasing level of interest in the application of such dialogue games within artificial agent societies. In particular, within the field of multi-agent systems, this theory of argumentation and dialogue games has become instrumental in designing rich interaction protocols and in providing agents with a means to manage and resolve conflicts. However, to date, much of the existing literature focuses on formulating theoretically sound and complete models for multi-agent systems. Nonetheless, in so doing, it has tended to overlook the computational implications of applying such models in agent societies, especially ones with complex social structures. Furthermore, the systemic impact of using argumentation in multi-agent societies and its interplay with other forms of social influences (such as those that emanate from the roles and relationships of a society) within such contexts has also received comparatively little attention. To this end, this paper presents a significant step towards bridging these gaps for one of the most important dialogue game types; namely argumentation-based negotiation (ABN). The contributions are three fold. First, we present a both theoretically grounded and computationally tractable ABN framework that allows agents to argue, negotiate, and resolve conflicts relating to their social influences within a multi-agent society. In particular, the model encapsulates four fundamental elements: (i) a scheme that captures the stereotypical pattern of reasoning about rights and obligations in an agent society, (ii) a mechanism to use this scheme to systematically identify social arguments to use in such contexts, (iii) a language and a protocol to govern the agent interactions, and (iv) a set of decision functions to enable agents to participate in such dialogues. Second, we use this framework to devise a series of concrete algorithms that give agents a set of ABN strategies to argue and resolve conflicts in a multi-agent task allocation scenario. In so doing, we exemplify the versatility of our framework and its ability to facilitate complex argumentation dialogues within artificial agent societies. Finally, we carry out a series of experiments to identify how and when argumentation can be useful for agent societies. In particular, our results show: a clear inverse correlation between the benefit of arguing and the resources available within the context; that when agents operate with imperfect knowledge, an arguing approach allows them to perform more effectively than a non-arguing one; that arguing earlier in an ABN interaction presents a more efficient method than arguing later in the interaction; and that allowing agents to negotiate their social influences presents both an effective and an efficient method that enhances their performance within a society.     

Learning Situation-Specific Coordination in Cooperative Multi-agent Systems

Achieving effective cooperation in a multi-agent system is a difficult problem for a number of reasons such as limited and possibly out-dated views of activities of other agents and uncertainty about the outcomes of interacting non-local tasks. In this paper, we present a learning system called COLLAGE, that endows the agents with the capability to learn how to choose the most appropriate coordination strategy from a set of available coordination strategies. COLLAGE relies on meta-level information about agents' problem solving situations to guide them towards a suitable choice for a coordination strategy. We present empirical results that strongly indicate the effectiveness of the learning algorithm.     

Using Domain-Independent Exception Handling Services to Enable Robust Open Multi-Agent Systems: The Case of Agent Death

This paper addresses a simple but critical question: how can we create robust multi-agent systems out of the often unreliable agents and infrastructures we can expect to find in open systems contexts? We propose an approach to this problem based on distinct exception handling (EH) services that enact coordination protocol-specific but domain-independent strategies to monitor agent systems for problems (‘exceptions’) and intervene when necessary to avoid or resolve them. The value of this approach is demonstrated for the ‘agent death’ exception in the Contract Net protocol; we show through simulation that the EH service approach provides substantially improved performance compared to existing approaches in a way that is appropriate for open multi-agent systems.     

A decentralized approach for convention emergence in multi-agent systems

The field of convention emergence studies how agents involved in repeated coordination games can reach consensus through only local interactions. The literature on this topic is vast and is motivated by human societies, mainly addressing coordination problems between human agents, such as who gets to redial after a dropped telephone call. In contrast, real-world engineering problems, such as coordination in wireless sensor networks, involve agents with limited resources and knowledge and thus pose certain restrictions on the complexity of the coordination mechanisms. Due to these restrictions, strategies proposed for human coordination may not be suitable for engineering applications and need to be further explored in the context of real-world application domains. In this article we take the role of designers of large decentralized multi-agent systems. We investigate the factors that speed up the convergence process of agents arranged in different static and dynamic topologies and under different interaction models, typical for engineering applications. We also study coordination problems both under partial observability and in the presence of faults (or noise). The main contributions of this article are that we propose an approach for emergent coordination, motivated by highly constrained devices, such as wireless nodes and swarm bots, in the absence of a central entity and perform extensive theoretical and empirical studies. Our approach is called Win-Stay Lose-probabilistic-Shift, generalizing two well-known strategies in game theory that have been applied in other domains. We demonstrate that our approach performs well in different settings under limited information and imposes minimal system requirements, due to its simplicity. Moreover, our technique outperforms state-of-the-art coordination mechanisms, guarantees full convergence in any topology and has the property that all convention states are absorbing.     

一种基于社会承诺的Agent组织模型OMSC

多Agent领域所面临的一个重大的挑战是解决开放异质的多Agent系统中自治Agent间的协调问题。多Agent为了协调它们之间的活动,需要进行交互。社会承诺作为一种通信和交互机制,为自治的多Agent提供了一种协调的途径。然而,仅靠交互难以实现多Agent间的协调。Agent组织作为一种协调模型可以有效地控制多Agent间的交互与合作。论文将社会承诺和Agent组织两种协调机制相结合,提出一种基于社会承诺的Agent组织模型OMSC,分析了Agent如何用社会承诺进行推理以及基于社会承诺的多Agent系统并给出了一个实例,为多Agent间的协调提供了一种新的方法。     

Agent and multi-agent applications to support distributed communities of practice: a short review

This paper concerns the relationship between agents or multi-agent systems and distributed communities of practice. It presents a review of a number of agent and multi-agent applications with features that could contribute to supporting distributed communities of practice. The association is promising because of features like autonomy, pro-activity, flexibility or ability to integrate systems that characterize agents and multi-agent systems. Furthermore, such an association is a step towards building mixed communities of humans and artificial agents. To understand how agents and multi-agent systems could answer some of the needs of distributed communities of practice, we organize the analyzed applications into five different categories defined by considering the main activities of a community, namely: Individual Participation, Synchronous Interactions, Asynchronous Interactions, Publishing and Community Cultivation. Such a classification helps us identify the relevant features of the current technology and determine some that should be further developed, e.g. to support community coordination or gather information related to virtual communities. For each application we selected, we present its main approach and point out its potential interest.     

Learning collaboration strategies for committees of learning agents

A main issue in cooperation in multi-agent systems is how an agent decides in which situations is better to cooperate with other agents, and with which agents does the agent cooperate. Specifically in this paper we focus on multi-agent systems composed of learning agents, where the goal of the agents is to achieve a high accuracy on predicting the correct solution of the problems they encounter. For that purpose, when encountering a new problem each agent has to decide whether to solve it individually or to ask other agents for collaboration. We will see that learning agents can collaborate forming committees in order to improve performance. Moreover, in this paper we will present a proactive learning approach that will allow the agents to learn when to convene a committee and with which agents to invite to join the committee. Our experiments show that learning results in smaller committees while maintaining (and sometimes improving) the problem solving accuracy than forming committees composed of all agents.     

Diagnosing resource usage failures in multi-agent systems

In the not-so-far future, autonomous vehicles will be ubiquitous and, consequently, need to be coordinated to avoid traffic jams and car accidents. A failure in one or more autonomous vehicles may break this coordination, resulting in reduced efficiency (due to traffic load) or even bodily harm (due to accidents). The challenge we address in this paper is to identify the root cause of such failures. Identifying the faulty vehicles in such cases is crucial in order to know which vehicles to repair to avoid future failures as well as for determining accountability (e.g., for legal purposes). More generally, this paper discusses multi-agent systems (MAS) in which the agents use a shared pool of resources and they coordinate to avoid resource contention by agreeing on a temporal resource allocation. The problem we address, called the Temporal Multi-Agent Resource Allocation (TMARA) diagnosis problem (TMARA-Diag), is to find the root cause of failures in such MAS that are caused by malfunctioning agents that use resources not allocated to them. As in the autonomous vehicles example, such failures may cause the MAS to perform suboptimally or even fail, potentially causing a chain reaction of failures, and we aim to identify the root cause of such failures, i.e., which agents did not follow the planned resource allocation. We show how to formalize TMARA-Diag as a model-based diagnosis problem and how to compile it to a set of logical constraints that can be compiled to Boolean satisfiability (SAT) and solved efficiently with modern SAT solvers. Importantly, the proposed solution does not require the agents to share their actual plans, only the agreed upon temporal resource allocation and the resources used at the time of failure. Such solutions are key in the development and success of intelligent, large, and security-aware MAS.     

基于近似模拟的多智能体系统分布式层次控制设计

本文主要探讨基于近似模拟的多智能体系统分布式层次控制的设计问题。为降低问题复杂度,首先建立了一个简单的抽象系统指导各智能体,继而讨论这一抽象系统和多智能体之间的模拟关系。借助于该抽象系统,给出多智能体系统完成一类协调任务的分布式层次控制器。同时利用模拟函数和共同Lyapunov函数,进一步分析了多个体系统在切换拓扑下的集体行为。     

自适应事件触发的马尔科夫跳变多智能体系统一致性

针对非线性马尔科夫跳变多智能体系统在有向固定拓扑下的领导跟随一致性问题,为减少智能体间不必要的通信传输,节约网络资源,保证系统性能,提出一种自适应事件触发控制策略.首先,将每一个智能体均视为马尔科夫跳变系统,且马尔科夫链的转移概率部分未知;通过简单的模型转换建立误差系统,将多智能体系统一致性问题转化为误差系统的稳定性问题;在此基础上,构造合适的Lyapunov-Krasovskii泛函并利用Jensen不等式和线性矩阵不等式等技术给出使多智能体系统达到领导跟随一致性的充分条件及控制器设计方法;通过求解线性矩阵不等式可以得到多智能体系统一致性控制器增益矩阵和事件触发参数矩阵;最后,通过数值仿真验证所提出方法的有效性.     

Highly-scalable traffic management of autonomous industrial transportation systems

In this paper, we present a novel method for highly-scalable coordination of free-ranging automated guided vehicles in industrial logistics and manufacturing scenarios. The primary aim of this method is to enhance the current industrial state-of-the-art multi-vehicle transportation systems, which, despite their long presence on the factory floor and significant advances over the last decades, still rely on a centralized controller and predetermined network of paths. In order to eliminate the major drawbacks of such systems, including poor scalability, low flexibility, and the presence of a single point of failure, in the proposed control approach vehicles autonomously execute their assigned pick-up and delivery operations by running a fully decentralized control algorithm. The algorithm integrates path planning and motion coordination capabilities and relies on a two-layer control architecture with topological workspace representation on the top layer and state-lattice representation on the bottom layer. Each vehicle plans its own shortest feasible path toward the assigned goal location and resolves conflict situations with other vehicles as they arise along the way. The motion coordination strategy relies on the private-zone mechanism ensuring reliable collision avoidance, and local negotiations within the limited communication radius ensuring high scalability as the number of vehicles in the fleet increases. We present experimental validation results obtained on a system comprising six Pioneer 3DX robots in four different scenarios and simulation results with up to fifty vehicles. We also analyze the overall quality of the proposed traffic management method and compare its performance to other state-of-the-art multi-vehicle coordination approaches.     

A multi-agent approach for integrated emergency vehicle dispatching and covering problem

The most important decisions that should be made by emergency vehicle managers are related to the allocation and the covering problems. The allocation (or dispatching) problem consists of deciding which vehicle must be assigned to assist an emergency in the best times. The covering problem aims at keeping the region under surveillance well-covered by relocating available vehicles. As components are geographically distributed, decentralized solution approaches may present several advantages. This paper develops a decentralized distributed solution approach based on multi-agent systems (MAS) to manage the emergency vehicles. The proposed system integrates the dispatching of vehicles to calls with zone coverage issues. This integration means that allocation and covering decisions are considered jointly. The idea of MAS has been applied in many others real-world contexts, and has been proven to provide more flexibility, reliability, adaptability and reconfigurability. To our knowledge, there is no existing work that uses MAS for real-time emergency vehicle allocation problem while accounting for the coverage requirements for future demands. We propose a multi-agent architecture that fit the real emergency systems, and that aims at keeping good performance compared to the centralized solution. The objective is to coordinate agents to reach good quality solutions in a distributed way. For this purpose two approaches are examined. The first one is used to show the impact of distributing data and control on the solution quality, since the dispatching decisions are based only on local evaluations of the fitness. The second approach is based on implicit agents' coordination using a more refined and efficient auction mechanism. The performance of each approach is compared to the centralized solution obtained by solving the proposed model with ILOG CPLEX solver. The obtained results show the importance of the coordination method to keep a good quality of service while distributing data and decision making, and prove the performance of the second approach.     

多智能体系统动态协调与分布式控制设计

多智能体系统的主要研究目的在于探索由个体之间的相互作用所产生的群体协调现象的内在机制和原理,而控制或反馈在多智能体协调运动中起着至关重要的作用．本文集中讨论了多智能体协调研究中的几个新兴的基本问题,包括输出调节、集合协调和覆盖．文中着重介绍了分布式估计和内模原理两种多智能体系统分布式输出调节方法及相关的研究进展：关于多智能体系统的目标集合协调,本文从集合聚集和集合优化两方面做了详尽论述：多智能体覆盖有多种分类方式,从覆盖对象的特征出发可将其划分为区域覆盖、边界覆盖和动态目标覆盖3种类型,并对它们的研究背景和最新成果予以介绍．另外文章还对多智能体系统协调控制的理论和应用研究进行了展望．     

Individual learning of coordination knowledge

Abstract. Social agents, both human and computational, inhabiting a world containing multiple active agents, need to coordinate their activities. This is because agents share resources, and without proper coordination or ‘rules of the road’, everybody will be interfering with the plans of others. As such, we need coordination schemes that allow agents to effectively achieve local goals without adversely affecting the problem-solving capabilities of other agents. Researchers in the field of Distributed Artificial Intelligence (DAI) have developed a variety of coordination schemes under different assumptions about agent capabilities and relationships. Whereas some of these researchers have been motivated by human cognitive biases, others have approached it as an engineering problem of designing the most effective coordination architecture or protocol. We evaluate individual and concurrent learning by multiple, autonomous agents as a means for acquiring coordination knowledge. We show that a uniform reinforcement learning algorithm suffices as a coordination mechanism in both cooperative and adversarial situations. Using a number of multi-agent learning scenarios with both tight and loose coupling between agents and with immediate as well as delayed feedback, we demonstrate that agents can consistently develop effective policies to coordinate their actions without explicit information sharing. We demonstrate the viabilityof using both the Q-learning algorithm and genetic algorithm based classifier systems with different pay-off schemes, namely the bucket brigade algorithm (BBA) and the profit sharing plan (PSP), for developing agent coordination on two different multi-agent domains. In addition, we show that a semi-random scheme for action selection is preferable to the more traditional fitness proportionate selection scheme used in classifier systems.     

A field-based versus a protocol-based approach for adaptive task assignment

Task assignment in multi-agent systems is a complex coordination problem, in particular in systems that are subject to dynamic and changing operating conditions. To enable agents to deal with dynamism and change, adaptive task assignment approaches are needed. In this paper, we study two approaches for adaptive task assignment that are characteristic for two classical families of task assignment approaches. FiTA is a field-based approach in which tasks emit fields in the environment that guide idle agents to tasks. DynCNET is a protocol-based approach that extends Standard Contract Net (CNET). In DynCNET, agents use explicit negotiation to assign tasks. We compare both approaches in a simulation of an industrial automated transportation system. Our experiences show that: (1) the performance of DynCNET and FiTA are similar, while both outperform CNET; (2) the complexity to engineer DynCNET is similar to FiTA but much more complex than CNET; (3) whereas task assignment with FiTA is an emergent solution, DynCNET specifies the interaction among agents explicitly allowing engineers to reason on the assignment of tasks, (4) FiTA is inherently robust to message loss while DynCNET requires substantial additional support. The tradeoff between (3) and (4) is an important criteria for the selection of an adaptive task assignment approach in practice.     

Multi-agent discrete-time graphical games and reinforcement learning solutions

This paper introduces a new class of multi-agent discrete-time dynamic games, known in the literature as dynamic graphical games. For that reason a local performance index is defined for each agent that depends only on the local information available to each agent. Nash equilibrium policies and best-response policies are given in terms of the solutions to the discrete-time coupled Hamilton–Jacobi equations. Since in these games the interactions between the agents are prescribed by a communication graph structure we have to introduce a new notion of Nash equilibrium. It is proved that this notion holds if all agents are in Nash equilibrium and the graph is strongly connected. A novel reinforcement learning value iteration algorithm is given to solve the dynamic graphical games in an online manner along with its proof of convergence. The policies of the agents form a Nash equilibrium when all the agents in the neighborhood update their policies, and a best response outcome when the agents in the neighborhood are kept constant. The paper brings together discrete Hamiltonian mechanics, distributed multi-agent control, optimal control theory, and game theory to formulate and solve these multi-agent dynamic graphical games. A simulation example shows the effectiveness of the proposed approach in a leader-synchronization case along with optimality guarantees.     

Multi-target consensus circle pursuit for multi-agent systems via a distributed multi-flocking method

This paper studies the multi-target consensus pursuit problem of multi-agent systems. For solving the problem, a distributed multi-flocking method is designed based on the partial information exchange, which is employed to realise the pursuit of multi-target and the uniform distribution of the number of pursuing agents with the dynamic target. Combining with the proposed circle formation control strategy, agents can adaptively choose the target to form the different circle formation groups accomplishing a multi-target pursuit. The speed state of pursuing agents in each group converges to the same value. A Lyapunov approach is utilised to analyse the stability of multi-agent systems. In addition, a sufficient condition is given for achieving the dynamic target consensus pursuit, and which is then analysed. Finally, simulation results verify the effectiveness of the proposed approaches.