Distributed learning and cooperative control for multi-agent systems

This paper presents an algorithm and analysis of distributed learning and cooperative control for a multi-agent system so that a global goal of the overall system can be achieved by locally acting agents. We consider a resource-constrained multi-agent system, in which each agent has limited capabilities in terms of sensing, computation, and communication. The proposed algorithm is executed by each agent independently to estimate an unknown field of interest from noisy measurements and to coordinate multiple agents in a distributed manner to discover peaks of the unknown field. Each mobile agent maintains its own local estimate of the field and updates the estimate using collective measurements from itself and nearby agents. Each agent then moves towards peaks of the field using the gradient of its estimated field while avoiding collision and maintaining communication connectivity. The proposed algorithm is based on a recursive spatial estimation of an unknown field. We show that the closed-loop dynamics of the proposed multi-agent system can be transformed into a form of a stochastic approximation algorithm and prove its convergence using Ljung’s ordinary differential equation (ODE) approach. We also present extensive simulation results supporting our theoretical results.     

Distributed cooperative stabilisation of continuous-time uncertain nonlinear multi-agent systems

This paper addresses the distributed cooperative stabilisation problem of continuous-time uncertain nonlinear multi-agent systems. By approximating the uncertain dynamics using neural networks, a distributed adaptive cooperative controller, based on the state information of the neighbouring agents, is proposed. The control design is developed for any undirected connected communication topologies without requiring the accurate model of each agent. This result is further extended to the output feedback case. An observer-based distributed cooperative controller is devised and a parameter dependent Riccati inequality is employed to prove stability of the overall multi-agent systems. This design is less complex than the other design methods and has a favourable decouple property between the observer design and the controller design for uncertain nonlinear multi-agent systems. For both cases, the developed controllers guarantee that all signals in the closed-loop network are uniformly ultimately bounded, and the states of all agents cooperatively converge to a small neighbourhood of origin. A comparative study is given to show the efficacy of the proposed method.     

Distributed cooperative reinforcement learning for multi-agent system with collision avoidance

In this work, we present an optimal cooperative control scheme for a multi-agent system in an unknown dynamic obstacle environment, based on an improved distributed cooperative reinforcement learning (RL) strategy with a three-layer collaborative mechanism. The three collaborative layers are collaborative perception layer, collaborative control layer, and collaborative evaluation layer. The incorporation of collaborative perception expands the perception range of a single agent, and improves the early warning ability of the agents for the obstacles. Neural networks (NNs) are employed to approximate the cost function and the optimal controller of each agent, where the NN weight matrices are collaboratively optimized to achieve global optimal performance. The distinction of the proposed control strategy is that cooperation of the agents is embodied not only in the input of NNs (in a collaborative perception layer) but also in their weight updating procedure (in the collaborative evaluation and collaborative control layers). Comparative simulations are carried out to demonstrate the effectiveness and performance of the proposed RL-based cooperative control scheme.     

有限时间内的动车组群分布式协同巡航多智能体控制

本文研究了动车组群系统的移动闭塞协同巡航控制问题,通过设计有限时间分布式协同控制算法实现了多车高速协同巡航.算法取消了动车组中心控制单元,将单列动车组的每个动力单元视为一个智能体,不同动车组之间的智能体形成多智能体群系统.设计了动车组智能体有限时间分布式协同控制算法,该算法首先实现了单列动车组各个动力单元快速追踪到期望的速度,且保证车钩位移在安全范围内,其次该算法还能实现动车组群高速巡航等间距运行,提高了线路利用率的同时避免了碰撞.最后进行了理论分析,证明了算法的稳定性和有限时间收敛性,并提供了仿真结果验证了该算法的有效性.     

改进的多agent合作求解过程

在WOOLDRIDGE和张双民的工作基础上,考虑合作提供方的主观愿望,给出一个多agent系统合作求解模型,并考虑合作步骤重复与失败的情况．该模型体现多agent系统的动态属性,使理论与实际更加接近．     

An evidential cooperative multi-agent system

The cooperative learning systems (COLS) are an interesting way of research in Artificial Intelligence. This is because an intelligence form can emerge by interacting simple agents in these systems. In literature, we can find many learning techniques, which can be improved by combining them to a cooperative learning, this one can be considered as a special case of bagging. In particular, learning classifier systems (LCS) are adapted to cooperative learning systems because LCS manipulate rules and, hence, knowledge exchange between agents is facilitated. However, a COLS has to use a combination mechanism in order to aggregate information exchanged between agents, this combination mechanism must take in consideration the nature of information manipulated by the agents. In this paper we investigate a cooperative learning system based on the Evidential Classifier System, the cooperative system uses Dempster–Shafer theory as a support to make data fusion. This is due to the fact that the Evidential Classifier System is itself based on this theory. We present some ways to make cooperation by using this architecture and discuss the characteristics of such architecture by comparing the obtained results with those obtained by an individual approach.     

Severity-sensitive norm-governed multi-agent planning

In making practical decisions, agents are expected to comply with ideals of behaviour, or norms. In reality, it may not be possible for an individual, or a team of agents, to be fully compliant—actual behaviour often differs from the ideal. The question we address in this paper is how we can design agents that act in such a way that they select collective strategies to avoid more critical failures (norm violations), and mitigate the effects of violations that do occur. We model the normative requirements of a system through contrary-to-duty obligations and violation severity levels, and propose a novel multi-agent planning mechanism based on Decentralised POMDPs that uses a qualitative reward function to capture levels of compliance: N-Dec-POMDPs. We develop mechanisms for solving this type of multi-agent planning problem and show, through empirical analysis, that joint policies generated are equally as good as those produced through existing methods but with significant reductions in execution time.     

多agent规划综述*

多Agent规划是智能规划和多Agent系统的交叉领域,随着智能规划领域研究范围的不断扩展和多Agent系统领域研究内容的不断深入,多Agent规划受到了越来越多的关注。有鉴于此,本文对多Agent规划的概念和主要方法进行全面综述。具体内容包括智能规划和多Agent系统的背景介绍、多Agent规划的各种形式化描述方式以及基于规划修复、合并或者马尔可夫决策过程的的分布式规划方法。最后,还给出了多Agent规划的发展趋势。     

局部合作多智能体Q-学习研究

强化学习在多Agent系统中面对的最大问题就是随着Agent数量的增加而导致的状态和动作空间的指数增长以及随之而来的缓慢的学习效率。采用了一种局部合作的Q-学习方法,只有在Agent之间有明确协作时才考察联合动作,否则,就只进行简单的个体Agent的Q-学习,从而使的学习时所要考察的状态动作对值大大减少。最后算法在捕食者-猎物的追逐问题和机器人足球仿真2D上的实验结果,与常用的多Agent强化学习技术相比有更好的效能。     

A cooperative multi-agent robotics system: Design and modelling

This paper presents the development of the robotic multi-agent system SMART. In this system, the agent concept is applied to both hardware and software entities. Hardware agents are robots, with three and four legs, and an IP-camera that takes images of the scene where the cooperative task is carried out. Hardware agents strongly cooperate with software agents. These latter agents can be classified into image processing, communications, task management and decision making, planning and trajectory generation agents. To model, control and evaluate the performance of cooperative tasks among agents, a kind of Petri Net, called Work-Flow Petri Net, is used. Experimental results shows the good performance of the system.     

Fault-tolerant cooperative tasking for multi-agent systems

A natural way for cooperative tasking in multi-agent systems is through a top-down design by decomposing a global task into subtasks for each individual agent such that the accomplishments of these subtasks will guarantee the achievement of the global task. In our previous works [Karimadini, M., and Lin, H. (2011c), ‘Guaranteed Global Performance Through Local Coordinations’, Automatica, 47, 890--898; Karimadini, M., and Lin, H. (2011a), ‘Cooperative Tasking for Deterministic Specification Automata’, submitted for publication, online available at: http://arxiv.org/abs/1101.2002], we presented necessary and sufficient conditions on the decomposability of a global task automaton between cooperative agents. As a follow-up work, this article deals with the robustness issues of the proposed top-down design approach with respect to event failures in the multi-agent systems. The main concern under event failure is whether a previously decomposable task can still be achieved collectively by the agents, and if not, we would like to investigate that under what conditions the global task could be robustly accomplished. This is actually the fault-tolerance issue of the top-down design, and the results provide designers with hints on which events are fragile with respect to failures, and whether redundancies are needed. The main objective of this article is to identify necessary and sufficient conditions on failed events under which a decomposable global task can still be achieved successfully. For such a purpose, a notion called passivity is introduced to characterise the type of event failures. The passivity is found to reflect the redundancy of communication links over shared events, based on which necessary and sufficient conditions for the reliability of cooperative tasking under event failures are derived, followed by illustrative examples and remarks for the derived conditions.     

基于Web服务的多代理协同平台

讨论了基于Web服务的多代理协同平台,可以较好地解决异构CSCW系统之间的组织、调度及通信问题,为虚拟企业环境下的协同技术提供支持。给出了面向Web服务工作流技术和KQML协议的协同平台的实现策略。     

分布式优化的多智能体方法

分布式优化作为分布式协调控制领域中的一个基本而重要的研究课题,近年来,不同领域的众多学者对其产生了广泛的研究兴趣.本文总结归纳了分布式优化的研究现状和近期的研究成果,重点对离线分布式优化和在线分布式优化进行了阐述,并从算法设计和收敛性分析这两个角度进行了剖析.特别地,针对一类混合均衡问题,本文介绍了一类分布式求解算法.最后,阐述了当前尚未解决的问题和未来的研究方向.     

Collaborative privacy preserving multi-agent planning

In many cases several entities, such as commercial companies, need to work together towards the achievement of joint goals, while hiding certain private information. To collaborate effectively, some sort of plan is needed to coordinate the different entities. We address the problem of automatically generating such a coordination plan while preserving the agents’ privacy. Maintaining privacy is challenging when planning for multiple agents, especially when tight collaboration is needed and a global high-level view of the plan is required. In this work we present the Greedy Privacy-Preserving Planner (GPPP), a privacy preserving planning algorithm in which the agents collaboratively generate an abstract and approximate global coordination plan and then individually extend the global plan to executable plans. To guide GPPP, we propose two domain independent privacy preserving heuristics based on landmarks and pattern databases, which are classical heuristics for single agent search. These heuristics, called privacy-preserving landmarks and privacy preserving PDBs, are agnostic to the planning algorithm and can be used by other privacy-preserving planning algorithms. Empirically, we demonstrate on benchmark domains the benefits of using these heuristics and the advantage of GPPP over existing privacy preserving planners for the multi-agent STRIPS formalism.     

随机多智能体系统的多率协同控制

本文针对一类带有非线性动力学的随机多智能体系统研究了基于多率采样机制的一致性控制问题。首先,引入多率采样机制异步地对智能体的状态进行采样,并且使用缓存器来同步智能体不同分量的状态信息。然后,提出了一种基于边的事件触发传输机制将每个智能体的状态信息定向传输给邻居智能体,从而减少了通信资源的使用和控制器的更新频率。接着,本文根据所设计的事件触发传输机制设计了相应的控制协议使得随机多智能体系统可以实现均方一致性。因为每条边上的事件都是独立发生的,所以智能体的邻居之间不需要保证信息的时钟同步。最后,给出了数值仿真例子。通过将同步采样机制、异步采样机制和多率采样机制进行比较,说明了所提出的边事件触发传输机制在多率采样下可以获得更好的系统性能。     

Distributed cooperative Web servers

Traditional techniques for a distributed web server design rely on manipulation of central resources, such as routers or DNS services, to distribute requests designated for a single IP address to multiple web servers. The goal of the distributed cooperative Web server (DCWS) system development is to explore application-level techniques for distributing web content. We achieve this by dynamically manipulating the hyperlinks stored within the web documents themselves. The DCWS system effectively eliminates the bottleneck of centralized resources, while balancing the load among distributed web servers. DCWS servers may be located in different networks, or even different continents and still balance load effectively. DCWS system design is fully compatible with existing HTTP protocol semantics and existing web client software products.     

Distributed MASON: A scalable distributed multi-agent simulation environment

Computational Social Science (CSS) involves interdisciplinary fields and exploits computational methods, such as social network analysis as well as computer simulation with the goal of better understanding social phenomena.Agent-Based Models (ABMs) represent an effective research tool for CSS and consist of a class of models, which, aim to emulate or predict complex phenomena through a set of simple rules (i.e., independent actions, interactions and adaptation), performed by multiple agents. The efficiency and scalability of ABMs systems are typically obtained distributing the overall computation on several machines, which interact with each other in order to simulate a specific model. Unfortunately, the design of a distributed simulation model is particularly challenging, especially for domain experts who sporadically are computer scientists and are not used to developing parallel code.D-MASON framework is a distributed version of the MASON library for designing and executing ABMs in a distributed environment ensuring scalability and easiness. D-MASON enable the developer to exploit the computing power of distributed environment in a transparent manner; the developer has to do simple incremental modifications to existing MASON models, without re-designing them.This paper presents several novel features and architectural improvements introduced in the D-MASON framework: an improved space partitioning strategy, a distributed 3D field, a distributed network field, a decentralized communication layer, a novel memory consistency mechanism and the integration to cloud environments.Full documentation, additional tutorials, and other material can be found at https://github.com/isislab-unisa/dmason where the framework can be downloaded.     

不确定多智能体互联动态系统分布式鲁棒H∞协同控制

研究一类具有时延和切换拓扑的不确定多智能体互联系统的分布式协同控制问题,提出一类分布式鲁棒H∞协同控制器.该控制器能够在满足期望的H∞性能指标的前提下,使得所有智能体鲁棒地跟踪虚拟Leader.针对所提出的分布式H∞协同控制器,借助Lyapunov-Krasovskii泛函,利用线性矩阵不等式,推导出一系列充分(必要)条件,并且在通信拓扑连通的前提条件下,给出整个闭环系统的稳定性证明.仿真实例表明了所提出方法的有效性.     

Decision-theoretic cooperative sensor planning

This paper describes a decision-theoretic approach to cooperative sensor planning between multiple autonomous vehicles executing a military mission. For this autonomous vehicle application, intelligent cooperative reasoning must be used to select optimal vehicle viewing locations and select optimal camera pan and tilt angles throughout the mission. Decisions are made in such a way as to maximize the value of information gained by the sensors while maintaining vehicle stealth. Because the mission involves multiple vehicles, cooperation can be used to balance the work load and to increase information gain. This paper presents the theoretical foundations of our cooperative sensor planning research and describes the application of these techniques to ARPA's Unmanned Ground Vehicle program