Distributed learning and cooperative control for multi-agent systems

This paper presents an algorithm and analysis of distributed learning and cooperative control for a multi-agent system so that a global goal of the overall system can be achieved by locally acting agents. We consider a resource-constrained multi-agent system, in which each agent has limited capabilities in terms of sensing, computation, and communication. The proposed algorithm is executed by each agent independently to estimate an unknown field of interest from noisy measurements and to coordinate multiple agents in a distributed manner to discover peaks of the unknown field. Each mobile agent maintains its own local estimate of the field and updates the estimate using collective measurements from itself and nearby agents. Each agent then moves towards peaks of the field using the gradient of its estimated field while avoiding collision and maintaining communication connectivity. The proposed algorithm is based on a recursive spatial estimation of an unknown field. We show that the closed-loop dynamics of the proposed multi-agent system can be transformed into a form of a stochastic approximation algorithm and prove its convergence using Ljung’s ordinary differential equation (ODE) approach. We also present extensive simulation results supporting our theoretical results.     

Distributed cooperative stabilisation of continuous-time uncertain nonlinear multi-agent systems

This paper addresses the distributed cooperative stabilisation problem of continuous-time uncertain nonlinear multi-agent systems. By approximating the uncertain dynamics using neural networks, a distributed adaptive cooperative controller, based on the state information of the neighbouring agents, is proposed. The control design is developed for any undirected connected communication topologies without requiring the accurate model of each agent. This result is further extended to the output feedback case. An observer-based distributed cooperative controller is devised and a parameter dependent Riccati inequality is employed to prove stability of the overall multi-agent systems. This design is less complex than the other design methods and has a favourable decouple property between the observer design and the controller design for uncertain nonlinear multi-agent systems. For both cases, the developed controllers guarantee that all signals in the closed-loop network are uniformly ultimately bounded, and the states of all agents cooperatively converge to a small neighbourhood of origin. A comparative study is given to show the efficacy of the proposed method.     

改进的多agent合作求解过程

在WOOLDRIDGE和张双民的工作基础上,考虑合作提供方的主观愿望,给出一个多agent系统合作求解模型,并考虑合作步骤重复与失败的情况．该模型体现多agent系统的动态属性,使理论与实际更加接近．     

An evidential cooperative multi-agent system

The cooperative learning systems (COLS) are an interesting way of research in Artificial Intelligence. This is because an intelligence form can emerge by interacting simple agents in these systems. In literature, we can find many learning techniques, which can be improved by combining them to a cooperative learning, this one can be considered as a special case of bagging. In particular, learning classifier systems (LCS) are adapted to cooperative learning systems because LCS manipulate rules and, hence, knowledge exchange between agents is facilitated. However, a COLS has to use a combination mechanism in order to aggregate information exchanged between agents, this combination mechanism must take in consideration the nature of information manipulated by the agents. In this paper we investigate a cooperative learning system based on the Evidential Classifier System, the cooperative system uses Dempster–Shafer theory as a support to make data fusion. This is due to the fact that the Evidential Classifier System is itself based on this theory. We present some ways to make cooperation by using this architecture and discuss the characteristics of such architecture by comparing the obtained results with those obtained by an individual approach.     

Severity-sensitive norm-governed multi-agent planning

In making practical decisions, agents are expected to comply with ideals of behaviour, or norms. In reality, it may not be possible for an individual, or a team of agents, to be fully compliant—actual behaviour often differs from the ideal. The question we address in this paper is how we can design agents that act in such a way that they select collective strategies to avoid more critical failures (norm violations), and mitigate the effects of violations that do occur. We model the normative requirements of a system through contrary-to-duty obligations and violation severity levels, and propose a novel multi-agent planning mechanism based on Decentralised POMDPs that uses a qualitative reward function to capture levels of compliance: N-Dec-POMDPs. We develop mechanisms for solving this type of multi-agent planning problem and show, through empirical analysis, that joint policies generated are equally as good as those produced through existing methods but with significant reductions in execution time.     

基于Web服务的多代理协同平台

讨论了基于Web服务的多代理协同平台,可以较好地解决异构CSCW系统之间的组织、调度及通信问题,为虚拟企业环境下的协同技术提供支持。给出了面向Web服务工作流技术和KQML协议的协同平台的实现策略。     

Fault-tolerant cooperative tasking for multi-agent systems

A natural way for cooperative tasking in multi-agent systems is through a top-down design by decomposing a global task into subtasks for each individual agent such that the accomplishments of these subtasks will guarantee the achievement of the global task. In our previous works [Karimadini, M., and Lin, H. (2011c), ‘Guaranteed Global Performance Through Local Coordinations’, Automatica, 47, 890--898; Karimadini, M., and Lin, H. (2011a), ‘Cooperative Tasking for Deterministic Specification Automata’, submitted for publication, online available at: http://arxiv.org/abs/1101.2002], we presented necessary and sufficient conditions on the decomposability of a global task automaton between cooperative agents. As a follow-up work, this article deals with the robustness issues of the proposed top-down design approach with respect to event failures in the multi-agent systems. The main concern under event failure is whether a previously decomposable task can still be achieved collectively by the agents, and if not, we would like to investigate that under what conditions the global task could be robustly accomplished. This is actually the fault-tolerance issue of the top-down design, and the results provide designers with hints on which events are fragile with respect to failures, and whether redundancies are needed. The main objective of this article is to identify necessary and sufficient conditions on failed events under which a decomposable global task can still be achieved successfully. For such a purpose, a notion called passivity is introduced to characterise the type of event failures. The passivity is found to reflect the redundancy of communication links over shared events, based on which necessary and sufficient conditions for the reliability of cooperative tasking under event failures are derived, followed by illustrative examples and remarks for the derived conditions.     

局部合作多智能体Q-学习研究

强化学习在多Agent系统中面对的最大问题就是随着Agent数量的增加而导致的状态和动作空间的指数增长以及随之而来的缓慢的学习效率。采用了一种局部合作的Q-学习方法,只有在Agent之间有明确协作时才考察联合动作,否则,就只进行简单的个体Agent的Q-学习,从而使的学习时所要考察的状态动作对值大大减少。最后算法在捕食者-猎物的追逐问题和机器人足球仿真2D上的实验结果,与常用的多Agent强化学习技术相比有更好的效能。     

A cooperative multi-agent robotics system: Design and modelling

This paper presents the development of the robotic multi-agent system SMART. In this system, the agent concept is applied to both hardware and software entities. Hardware agents are robots, with three and four legs, and an IP-camera that takes images of the scene where the cooperative task is carried out. Hardware agents strongly cooperate with software agents. These latter agents can be classified into image processing, communications, task management and decision making, planning and trajectory generation agents. To model, control and evaluate the performance of cooperative tasks among agents, a kind of Petri Net, called Work-Flow Petri Net, is used. Experimental results shows the good performance of the system.     

Collaborative privacy preserving multi-agent planning

In many cases several entities, such as commercial companies, need to work together towards the achievement of joint goals, while hiding certain private information. To collaborate effectively, some sort of plan is needed to coordinate the different entities. We address the problem of automatically generating such a coordination plan while preserving the agents’ privacy. Maintaining privacy is challenging when planning for multiple agents, especially when tight collaboration is needed and a global high-level view of the plan is required. In this work we present the Greedy Privacy-Preserving Planner (GPPP), a privacy preserving planning algorithm in which the agents collaboratively generate an abstract and approximate global coordination plan and then individually extend the global plan to executable plans. To guide GPPP, we propose two domain independent privacy preserving heuristics based on landmarks and pattern databases, which are classical heuristics for single agent search. These heuristics, called privacy-preserving landmarks and privacy preserving PDBs, are agnostic to the planning algorithm and can be used by other privacy-preserving planning algorithms. Empirically, we demonstrate on benchmark domains the benefits of using these heuristics and the advantage of GPPP over existing privacy preserving planners for the multi-agent STRIPS formalism.     

Distributed optimal consensus of second-order multi-agent systems

Science China Information Sciences -     

Decision-theoretic cooperative sensor planning

This paper describes a decision-theoretic approach to cooperative sensor planning between multiple autonomous vehicles executing a military mission. For this autonomous vehicle application, intelligent cooperative reasoning must be used to select optimal vehicle viewing locations and select optimal camera pan and tilt angles throughout the mission. Decisions are made in such a way as to maximize the value of information gained by the sensors while maintaining vehicle stealth. Because the mission involves multiple vehicles, cooperation can be used to balance the work load and to increase information gain. This paper presents the theoretical foundations of our cooperative sensor planning research and describes the application of these techniques to ARPA's Unmanned Ground Vehicle program     

Distributed suboptimal controller synthesis for multi-agent systems

In this paper, the distributed suboptimal LQR control problem for discrete-time multi-agent systems is considered. The distributed controllers that each subsystem can sense and/or exchange information with only a subset of its neighbors are designed, which are based on the topological structure of the system. Using the “averaged” optimization approach, the suboptimal feedback gain matrices are presented and the distributed suboptimal controllers are obtained. Under mild conditions, the asymptotical convergence of the modified Riccati equation is proved. Accordingly, the infinite-time distributed suboptimal controllers are presented. Examples are given to show the efficiency of the proposed results.     

Distributed algorithms for cooperative control

Expressing a cooperative control system requires combining tools from control theory and distributed computation. After reviewing several possible formalisms appropriate for the job, the authors settle on the computation and control language and illustrate its main features and advantages using a cooperative tracking task. We demonstrate some of the concepts involved in using a multirobot task. We also discuss CCL's ability to be a programming language as well as a modeling tool, which lets us directly simulate or execute CCL models.     

一种基于多主体技术的分布式图像聚类算法

针对医学图像具有对比度较低,不同组织之间的模糊性较高的特点,给出一种基于多主体和数学形态学灰度形态运算的聚类算法。算法采用agent技术和多结构元素结合的模式,用结构元素做智能个体,每个不同类型的agents随机散布在离散空间格点上,在同时刻控制系统驱动下agents根据其自身结构元素的类型用给出的邻域平均算子自主选择作相应的运算进而实现图像聚类。算法无须先验知识和预处理操作,对初始聚类点不敏感,无须事先输入聚类簇数。算法具有分布式并行计算功能和自主分析能力。实验结果验证了该算法的可行性和可靠性。     

Finite-time multi-agent deployment: A nonlinear PDE motion planning approach

The systematic flatness-based motion planning using formal power series and suitable summability methods is considered for the finite-time deployment of multi-agent systems into planar formation profiles along predefined spatial–temporal paths. Thereby, a distributed-parameter setting is proposed, where the collective leader–follower agent dynamics is modeled by two boundary controlled nonlinear time-varying PDEs governing the motion of an agent continuum in the plane. The discretization of the PDE model directly induces a decentralized communication and interconnection structure for the multi-agent system, which is required to achieve the desired spatial–temporal paths and deployment formations.     

面向自主网格的多Agent动态协同图构造算法

网格环境的复杂性和动态性迫切需要自主计算技术的支持。在前期工作中给出地自主网格体系结构基础上,为解决任务执行过程中资源或服务失效情况下多Agent间协同的问题,提出了多Agent动态协同图的概念和任务偏序集驱动地多Agent动态协同图构造算法。图中的顶点是由Agent和自主网格服务组成的序偶,构造算法由任务偏序集到服务集的映射,逐层构造图中的顶点。该图通过Agent对本地服务的感知和Agent间的通信,达到任务执行过程中服务间自主协同的目标。模拟实验的结果验证了算法的正确性,表明算法的时间复杂度主要由任务哈斯图的层数决定,并且Agent的感知时间具有鲁棒性。