Heuristic dynamic programming-based learning control for discrete-time disturbed multi-agent systems

Owing to extensive applications in many fields, the synchronization problem has been widely investigated in multi-agent systems. The synchronization for multi-agent systems is a pivotal issue, which means that under the designed control policy, the output of systems or the state of each agent can be consistent with the leader. The purpose of this paper is to investigate a heuristic dynamic programming (HDP)-based learning tracking control for discrete-time multi-agent systems to achieve synchronization while considering disturbances in systems. Besides, due to the difficulty of solving the coupled Hamilton– Jacobi–Bellman equation analytically, an improved HDP learning control algorithm is proposed to realize the synchronization between the leader and all following agents, which is executed by an action-critic neural network. The action and critic neural network are utilized to learn the optimal control policy and cost function, respectively, by means of introducing an auxiliary action network. Finally, two numerical examples and a practical application of mobile robots are presented to demonstrate the control performance of the HDP-based learning control algorithm.     

Global behavior of dynamical agents in directed network

This paper investigates the global behavior of controlled dynamical agents in directed networks. The agents are Lyapunov stable, are distributed in a line, and communicate through a directed network. The communication topology of the network is characterized by a directed graph and the control protocol is designed in simple linear decentralized feedback law. We study the different conditions under which agents will achieve aggregation, and critical and divergent trajectories, respectively. Our investigation on the dynamical agent system under network is extended to the time-delay network case. Furthermore, we study the case with two pre-specified virtual leaders in the system. Numerical simulations are given and demonstrate that our theoretical results are effective.     

MADIDS：一种基于移动代理的新型分布式入侵检测系统

When traditional Intrusion Detection System(IDS)is used to detect and analyze the great flow data transfer in high-speed network,it usually causes the computation bottleneck.This paper presents a new Mobile Agent Distributed IDS(MADIDS) system based on the mobile agents.This system is specifically designed to process the great flow data transfer in high-speed network.In MADIDS,the agents that are set at each node process the data transfer by distributed computation architecture.Meanwhile by using the reconfiguration quality of the mobile agents,the load balance of distributed computation can be dynamically implemented to gain the high-performance computing ability.This ability makes the detecting and analyzing of high-speed network possible.MADIDS can effectively solve the detection and analysis performance bottleneck caused by the great flow data transfer in high-speed network.It enhances the performance of IDS in high-speed network.In this paper,we construct the infrastructure and theoretical model of MADIDS,and the deficiencies of MADIDS and future research work are also indicated.     

Multiagent flocking with formation in a constrained environment

We consider the problem of controlling a group of mobile agents to form a designated formation while flocking within a constrained environment. We first propose a potential field based method to drive the agents to move in connection with their neighbors, and regulate their relative positions to achieve the specific formation. The communication topology is preserved during the motion. We then extend the method to flocking with environmental constraints. Stability properties are analyzed to guarantee that all agents eventually form the desired formation while flocking, and flock safely without collision with the environment boundary. We verify our algorithm through simulations on a group of agents performing maximum coverage flocking and traveling through an unknown constrained environment.     

无线传感网络中使用动态代理的节点收敛算法

This paper proposes fault tolerant algorithms for routing mobile agents in a single channel wireless sensor network which cover every node in the network. These algorithms use local knowledge (assume no knowledge of global deployment or topology, etc). We propose the algorithms and show mathematical analysis to support our claims. The paper ends with simulation studies and discussion of results.     

Methodological Approaches to Development and Implementation of Multi-agents Investment DSS Using JADE Platform

In these latter days software agents are used for the development and implementation of intellectual decision support systems. In order to implement intelligence in a system some or several dozen of software agents are used and the made system becomes multi-agent. For the development of these systems a set of methodologies, i.e., the sequence of consequent steps of analysis, designing and implementation, is offered. The carried out analysis of the methodologies showed that as a rule they are limited by the spectrum of their pending problem （within the pales of the requirements of specific applied task, within the pales of the possibilities of technical implementation） or within the pales of amount of detail. The variety of methodologies is influenced by the fact that for the development of these systems the requirements and attitudes are offered by the specialists of related spheres such as software, numeral intellect engineers. In the course of the development of hardware and software appeared possibilities to implement mobile multi-agents systems, however, there is no one united mobile multi-agent systems design methodology, whereas existing systems are underdeveloped and their number is small. In this article we introduce the course of the designing of an intellectual real time multi-agent investment management decision support information system adapting and combining some methodologies where the choice to use either communicating or mobile agents is the question of rather technical implementation than methodological. In the article we introduce two ways of system implementation by JADE platform： the first one-using communicating agents, and the second one-using mobile agents.     

Consensus control of feedforward nonlinear multi-agent systems:a time-varying gain method

In this paper,the leader–follower consensus of feedforward nonlinear multi-agent systems is achieved by designing the distributed output feedback controllers with a time-varying gain.The agents dynamics are assumed to be in upper triangular structure and satisfy Lipschitz conditions with an unknown constant multiplied by a time-varying function.A time-varying gain,which increases monotonously and tends to infinity,is proposed to construct a compensator for each follower agent.Based on a directed communication topology,the distributed output feedback controller with a time-varying gain is designed for each follower agent by only using the output information of the follower and its neighbors.It is proved by the Lyapunov theorem that the leader–follower consensus of the multi-agent system is achieved by the proposed consensus protocol.The effectiveness of the proposed time-varying gain method is demonstrated by a circuit system.     

Active network supports for mobile IP

The basic mobile IP protocol is difficult to implement on the traditional IP network and not flexible and efficient under certain conditions.For example,firewalls or boundary routers may drop packets sent by mobile nodes for security reasons.Traditional networking devices such as routers cannot dynamically load and unload extended services to achieve different qualities of services.In this paper,a new scheme of using the active network to support the mobile IP is presented.The Softnet,a prototype of active networks based on mobile agents,is introduced.The active network is characterized by the programmability of its intermediate nodes and therefore presents dynaic and flexible behaviors.Special services can be dynamically deployed onto the active nodes in the Softnet.This property is definitely required in implementing the mobile IP protocols.The SOftnet.This property is definitely required in implementing the mobile IP protocols.The Softnet.This property is definitely required in implementing the mobile IP protocols.The Softnet supports not only the basic mobile IP protocol but also other extended mobile IP protocols.Virtual networks for mobile IP services are dynamically formed by mobile agents in the Softnet to provide different qualities of services.     

Consensus control of feedforward nonlinear multi-agent systems: A time-varying gain method

In this paper, the leader–follower consensus of feedforward nonlinear multi-agent systems is achieved by designing the distributed output feedback controllers with a time-varying gain. The agents dynamics are assumed to be in upper triangular structure and satisfy Lipschitz conditions with an unknown constant multiplied by a time-varying function. A time-varying gain, which increases monotonously and tends to infinity, is proposed to construct a compensator for each follower agent. Based on a directed communication topology, the distributed output feedback controller with a time-varying gain is designed for each follower agent by only using the output information of the follower and its neighbors. It is proved by the Lyapunov theorem that the leader–follower consensus of the multi-agent system is achieved by the proposed consensus protocol. The effectiveness of the proposed time-varying gain method is demonstrated by a circuit system.     

Dynamic consensus of high-order multi-agent systems and its application in the motion control of multiple mobile robots

In this paper, the leader-following consensus problem for multi-agent linear dynamic systems is considered. All agents and leader have identical multi-input multi-output (MIMO) linear dynamics that can be of any order, and only the output information of each agent is delivered throughout the communication network. When the interaction topology is fixed, the leader-following consensus is attained by H∞ dynamic output feedback control, and the sufficient condition of robust controllers is equal to the solvability of linear matrix inequality (LMI). The whole analysis is based on spectral decomposition and an equivalent decoupled structure achieved, and the stability of the system is proved. Finally, we extended the theoretical results to the case that the interaction topology is switching. The simulation results for multiple mobile robots show the effectiveness of the devised methods.     

基于有向网络的智能群体群集运动控制

对具有二次积分动态的智能群体(flock/swarm),在有向网络取得群集运动(flocking/swarming)进行了研究．提出了一个分散控制方法对智能群体进行分散控制．用有向图模型表示智能体之间的相互作用及通信关系．对固定的网络拓扑,控制互连拓扑是固定的,时不变的,运用传统的LaSalle不变集原理,代数图论的有关技巧进行了稳定性分析,并得到以下主要结论:i)智能群体速度方向渐进收敛,并保持方向一致;ii)智能群体速度大小渐进收敛,并保持大小相等;iii)有邻接关系的智能体(Agent)之间没有碰撞发生;iv)智能群体的势场函数被最小化．理论分析显示,有向图的弱连通性及一种称为平衡图的有向图在系统的稳定性分析中扮演着关键角色．最后,给出了一个仿真例子对理论结果进行了验证．     

带强连通分支的多智能体系统可控包含控制

研究有强连通子图拓扑结构的有向多智能体系统领导者选择及可控包含控制问题.根据网络拓扑结构,智能体被分为两类:单元智能体和一般智能体.首先设计强连通子图中个体组成的单元智能体的一致性协议实现各个单元的一致;后由单元智能体和一般智能体构成新的拓扑结构,结合复杂网络可控性理论与二分图最大匹配算法确定满足网络可控的最少领导者集合,并为所有智能个体设计相应的控制协议,驱使跟随者渐近收敛到多个领导者组成的动态凸包中,从而实现网络的可控包含控制.仿真结果验证了理论分析的正确性.     

Adaptive second-order consensus of networked mobile agents with nonlinear dynamics

We investigate second-order consensus of multiple nonlinear dynamical mobile agents with a virtual leader in a dynamic proximity network. We assume that only a small fraction of agents in the group have access to the information about the position and velocity of the virtual leader through, for example, certain pre-designed communication mechanism such as wireless broadcasting or sensing. To avoid fragmentation, we propose a connectivity-preserving second-order consensus algorithm. Under the assumption that the initial network is connected, we introduce local adaptation strategies for both the weights on the velocity navigational feedback and the velocity coupling strengths that enable all agents to synchronize with the virtual leader even when only one agent is informed, without requiring any knowledge of the agent dynamics. We finally provide some convincing simulation results to illustrate the theoretical results.     

Flocking in Fixed and Switching Networks

This note analyzes the stability properties of a group of mobile agents that align their velocity vectors, and stabilize their inter-agent distances, using decentralized, nearest-neighbor interaction rules, exchanging information over networks that change arbitrarily (no dwell time between consecutive switches). These changes introduce discontinuities in the agent control laws. To accommodate for arbitrary switching in the topology of the network of agent interactions we employ nonsmooth analysis. The main result is that regardless of switching, convergence to a common velocity vector and stabilization of inter-agent distances is still guaranteed as long as the network remains connected at all times     

Leader election for anonymous asynchronous agents in arbitrary networks

We consider the problem of leader election among mobile agents operating in an arbitrary network modeled as an undirected graph. Nodes of the network are unlabeled and all agents are identical. Hence the only way to elect a leader among agents is by exploiting asymmetries in their initial positions in the graph. Agents do not know the graph or their positions in it, hence they must gain this knowledge by navigating in the graph and share it with other agents to accomplish leader election. This can be done using meetings of agents, which is difficult because of their asynchronous nature: an adversary has total control over the speed of agents. When can a leader be elected in this adversarial scenario and how to do it? We give a complete answer to this question by characterizing all initial configurations for which leader election is possible and by constructing an algorithm that accomplishes leader election for all configurations for which this can be done.     

Controllability of a Leader–Follower Dynamic Network With Switching Topology

This note studies the controllability of a leader-follower network of dynamic agents linked via neighbor rules. The leader is a particular agent acting as an external input to steer the other member agents. Based on switched control system theory, we derive a simple controllability condition for the network with switching topology, which indicates that the controllability of the whole network does not need to rely on that of the network for every specific topology. This merit provides convenience and flexibility in design and application of multiagent networks. For the fixed topology case, we show that the network is uncontrollable whenever the leader has an unbiased action on every member, regardless of the connectivity of the members themselves. This gives new insight into the relation between the controllability and the connectivity of the leader-follower network. We also give a formula for formation control of the network.     

A flocking-based approach to maintain connectivity in mobile wireless ad hoc networks

Mobile ad hoc networks (MANET) have a set of unique challenges, particularly due to mobility of nodes, that need to be addressed to realize their full potentials. Because the mobile nodes of a MANET are free to move rapidly and arbitrarily, the network topology may change unexpectedly. This paper presents a decentralized approach to maintain the connectivity of a MANET using autonomous, intelligent agents. Autonomous agents are special mobile nodes in a MANET, but unlike other nodes, their function is to proactively prevent network bottlenecks and service problems by intelligently augmenting the network topology. To achieve this function without depending on a central network management system, autonomous agents are expected to dynamically relocate themselves as the topology of the network changes during the mission time. A flocking-based heuristic algorithm is proposed to determine agent locations. A computational study is performed to investigate the effect of basic flocking behaviors on the connectivity of a MANET.     

多智能体系统自适应跟踪控制

基于带有非线性动态的二阶多智能体系统,研究了在有动态领导者条件下的跟踪一致性问题。假设跟随者只能获取邻居智能体的相对状态信息,只有一部分跟随者可以获得领导者的位置和速度信息,领导者的控制输入非零且不被任何一个跟随者可知。在通信拓扑为无向连通图的条件下,为了避免全局信息的不确定性,设计了分布式自适应控制协议。将系统的一致性问题转化为误差系统的一致性问题,通过Lyapunov稳定性理论和矩阵理论分析得到了该协议使系统达到一致的充分条件。最后用仿真例子证明了设计方法的有效性。     

Contiguous Search Problem in Sierpiński Graphs

In this paper we consider the problem of capturing an intruder in a particular fractal graph, the Sierpiński graph SG _n . The problem consists of having a team of mobile software agents that collaborate in order to capture the intruder. The intruder is a mobile entity that escapes from the team of agents, moving arbitrarily fast inside the network, i.e., traversing any number of contiguous nodes as long as no other agent resides on them. The agents move asynchronously and they know the network topology they are in is a Sierpiński graph SG _n . We first derive lower bounds on the minimum number of agents, number of moves and time steps required to capture the intruder. We then consider some variations of the model based on the capabilities of the agents: visibility, where the agents can “see” the state of their neighbors and thus can move autonomously; locality, where the agents can only access local information and thus their moves have to be coordinated by a leader. For each model, we design a capturing strategy and we make some observations. One of our goals is to continue a previous study on what is the impact of visibility on complexity: in this topology we are able to reach an optimal bound on the number of agents required by both cleaning strategies. However, the strategy in the visibility model is fully distributed, whereas the other strategy requires a leader. Moreover, the second strategy requires a higher number of moves and time steps. A preliminary version of this paper has been presented at the 4th International Conference on Fun with Algorithms (FUN’07) 17.     

Maximizing Convergence Speed for Second Order Consensus in Leaderless Multi-Agent Systems

The paper deals with the consensus problem in a leaderless network of agents that have to reach a common velocity while forming a uniformly spaced string. Moreover, the final common velocity (reference velocity) is determined by the agents in a distributed and leaderless way. Then, the consensus protocol parameters are optimized for networks characterized by a communication topology described by a class of directed graphs having a directed spanning tree, in order to maximize the convergence rate and avoid oscillations. The advantages of the optimized consensus protocol are enlightened by some simulation results and comparison with a protocol proposed in the related literature. The presented protocol can be applied to coordinate agents such as mobile robots, automated guided vehicles (AGVs) and autonomous vehicles that have to move with the same velocity and a common inter-space gap.