多Agent系统问题求解的广义粒子模型方法

提出一种多Agent系统分布式问题求解的新的广义粒子模型,将复杂环境下多Agent系统资源分配和任务规划的优化问题转变为广义粒子模型中的粒子运动学和动力学问题.广义粒子模型可以描述和处理的复杂环境包括多Agent系统中的Agent之间存在的随机、并发、多类型的交互行为.各Agent有不同的个性、自治性、生命周期、拥塞程度和故障几率等.本文讨论了广义粒子模型和多Agent系统分布式问题求解的关系,提出了广义粒子模型的数学物理模型和多Agent系统分布式问题求解算法,并且证明了它们的正确性、收敛性、稳定平衡性等基本性质.通过复杂环境下多Agent系统资源分配和任务规划问题的实验和比较,证实了广义粒子模型方法的有效性及其特点.     

一种基于P学习的分布式并行多任务分配算法

并行多任务分配是多agent系统中极具挑战性的课题, 主要面向资源分配、灾害应急管理等应用需求, 研究如何把一组待求解任务分配给相应的agent联盟去执行. 本文提出了一种基于自组织、自学习agent的分布式并行多任务分配算法, 该算法引入P学习设计了单agent寻找任务的学习模型, 并给出了agent之间通信和协商策略. 对比实验说明该算法不仅能快速寻找到每个任务的求解联盟, 而且能明确给出联盟中各agent成员的实际资源承担量, 从而可以为实际的控制和决策任务提供有价值的参考依据.     

MASACAD: a multiagent based approach to information customization

MASACAD is a multiagent information customization system that adopts the machine-learning paradigm to advise students by mining the Web. In the distributed problem-solving paradigm, systems can distribute among themselves the processes necessary to accomplish a given task. Given the number of problems that distributed processing can address, AI researchers have directed significant effort toward developing specialized problem-solving systems that can interact in their search for a solution. The multiagent-system paradigm embodies this approach.     

Evolutionary patterns of agent organizations

Problems approached by multi-agent systems are typically complex. It is usually difficult to know at system design stage how many agents need to be in the system, what each agent's role is, and how the agents should interact to get optimal performance out of the group. The aim of the testbed presented here is to investigate which kinds of multi-agent systems could be developed to solve ranges of problems, avoiding the need to reorganize the agents from scratch for each task. The agent organization process explored here is based on the agents' knowledge, and not on their tasks. This opens up a new approach for distributed artificial intelligence designers to have their domain organized before the allocation of tasks among agents. These kinds of organizations should be more robust for solving different problems related to the same knowledge. We define information oriented domains for that purpose. An evolutionary approach to the design of a multi-agent system is suggested. Our model is based on a cellular automaton whose rules of dynamics induce the formation of an organization of agents. Patterns of organization obtained empirically are presented. Our knowledge-based organization approach is analyzed both from theoretical and practical perspectives     

Single neuron computation: from dynamical system to feature detector

White noise methods are a powerful tool for characterizing the computation performed by neural systems. These methods allow one to identify the feature or features that a neural system extracts from a complex input and to determine how these features are combined to drive the system's spiking response. These methods have also been applied to characterize the input-output relations of single neurons driven by synaptic inputs, simulated by direct current injection. To interpret the results of white noise analysis of single neurons, we would like to understand how the obtained feature space of a single neuron maps onto the biophysical properties of the membrane, in particular, the dynamics of ion channels. Here, through analysis of a simple dynamical model neuron, we draw explicit connections between the output of a white noise analysis and the underlying dynamical system. We find that under certain assumptions, the form of the relevant features is well defined by the parameters of the dynamical system. Further, we show that under some conditions, the feature space is spanned by the spike-triggered average and its successive order time derivatives.     

Representations of task assignments in distributed systems using Young tableaux and symmetric groups

This article presents a novel approach to representing task assignments for partitioned agents (respectively, tasks) in distributed systems. A partition of agents (respectively, tasks) is represented by a Young tableau, which is one of the main tools in studying symmetric groups and combinatorics. In this article, we propose a task, agent and assignment tableau in order to represent a task assignment for partitioned agents (respectively, tasks) in a distributed system. This article is concerned with representations of task assignments rather than finding approximate or near optimal solutions for task assignments. A Young tableau approach allows us to raise the expressiveness of partitioned agents (respectively, tasks) and their task assignments.     

A HYBRID ARCHITECTURE FOR DISTRIBUTED AUTONOMOUS CONTROL AGENTS

Abstract

Business environments, like most other real systems, are distributed in nature. However, the fact that the environment itself is distributed does not justify a distributed control architecture. On the other hand, a centralized architecture is not exactly an ideal form of control. The architecture proposed herein is a hybrid control architecture that maps the decision-making environment of many situations ranging from factory floors to traffic control. The architecture assumes the presence of self-reliant (autonomous) decision-making agents, which are required to respond in a certain amount of time. Cooperation between these agents is assumed, and therefore each agent is expected to perform as best as it can in order to complete a well-defined task. No central coordination or control agent is assumed. Thus an asynchronous behavior is assumed.     

A state space analysis of propose-and-revise

A central research topic in the area of knowledge engineering is the reuse of problem-solving methods for developing knowledge based systems. For being able to reuse a problem-solving method it is important to know under which restrictions a problem-solving method is appropriate to solve a given problem. This paper describes the problem-solving method propose-and-revise as well as the way this problem-solving method searches in its problem space for a solution. A quantitative analysis of the efficiency of this search process is given. Additionally, task and domain specific properties and restrictions and their influence on the efficiency of the search process are considered. For these purposes an instance of the problem-solving method is transformed to a corresponding instance of a Stanford Research Institute Problem Solver (STRIPS) planning system. Then the problem-solving method is considered as an additional control strategy for such a planning system. By this way the various insights and analysis results which are available in the area of planning systems may be exploited for the analysis of problem-solving methods. ©1999 John Wiley & Sons, Inc.     

An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games

In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective. Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED) from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms. The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically, we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED. Author funded by a doctoral grant of the institute for advancement of scientific technological research in Flanders (IWT).     

A distributed problem-solving approach to collaborative facility engineering

A framework for collaborative facility engineering is presented. The framework is based on a distributed problem-solving approach to collaborative facility engineering and employs an integration approach called Agent-Based Software Engineering as an implementation vehicle of this approach. The focal entity of this framework is a Multiagent Design Team (MDT) that comprises a collection of software agents (e.g. design software applications with a certain standard communication interface) and a design specialist, which together perform specific design tasks. Multiagent design teams are autonomous and form an organizational structure based on a federation architecture. Every multiagent design team surrenders its autonomy to a system program called facilitator, which coordinates the interaction among software agents in the federation architecture. Facilitators can be viewed as representatives of one or more teams that facilitate the exchange of design information and knowledge in support of the design tasks they perform. In the federation architecture, design specialists collaborate by exchanging design information with others via their software agents, and by identifying and resolving design conflicts by negotiation. In addition to a discussion of the framework's primary components, its realization in an integrated distributed environment for collaborative building engineering is described.     

Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features

In multi-agent reinforcement learning (MARL), the behaviors of each agent can influence the learning of others, and the agents have to search in an exponentially enlarged joint-action space. Hence, it is challenging for the multi-agent teams to explore in the environment. Agents may achieve suboptimal policies and fail to solve some complex tasks. To improve the exploring efficiency as well as the performance of MARL tasks, in this paper, we propose a new approach by transferring the knowledge across tasks. Differently from the traditional MARL algorithms, we first assume that the reward functions can be computed by linear combinations of a shared feature function and a set of task-specific weights. Then, we define a set of basic MARL tasks in the source domain and pre-train them as the basic knowledge for further use. Finally, once the weights for target tasks are available, it will be easier to get a well-performed policy to explore in the target domain. Hence, the learning process of agents for target tasks is speeded up by taking full use of the basic knowledge that was learned previously. We evaluate the proposed algorithm on two challenging MARL tasks: cooperative box-pushing and non-monotonic predator-prey. The experiment results have demonstrated the improved performance compared with state-of-the-art MARL algorithms.      

Fuzzy Constraint-Based Agent Negotiation

Conflicts between two or more parties arise for various reasons and perspectives. Thus, resolution of conflicts frequently relies on some form of negotiation. This paper presents a general problem-solving framework for modeling multi-issue multilateral negotiation using fuzzy constraints. Agent negotiation is formulated as a distributed fuzzy constraint satisfaction problem (DPCSP). Fuzzy constrains are thus used to naturally represent each agent's desires involving imprecision and human conceptualization, particularly when lexical imprecision and subjective matters are concerned. On the other hand, based on fuzzy constraint-based problem-solving, our approach enables an agent not only to systematically relax fuzzy constraints to generate a proposal, but also to employ fuzzy similarity to select the alternative that is subject to its acceptability by the opponents. This task of problem-solving is to reach an agreement that benefits all agents with a high satisfaction degree of fuzzy constraints, and move towards the deal more quickly since their search focuses only on the feasible solution space. An application to multilateral negotiation of a travel planning is provided to demonstrate the usefulness and effectiveness of our framework.     

Contextual Resource Negotiation-Based Task Allocation and Load Balancing in Complex Software Systems

In the complex software systems, software agents always need to negotiate with other agents within their physical and social contexts when they execute tasks. Obviously, the capacity of a software agent to execute tasks is determined by not only itself but also its contextual agents; thus, the number of tasks allocated on an agent should be directly proportional to its self-owned resources as well as its contextual agents' resources. This paper presents a novel task allocation model based on the contextual resource negotiation. In the presented task allocation model, while a task comes to the software system, it is first assigned to a principal agent that has high contextual enrichment factor for the required resources; then, the principal agent will negotiate with its contextual agents to execute the assigned task. However, while multiple tasks come to the software system, it is necessary to make load balancing to avoid overconvergence of tasks at certain agents that are rich of contextual resources. Thus, this paper also presents a novel load balancing method: if there are overlarge number of tasks queued for a certain agent, the capacities of both the agent itself and its contextual agents to accept new tasks will be reduced. Therefore, in this paper, the task allocation and load balancing are implemented according to the contextual resource distribution of agents, which can be well suited for the characteristics of complex software systems; and the presented model can reduce more communication costs between allocated agents than the previous methods based on self-owned resource distribution of agents.     

Automated discovery of concise predictive rules for intrusion detection

This paper details an essential component of a multi-agent distributed knowledge network system for intrusion detection. We describe a distributed intrusion detection architecture, complete with a data warehouse and mobile and stationary agents for distributed problem-solving to facilitate building, monitoring, and analyzing global, spatio-temporal views of intrusions on large distributed systems. An agent for the intrusion detection system, which uses a machine learning approach to automated discovery of concise rules from system call traces, is described.

We use a feature vector representation to describe the system calls executed by privileged processes. The feature vectors are labeled as good or bad depending on whether or not they were executed during an observed attack. A rule learning algorithm is then used to induce rules that can be used to monitor the system and detect potential intrusions. We study the performance of the rule learning algorithm on this task with and without feature subset selection using a genetic algorithm. Feature subset selection is shown to significantly reduce the number of features used while improving the accuracy of predictions.     

Development of A Behavior‐Based Cooperative Search Strategy for Distributed Autonomous Mobile Robots Using Zigbee Wireless Sensor Network

To achieve efficient and objective search tasks in an unknown environment, a cooperative search strategy for distributed autonomous mobile robots is developed using a behavior‐based control framework with individual and group behaviors. The sensing information of each mobile robot activates the individual behaviors to facilitate autonomous search tasks to avoid obstacles. An 802.15.4 ZigBee wireless sensor network then activates the group behaviors that enable cooperative search among the mobile robots. An unknown environment is dynamically divided into several sub‐areas according to the locations and sensing data of the autonomous mobile robots. The group behaviors then enable the distributed autonomous mobile robots to scatter and move in the search environment. The developed cooperative search strategy successfully reduces the search time within the test environments by 22.67% (simulation results) and 31.15% (experimental results).     

A feature selection method for a sample-based stochastic policy

Stochastic policy gradient methods have been applied to a variety of robot control tasks such as robot’s acquisition of motor skills because they have an advantage in learning in high-dimensional and continuous feature spaces by combining some heuristics like motor primitives. However, when we apply one of them to a real-world task, it is difficult to represent the task well by designing the policy function and the feature space due to the lack of enough prior knowledge about the task. In this research, we propose a method to extract a preferred feature space autonomously to achieve a task using a stochastic policy gradient method for a sample-based policy. We apply our method to a control of linear dynamical system and the computer simulation result shows that a desirable controller is obtained and that the performance of the controller is improved by the feature selection.     

Learning in context: enhancing machine learning with context-based reasoning

This article describes how an experiment to train an agent to perform a task, which had originally failed, was made successful by incorporating a contextual structure that decomposed the tasks into contexts through Context-based Reasoning. The task involved a simulation of a crane that was used by a human operator to move boxes from arbitrary locations throughout a wide area to a designated drop off location in the environment. Initial attempts to teach an agent how to perform the task through observation in a context-free manner yielded poor performance. However, when the task to be learned was decomposed into separate contexts and the agents learned each context independently, the performance improved significantly. The paper describes the process that enabled the improvements achieved and discusses the tests and results that demonstrated the improvement.     

Complex Task Allocation in Mobile Surveillance Systems

In mobile surveillance systems, complex task allocation addresses how to optimally assign a set of surveillance tasks to a set of mobile sensing agents to maximize overall expected performance, taking into account the priorities of the tasks and the skill ratings of the mobile sensors. This paper presents a market-based approach to complex task allocation. Complex tasks are the tasks that can be decomposed into subtasks. Both centralized and hierarchical allocations are investigated as winner determination strategies for different levels of allocation and for static and dynamic search tree structures. The objective comparison results show that hierarchical dynamic tree task allocation outperforms all the other techniques especially in complex surveillance operations where large number of robots is used to scan large number of areas.     

Coordination of intelligent agents in real-time search

Search is a fundamental problem-solving method in artificial intelligence. Traditional off-line search algorithms attempt to find an optimal solution whereas real-time search algorithms try to find a suboptimal solution more quickly than traditional algorithms to meet real-time constraints. In this work, a new multi-agent real-time search algorithm is developed and its effectiveness is illustrated on a sample domain, namely maze problems. Searching agents can see their environment with a specified visual depth and hence can partially observe their environment. An agent makes use of its partial observation to select a next move, instead of using only one-move-ahead information. Furthermore agents cooperate through a marking mechanism to be able to search different parts of the search space. When an agent selects its next move, it marks its direction of move before executing the move. When another agent comes to this position, it sees this mark and, if possible, moves in a different direction than the previously selected direction. In this way, marking helps agents coordinate their moves with other agents. Although coordination brings an overhead, from experiments we observe that this mechanism is effective in both search time and solution length in maze problems.     

Developing cooperation mechanism for multi-agent systems with Petri nets

Although the contract net protocol answers some of the questions in cooperative distributed problem solving (CDPS), it raises many others that CDPS researchers are still trying to answer. In contract net protocol, an agent may play the role of a manager or a bidder. Without a coordination mechanism, a manager may acquire excessive resources from the bidders in forming a collaborative network to execute the assigned task and thus hinder the progress of the tasks assigned to other managers due to resource contention. As a result, application of contract net protocol may not always lead to feasible solutions to accomplish tasks effectively. As a general framework for exchanging messages, the original contract net protocol does not prescribe how agents should cooperate. How to develop a collaborative mechanism to effectively perform the tasks is an important issue. This paper aims to improve the insufficiency of the contract net by developing a mechanism to facilitate cooperation of agents to accomplish their tasks while avoiding undesirable states and enhance the overall system performance in manufacturing systems. To achieve these objectives, detail process models about how agents accomplish their tasks are required. Due to the advantages in modeling concurrent, synchronous and/or asynchronous activities, Petri nets are adopted in this paper. Based on Petri net models, we study the information needed for agents to make cooperative decisions, mechanism to make agents cooperate, and how to enhance the performance in the system level by taking advantage of the agents’ cooperation capabilities. To characterize the condition for cooperation, we represent the collaborative networks formed based on the contract net protocol with Petri nets and then find the condition for a collaborative network to be feasible. The feasible condition also serves as a condition for the development of cooperation mechanism for managers. We propose a cooperation mechanism based on the idea of resource donation, including unilateral resource donation and reciprocal resource donation. Implementation architecture has also been proposed to realize our methodology.