Continual planning and acting in dynamic multiagent environments

In order to behave intelligently, artificial agents must be able to deliberatively plan their future actions. Unfortunately, realistic agent environments are usually highly dynamic and only partially observable, which makes planning computationally hard. For most practical purposes this rules out planning techniques that account for all possible contingencies in the planning process. However, many agent environments permit an alternative approach, namely continual planning, i.e. the interleaving of planning with acting and sensing. This paper presents a new principled approach to continual planning that describes why and when an agent should switch between planning and acting. The resulting continual planning algorithm enables agents to deliberately postpone parts of their planning process and instead actively gather missing information that is relevant for the later refinement of the plan. To this end, the algorithm explictly reasons about the knowledge (or lack thereof) of an agent and its sensory capabilities. These concepts are modelled in the planning language (MAPL). Since in many environments the major reason for dynamism is the behaviour of other agents, MAPL can also model multiagent environments, common knowledge among agents, and communicative actions between them. For Continual Planning, MAPL introduces the concept of of assertions, abstract actions that substitute yet unformed subplans. To evaluate our continual planning approach empirically we have developed MAPSIM, a simulation environment that automatically builds multiagent simulations from formal MAPL domains. Thus, agents can not only plan, but also execute their plans, perceive their environment, and interact with each other. Our experiments show that, using continual planning techniques, deliberate action planning can be used efficiently even in complex multiagent environments.     

Real-time path planning in dynamic virtual environments using multiagent navigation graphs

We present a novel approach for efficient path planning and navigation of multiple virtual agents in complex dynamic scenes. We introduce a new data structure, Multi-agent Navigation Graph (MaNG), which is constructed using first- and second-order Voronoi diagrams. The MaNG is used to perform route planning and proximity computations for each agent in real time. Moreover, we use the path information and proximity relationships for local dynamics computation of each agent by extending a social force model [Helbing05]. We compute the MaNG using graphics hardware and present culling techniques to accelerate the computation. We also address undersampling issues and present techniques to improve the accuracy of our algorithm. Our algorithm is used for real-time multi-agent planning in pursuit-evasion, terrain exploration and crowd simulation scenarios consisting of hundreds of moving agents, each with a distinct goal.     

A layered approach to learning coordination knowledge in multiagent environments

Multiagent learning involves acquisition of cooperative behavior among intelligent agents in order to satisfy the joint goals. Reinforcement Learning (RL) is a promising unsupervised machine learning technique inspired from the earlier studies in animal learning. In this paper, we propose a new RL technique called the Two Level Reinforcement Learning with Communication (2LRL) method to provide cooperative action selection in a multiagent environment. In 2LRL, learning takes place in two hierarchical levels; in the first level agents learn to select their target and then they select the action directed to their target in the second level. The agents communicate their perception to their neighbors and use the communication information in their decision-making. We applied 2LRL method in a hunter-prey environment and observed a satisfactory cooperative behavior. Guray Erus received the B.S. degree in computer engineering in 1999, and the M.S. degree in cognitive sciences, in 2002, from Middle East Technical University (METU), Ankara, Turkey. He is currently a teaching and research assistant in Rene“ Descartes University, Paris, France, where he prepares a doctoral dissertation on object detection on satellite images, as a member of the intelligent perception systems group (SIP-CRIP5). His research interests include multi-agent systems and image understanding. Faruk Polat is a professor in the Department of Computer Engineering of Middle East Technical University, Ankara, Turkey. He received his B.Sc. in computer engineering from the Middle East Technical University, Ankara, in 1987 and his M.S. and Ph.D. degrees in computer engineering from Bilkent University, Ankara, in 1989 and 1993, respectively. He conducted research as a visiting NATO science scholar at Computer Science Department of University of Minnesota, Minneapolis in 1992–93. His research interests include artificial intelligence, multi-agent systems and object oriented data models.     

Method of multiagent scheduling of resources in cloud computing environments

The paper describes a method of scheduling distributed computing resources in cloud environments for solving user tasks using a variety of software agents physically located on separate processor units connected to the cloud infrastructure and representing their interests in the process of computing. The advantages of the proposed approach are as follows: Firstly, the fast tracking of all resource changes occurring to the processing unit using agents and real time correction of the computing process taking into account these changes, which in turn makes it possible to use computing resources with a dynamically changing performance in the cloud environment (e.g., personal privately owned computers), and, secondly, a cost reduction for the cloud infrastructure because there is no need to introduce expensive dedicated nodes that perform service functions into its structure.     

Allocating structured tasks in heterogeneous agent teams

Task allocation is an important aspect of multiagent coordination. However, there are many challenges in developing appropriate strategies for multiagent teams so that they operate efficiently. Real‐world scenarios such as flooding disasters usually require the use of heterogeneous robots and the execution of tasks with different structures and complexities. In this paper, we propose a decentralized task allocation mechanism considering different types of tasks for heterogeneous agent teams where agents play different roles and carry out tasks according to their own capabilities. We have run several experiments to evaluate the proposed mechanism. The results show that the proposed mechanism appears to scale well and provides near‐optimal allocations.     

A timed mobile agent planning approach for distributed information retrieval in dynamic network environments

The number of mobile agents and total execution time are two factors used to represent the system overhead that must be considered as part of mobile agent planning (MAP) for distributed information retrieval. In addition to these two factors, the time constraints at the nodes of an information repository must also be taken into account when attempting to improve the quality of information retrieval. In previous studies, MAP approaches could not consider dynamic network conditions, e.g., variable network bandwidth and disconnection, such as are found in peer-to-peer (P2P) computing. For better performance, mobile agents that are more sensitive to network conditions must be used. In this paper, we propose a new MAP approach that we have named Timed Mobile Agent Planning (Tmap). The proposed approach minimizes the number of mobile agents and total execution time while keeping the turnaround time to a minimum, even if some nodes have a time constraint. It also considers dynamic network conditions to reflect the dynamic network condition more accurately. Moreover, we incorporate a security and fault-tolerance mechanism into the planning approach to better adapt it to real network environments.     

Automated Web navigation using multiagent adaptive dynamic programming

Today a massive amount of information available on the WWW often makes searching for information of interest a long and tedious task. Chasing hyperlinks to find relevant information may be daunting. To overcome such a problem, a learning system, cognizant of a user's interests, can be employed to automatically search for and retrieve relevant information by following appropriate hyperlinks. In this paper, we describe the design of such a learning system for automated Web navigation using adaptive dynamic programming methods. To improve the performance of the learning system, we introduce the notion of multiple model-based learning agents operating in parallel, and describe methods for combining their models. Experimental results on the WWW navigation problem are presented to indicate that combining multiple learning agents, relying on user feedback, is a promising direction to improve learning speed in automated WWW navigation.     

Analyzing and visualizing multiagent rewards in dynamic and stochastic domains

The ability to analyze the effectiveness of agent reward structures is critical to the successful design of multiagent learning algorithms. Though final system performance is the best indicator of the suitability of a given reward structure, it is often preferable to analyze the reward properties that lead to good system behavior (i.e., properties promoting coordination among the agents and providing agents with strong signal to noise ratios). This step is particularly helpful in continuous, dynamic, stochastic domains ill-suited to simple table backup schemes commonly used in TD(λ)/Q-learning where the effectiveness of the reward structure is difficult to distinguish from the effectiveness of the chosen learning algorithm. In this paper, we present a new reward evaluation method that provides a visualization of the tradeoff between the level of coordination among the agents and the difficulty of the learning problem each agent faces. This method is independent of the learning algorithm and is only a function of the problem domain and the agents’ reward structure. We use this reward property visualization method to determine an effective reward without performing extensive simulations. We then test this method in both a static and a dynamic multi-rover learning domain where the agents have continuous state spaces and take noisy actions (e.g., the agents’ movement decisions are not always carried out properly). Our results show that in the more difficult dynamic domain, the reward efficiency visualization method provides a two order of magnitude speedup in selecting good rewards, compared to running a full simulation. In addition, this method facilitates the design and analysis of new rewards tailored to the observational limitations of the domain, providing rewards that combine the best properties of traditional rewards.     

RoboCup Rescue as multiagent task allocation among teams: experiments with task interdependencies

This paper addresses distributed task allocation among teams of agents in a RoboCup Rescue scenario. We are primarily concerned with testing different mechanisms that formalize issues underlying implicit coordination among teams of agents. These mechanisms are developed, implemented, and evaluated using two algorithms: Swarm-GAP and LA-DCOP. The latter bases task allocation on a comparison between an agent’s capability to perform a task and the capability demanded by this task. Swarm-GAP is a probabilistic approach in which an agent selects a task using a model inspired by task allocation among social insects. Both algorithms were also compared to another one that allocates tasks in a greedy way. Departing from previous works that tackle task allocation in the rescue scenario only among fire brigades, here we consider the various actors in the RoboCup Rescue, a step forward in the direction of realizing the concept of extreme teams. Tasks are allocated to teams of agents without explicit negotiation and using only local information. Our results show that the performance of Swarm-GAP and LA-DCOP are similar and that they outperform a greedy strategy. Also, it is possible to see that using more sophisticated mechanisms for task selection does pay off in terms of score.     

Game-based e-retailing in GOLEM agent environments

We present a prototype multi-agent system whose goal is to support a 3D application for e-retailing. The prototype demonstrates how the use of agent environments can be amongst the most promising and flexible approaches to engineer e-retailing applications. We illustrate this point by showing how the agent environment GOLEM supports social interactions and how it combines them with semantic-web technologies to develop the e-retailing application. We also describe the features of GOLEM that allow a user to engage in e-retailing activities in order to explore the virtual social environment by searching and dynamically discovering new agents, products and services.     

Implementing information systems with project teams using ethnographic-action research

Architecture, engineering, and construction (AEC) projects are characterized by a large variation in requirements and work routines. Therefore, it is difficult to develop and implement information systems to support projects. To address these challenges, this paper presents a project-centric research and development methodology that combines ethnographic observation of practitioners working in local project organizations to understand their local requirements and the iterative improvement of information systems directly on projects in small action research implementation cycles. The paper shows the practical feasibility of the theoretical methodology using cases from AEC projects in North America and Europe. The cases provide evidence that ethnographic-action research is well suited to support the development and implementation of information systems. In particular, the paper shows that the method enabled researchers on the cases to identify specific problems on AEC projects and, additionally, helped these researchers to adapt information systems accordingly in close collaboration with the practitioners working on these projects.     

Secure multicast in dynamic environments

A secure multicast framework should only allow authorized members of a group to decrypt received messages; usually, one “group key” is shared by all approved members. However, this raises the problem of “one affects all”, whereby the actions of one member affect the whole group. Many researchers have solved the problem by dividing a group into several subgroups, but most current solutions require key distribution centers to coordinate secure data communications between subgroups. We believe this is a constraint on network scalability. In this paper, we propose a novel framework to solve key management problems in multicast networks. Our contribution is threefold: (1) We exploit the ElGamal cryptosystem and propose a technique of key composition. (2) Using key composition with proxy cryptography, the key distribution centers used in secure multicast frameworks are eliminated. (3) For key composition, the framework is designed to resist node failures and support topology reconstruction, which makes it suitable for dynamic network environments. Without reducing the security or performance of proxy cryptography, we successfully eliminate the need for key distribution centers. Our analysis shows that the proposed framework is secure, and comparison with other similar frameworks demonstrates that it is efficient in terms of time and space complexity. In addition, the costs of most protocol operations are bounded by constants, regardless of a group’s size and the number of branches of transit nodes.     

动态环境中的进化算法

目前关于进化算法（EA）的研究主要局限于静态优化问题,然而很多现实世界中的问题是动态的,对于这类时变的优化问题通常并不是要求EA发现极值点,而是需要EA能够尽可能紧密地跟踪极值点在搜索空间内的运行轨迹.为此,综述了使EA适用于动态优化问题的各种方法,如增加种群多样性、保持种群多样性、引入某种记忆策略和采用多种群策略等.     

Fuzzy classification in dynamic environments

The persistence and evolution of systems essentially depend on their adaptivity to new situations. As an expression of intelligence, adaptivity is a distinguishing quality of any system that is able to learn and to adjust itself in a flexible manner to new environmental conditions and such ability ensures self-correction over time as new events happen, new input becomes available, or new operational conditions occur. This requires self-monitoring of the performance in an ever-changing environment. The relevance of adaptivity is established in numerous domains and by versatile real-world applications. The present paper presents an incremental fuzzy rule-based system for classification purposes. Relying on fuzzy min–max neural networks, the paper explains how fuzzy rules can be continuously online generated to meet the requirements of non-stationary dynamic environments, where data arrives over long periods of time. The approach proposed to deal with an ambient intelligence application. The simulation results show its effectiveness in dealing with dynamic situations and its performance when compared with existing approaches.     

Causal maps: theory, implementation, and practical applications in multiagent environments

Analytical techniques are generally inadequate for dealing with causal interrelationships among a set of individual and social concepts. Usually, causal maps are used to cope with this type of interrelationships. However, the classical view of causal maps is based on an intuitive view with ad hoc rules and no precise semantics of the primitive concepts, nor a sound formal treatment of relations between concepts. We solve this problem by proposing a formal model for causal maps with a precise semantics based on relational algebra and the software tool, CM-RELVIEW, in which it has been implemented. Then, we investigate the issue of using this tool in multiagent environments by explaining through different examples how and why this tool is useful for the following aspects: 1) the reasoning on agents' subjective views, 2) the qualitative distributed decision making, and 3) the organization of agents considered as a holistic approach. For each of these aspects, we focus on the computational mechanisms developed within CM-RELVIEW to support it.     

Human-agent teamwork in dynamic environments

Teamwork between humans and computer agents has become increasingly prevalent. This paper presents a behavioral study of fairness and trust in a heterogeneous setting comprising both computer agents and human participants. It investigates people’s choice of teammates and their commitment to their teams in a dynamic environment in which actions occur at a fast pace and decisions are made within tightly constrained time frames, under conditions of uncertainty and partial information. In this setting, participants could form teams by negotiating over the division of a reward for the successful completion of a group task. Participants could also choose to defect from their existing teams in order to join or create other teams. Results show that when people form teams, they offer significantly less reward to agents than they offer to people. The most significant factor affecting people’s decisions whether to defect from their existing teams is the extent to which they had successful previous interactions with other team members. Also, there is no significant difference in people’s rate of defection from agent-led teams as compared to their defection from human-led teams. These results are significant for agent designers and behavioral researchers who study human-agent interactions.     

动态环境中的Memetic算法

针对近几年在进化计算领域被广泛关注的动态优化问题,提出了一种基于粒子群优化(PSO)的Memetic算法.在一种环状拓扑结构的局部PSO模型中,利用模糊认知局域搜索策略来改善部分粒子的质量,同时引入一种自组织随机移民策略来保持算法的种群多样性.通过对一组标准动态测试问题的仿真实验,能够证明所提出的算法在动态环境中的有效性和适应能力.     

Place recognition in dynamic environments

We have developed a technique for place learning and place recognition in dynamic environments. Our technique associates evidence grids with places in the world and uses hill climbing to find the best alignment between current perceptions and learned evidence grids. We present results from five experiments performed using a real mobile robot in a real-world environment. These experiments measured the effects of transient and lasting changes in the environment on the robot's ability to localize. In addition, these experiments tested the robot's ability to recognize places from different viewpoints and verified the scalability of this approach to environments containing large numbers of places. Our results demonstrate that places can be recognized successfully despite significant changes in their appearance, despite the presence of moving obstacles, and despite observing these places from different viewpoints during place learning and place recognition. © 1997 John Wiley & Sons, Inc.     

Implementing dynamic minimal-prefix tries

A modified trie-searching algorithm and corresponding data structure are introduced which permit rapid search of a dictionary for a symbol or a valid abbreviation. The dictionary-insertion algorithm automatically determines disambiguation points, where possible, for each symbol. The search operation will classify a symbol as one of the following: unknown (i.e. not a valid symbol), ambiguous (i.e. is a prefix of more than one valid symbol) or known. The search operation is performed in linear time proportional to the length of the input symbol, rather than the complexity of the trie. An example implementation is given in the C programming language.     

Attitude based teams in a hostile dynamic world

In this paper, we propose a teamwork model based on the concept of a mental attribute called attitude. Our team model presents team as a collective abstract attitude, in which is embedded a novel way of solving problems and conflicts in our domain. We argue that this collective attitude is further decomposed into the individual attitudes of the agents towards various team attributes. We then evaluate the team problem solving behaviours of the agents in a simulated fire world using teams with and without different types of attitudes. The application and implementation of this model to a virtual fire world has revealed a promising prospect in developing team agents.