Design principles for autonomous agents: A case study of classification

Building artificial lifelike autonomous agents is still considered an art, rather than a science. A generally accepted precise methodology is missing, and—given the properties of the real world—it is doubtful whether such a methodology will ever be developed. Nevertheless, it is possible to define criteria and provide heuristics for good designs. We have developed a number of design principles which, when applied, lead to what we would consider good designs from a cognitive science or artificial life (ALife) perspective. The paper illustrates some of these principles using a case study of classification. Presented at the Internatial Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

The metacognitive loop I: Enhancing reinforcement learning with metacognitive monitoring and control for improved perturbation tolerance

Maintaining adequate performance in dynamic and uncertain settings has been a perennial stumbling block for intelligent systems. Nevertheless, any system intended for real-world deployment must be able to accommodate unexpected change—that is, it must be perturbation tolerant. We have found that metacognitive monitoring and control—the ability of a system to self-monitor its own decision-making processes and ongoing performance, and to make targeted changes to its beliefs and action-determining components—can play an important role in helping intelligent systems cope with the perturbations that are the inevitable result of real-world deployment. In this article we present the results of several experiments demonstrating the efficacy of metacognition in improving the perturbation tolerance of reinforcement learners, and discuss a general theory of metacognitive monitoring and control, in a form we call the metacognitive loop.     

The sensory ego-sphere: a mediating interface between sensors and cognition

The Sensory Ego-Sphere (SES) is an interface for a robot that serves to mediate information between sensors and cognition. The SES can be visualized as a sphere centered on the coordinate frame of the robot, spatially indexed by polar and azimuthal angles. Internally, the SES is a graph with a fixed number of edges that partitions surrounding space and contains localized sensor information from the robot. This paper describes the SES and gives the results of implementing the SES on multiple robots, both humanoid and mobile, to support essential functions such as a localized short-term memory, spatio-temporal sensory-motor event detection, attentional processing, data sharing, and ego-centric navigation. This research was supported in part by DARPA-IPTO grant DASG60-99-1-0005, and NASA-JSC grants NAG9-1428, NAG9-1446 and NAG9-1515.     

The Simplicity and Power model for inductive inference

With this paper we wish to present a simplicity (informally ‘simple explanations are the best’) formalism that is easily and directly applicable to modeling problems in cognitive science. While simplicity has been extensively advocated as a psychologically relevant principle, a general modeling formalism has been lacking. The Simplicity and Power model (SP) is a particular simplicity-based framework, that has been supported in machine learning (Wolff, Unifying computing and cognition: the SP theory and its applications, 2006). We propose its utility in cognitive modeling. For illustration, we provide SP demonstrations of the trade-off between encoding with whole exemplars versus parts of stimuli in learning and the effect of wide versus narrow distributions in categorization. In both cases, SP computations show how simplicity can account for these contrasts, in terms of how the frequency of individual exemplars in training compares to the frequency of their constituent parts.     

The coordination generalized particle model—An evolutionary approach to multi-sensor fusion

The rising popularity of multi-source, multi-sensor networks supporting real-life applications calls for an efficient and intelligent approach to information fusion. Traditional optimization techniques often fail to meet the demands. The evolutionary approach provides a valuable alternative due to its inherent parallel nature and its ability to deal with difficult problems. We present a new evolutionary approach based on the coordination generalized particle model (C-GPM) which is founded on the laws of physics. C-GPM treats sensors in the network as distributed intelligent agents with various degrees of autonomy. Existing approaches based on intelligent agents cannot completely answer the question of how their agents could coordinate their decisions in a complex environment. The proposed C-GPM approach can model the autonomy of as well as the social coordinations and interactive behaviors among sensors in a decentralized paradigm. Although the other existing evolutionary algorithms have their respective advantages, they may not be able to capture the entire dynamics inherent in the problem, especially those that are high-dimensional, highly nonlinear, and random. The C-GPM approach can overcome such limitations. We develop the C-GPM approach as a physics-based evolutionary approach that can describe such complex behaviors and dynamics of multiple sensors.     

FroggyBobby: An exergame to support children with motor problems practicing motor coordination exercises during therapeutic interventions

Children with motor coordination problems exhibit impairments in gross-motor eye-body coordination. Exergames are good in supporting motor therapeutic interventions. However, available exergames are not designed to support the needs of these children, presenting excessive cognitive load and suggesting practice of inappropriate body interactions. In this work, we investigate the design of exergames to support children with motor problems during gross-motor therapeutic interventions. We followed an iterative user-centered design methodology to design FroggyBobby – an exergame to support children with motor problems when practicing eye-body coordination exercises. Our design involved two qualitative studies: a study to understand the problem space, and a study to validate an initial design of FroggyBobby. Additionally, we held participatory design sessions to uncover the gameful mechanisms and appropriate motor coordination exercises to support motor therapeutic interventions. To evaluate FroggyBobby, we conducted a deployment study with children with low-functioning autism presenting motor problems and psychologist-teachers at a school-clinic for autism. FroggyBobby was found easy to use, fun, and engaging by children with autism and teachers. Psychologist-teachers perceived FroggyBobby impacted motor functioning, socialization and body awareness, and successfully supported motor therapeutic interventions. We close reflecting how our results present new insights from a design, technical, and therapeutic point of view.     

Implicit commitments: Heterogeneous agent coordination for retrieval and assembly of distributed image data

This paper describes a multiple agent system for distributed planning, specifically for intelligent network information retrieval. The system consists of a set of reactive agents located at each information- or capability-provider site plus an agent to track each information request. The agents negotiate with each other to assemble a distributed plan for satisfaction of each information need and each agent also interfaces with local retrieval and data processing functions. The idea of an implicit commitment is presented as providing a way to enable automated coordination of resources (information and capabilities) localized among heterogeneous agents. The advantages of a shared machine implemented by such a commitment protocol are discussed and an implementation called TRACS for retrieval and assembly of satellite image data is presented.This research was carried out at Teleos Research Corporation under a contract with Lockheed Space and Missiles Corporation. I would like to thank Stanley Rosenschein, Marianne Winslett, and two anonymous reviewers for their comments and suggestions.     

MESHMdl event spaces — A coordination middleware for self-organizing applications in ad hoc networks

Mobile ad hoc networks (MANETs) are gaining importance as a promising technology for flexible, proximity-based, mobile communication. However, the inherent dynamics of MANETs imposes strong limitations on the design of distributed applications. They need to be able to adapt to changing conditions quickly and organize themselves in terms of component placement and communication habits. In this paper, we present MESHMdl, a middleware that provides a high level of awareness and decoupling for application components to make them more flexible and adaptable. We focus on the Event Space as the central communication medium of MESHMdl. The Event Space offers a simple, unified communication interface for inter-agent communication as well as for communication with the middleware and resource access. Furthermore, it serves as a means for flexibly extending a MESHMdl daemon. We investigate the performance of the Event Space on different mobile devices and show that it is superior to comparable systems.     

Distributed Constraint Optimization Problems: Review and perspectives

Intelligent agents is a research area of the Artificial Intelligence intensely studied since the 1980s. Multi-agent systems represent a powerful paradigm of analyzing, projecting, and developing complex systems. One of the main difficulties in modeling a multi-agent system is defining the coordination model, due to the autonomous behavior of the agents. Distributed Constraint Optimization Problems (DCOP) have emerged as one of most important formalisms for coordination and distributed problem solving in multi-agent systems and are capable of modeling a large class of real world problems naturally. This work aims to provide an overview and critical review of DCOP, addressing the most popular methods and techniques, the evolution and comparison of algorithms, and future perspectives on this promising research area.     

The coordination of work activities: Cooperation and conflict in a hospital context

An understanding of the ways in which work coordination is achieved in practice is essential to the development of effective CSCW technologies. However, previous studies are limited in their focus on small, self-contained work groups. In this analysis of work coordination in a hospital context, a broader perspective was adopted, allowing examination of activities across time, group and location. The use of a relevant structured methodology and a focus on deviations from formal procedures enabled the consideration of a range of contextual factors in interaction: Important aspects of work coordination to emerge included: status influences on the effectiveness of working practices; the social and political uses of information; conflicts between work goals and between motivations for coordinating activities; the role of informal practices; and the use of formal procedures to regulate inter-group relations. The implications of these issues for CSCW design in the hospital context are illustrated.     

Three and higher dimensional autonomous formations: Rigidity, persistence and structural persistence

In this paper, we generalize the notion of persistence, which has been originally introduced for two-dimensional formations, to R^d for d?3, seeking to provide a theoretical framework for real world applications, which often are in three-dimensional space as opposed to the plane. Persistence captures the desirable property that a formation moves as a cohesive whole when certain agents maintain their distances from certain other agents. We verify that many of the properties of rigid and/or persistent formations established in R² are also valid for higher dimensions. Analysing the closed subgraphs and directed paths in persistent graphs, we derive some further properties of persistent formations. We also provide an easily checkable necessary condition for persistence. We then turn our attention to consider some practical issues raised in multi-agent formation control in three-dimensional space. We display a new phenomenon, not present in R², whereby subsets of agents can behave in a problematic way. When this behaviour is precluded, we say that the graph depicting the multi-agent formation has structural persistence. In real deployment of controlled multi-agent systems, formations with underlying structurally persistent graphs are of interest. We analyse the characteristics of structurally persistent graphs and provide a streamlined test for structural persistence. We study the connections between the allocation of degrees of freedom (DOFs) across agents and the characteristics of persistence and/or structural persistence of a directed graph. We also show how to transfer DOFs among agents, when the formation changes with new agent(s) added, to preserve persistence and/or structural persistence.     

ANGLE: An autonomous, normative and guidable agent with changing knowledge

Some emerging computing systems (especially autonomic computing systems) raise several challenges to autonomous agents, including (1) how to reflect the dynamics of business requirements, (2) how to coordinate with external agents with sufficient level of security and predictability, and (3) how to perform reasoning with dynamic and incomplete knowledge, including both informational knowledge (observations) and motivational knowledge (for example, policy rules and contract rules). On the basis of defeasible logic and argumentation, this paper proposes an autonomous, normative and guidable agent model, called ANGLE, to cope with these challenges. This agent is established by combining beliefs-desires-intentions (BDI) architecture with policy-based method and the mechanism of contract-based coordination. Its architecture, knowledge representation, as well as reasoning and decision-making, are presented in this paper. ANGLE is characteristic of the following three aspects. First, both its motivational knowledge and informational knowledge are changeable, and allowed to be incomplete, inconsistent/conflicting. Second, its knowledge is represented in terms of extended defeasible logic with modal operators. Different from the existing defeasible theories, its theories (including belief theory, goal theory and intention theory) are dynamic (called dynamic theories), reflecting the variations of observations and external motivational knowledge. Third, its reasoning and decision-making are based on argumentation. Due to the dynamics of underlying theories, argument construction is not a monotonic process, which is different from the existing argumentation framework where arguments are constructed incrementally.     

The dynamics of adaptive behavior: A research program

This paper provides a comprehensive overview of the motivations, methodology and current status of an ongoing research program whose long-term goal is to elucidate the essential principles of a theory of adaptive behavior. The thoroughly dynamical nature of both adaptive behavior itself and the causal mechanisms that support it is emphasized throughout. An initial mapping of the basic concepts of adaptive behavior into the language of dynamical systems theory is proposed, and some of the general consequences of this preliminary theoretical framework are discussed. The two key ideas of this framework are (1) that an agent and its environment should be understood as two coupled dynamical systems whose mutual interaction is jointly responsible for the agent's behavior, and (2) that an agent's need to maintain its existence in its environment defines a viability constraint on its behavioral dynamics. A constructive research methodology involving the use of evolutionary algorithms to evolve continuous-time recurrent neural networks for controlling the behavior of model agents is described, and several examples of this methodology are presented, including models of chemotaxis, walking, sequential decision-making and learning. Finally, a detailed dynamical analysis of one evolved walking circuit is presented. This analysis illustrates the kinds of insights that can be obtained by treating agents as dynamical systems and applying the tools of dynamical systems theory to their behavior.     

Awareness and handoffs in home care: coordination among informal caregivers

Informal caregivers, such as family members and friends, are important for maintaining the health and wellness of the elderly and the chronically ill. However, the complexity of informal caregiving has been under-studied in prior research. In this study, we attempt to answer two main questions – what makes the coordination of patient care so difficult among multiple informal caregivers, and what are the challenges that should be considered in designing technologies to facilitate informal care coordination? To answer these questions, we conducted a qualitative interview study with 15 individuals with caregiving experience. The findings of the study identified the main challenges faced by caregivers in maintaining awareness, coordinating handoffs among informal caregivers, and easing tensions among caregivers and patients. These findings revealed considerations unique for informal caregiver coordination, and helped inform the design of technologies to support the care coordination.     

A revised sales rebate contract with effort‐dependent demand: a channel coordination approach

In this paper, simultaneous coordination of order quantity and sales effort (SE) decisions in a supplier/retailer system with stochastic effort‐dependent demand is investigated. The main aim of the proposed model is to attain an optimal balance that results in a Pareto‐efficient solution for both channel members. A revised sales rebate (RSR) contract is developed to achieve channel coordination. In addition to the usual incentive approach of sales rebate schemes, a punitive approach is designed for the new proposed contract as a stockout penalty. Furthermore, some numerical experiments are examined to analyze the performance of the presented model under three decision‐making scenarios (i.e., decentralized, centralized, and RSR). Additionally, some in‐depth sensitivity analyses are conducted to examine the behavior of the supply chain performance under alteration of different parameters. The results show that the proposed RSR contract leads to channel coordination, while both channel members experience a Pareto improving situation. Moreover, it is proved that the RSR contract has significant potential on neutralizing adverse impacts of demand fluctuations on channel performance indicators.     

Agents for information retrieval: Issues of mobility and coordination

This paper focuses on agent-based applications for information retrieval on the Web, by specifically analysing mobility and coordination issues. On the one hand, mobile agents well suit the requirements of information retrieval in the new dynamic scenario derived from the Internet. This is due to their capability of moving to the place where the information is stored – therefore saving bandwidth – and to their robustness in the presence of unreliable connections. On the other hand, the search for information by several mobile active agents calls for suitable models to rule the interactions among agents and between agents and execution environments. The paper surveys different coordination approaches and evaluates their impact in information retrieval applications based on mobile agents. The survey outlines the advantages of uncoupled coordination models and points out the suitability of a coordination model based on reactive and programmable tuple spaces: they may increase the safety and the security of the environment while simplifying the task of programming distributed mobile agent applications.     

The Iterated Prisoner's Dilemma: early experiences with Learning Classifier System-based simple agents

Prior research on artificial agents/agencies involves entities using specifically tailored operational strategies (e.g., for information retrieval, purchase negotiation). In some situations, however, an agent must interact with others whose strategies are initially unknown and whose interests may counter its own. In such circumstances, pre-defining effective counter-strategies could become difficult or impractical. One solution, which may be viable in certain contexts, is to create agents that self-evolve increasingly effective strategies from rudimentary beginnings, during actual deployment. Using the Iterated Prisoner's Dilemma (IPD) problem as a generic agent-interaction setting, we use the Learning Classifier System (LCS) paradigm to construct autonomously adapting “simple” agents. A simple agent attempts to cope by maintaining an evolving but potentially perennially incomplete and imperfect knowledge base. These agents operate against specifically tailored (non-adaptive) agents. We present a preliminary suite of simulation experiments and results. The promise evidenced leads us to articulate several additional areas of interesting investigations that we are pursuing.     

Emergency department status boards: user-evolved artefacts for inter- and intra-group coordination

Large, easily viewed status boards are commonly used in some healthcare settings such as emergency departments, operating theaters, intensive care units, and inpatient wards. Because these artefacts were developed by front-line users, and have little to no supervisory or regulatory control, they offer valuable insights into the theories of work and hazard held by those users. Although the status boards case were locally developed over many years for within-group coordination, they have also become useful for between-group coordination across organizational boundaries. In this paper, we compare and contrast the use of such status boards in two disparate settings: a US emergency department, and a UK pediatric ward, and note striking similarities in their form and usage, despite the large differences in setting.     

Cooperation and Competition in ALIAS: A Logic Framework for Agents that Negotiate

This paper presents ALIAS, an agent architecture based on intelligent logic agents, where the main form of agent reasoning is abduction. The system is particularly suited for solving problems where knowledge is incomplete, where agents may need to make reasonable hypotheses about the problem domain and other agents, and where the raised hypotheses have to be consistent for the overall set of agents. ALIAS agents are pro-active, exhibiting a goal-directed behavior, and autonomous, since each one can solve problems using its own private knowledge base. ALIAS agents are also social, because they are able to interact with other agents, in order to cooperatively solve problems. The coordination mechanisms are modeled by means of LAILA, a logic-based language which allows to express intra-agent reasoning and inter-agent coordination. As an application, we show how LAILA can be used to implement inter-agent dialogues, e.g., for negotiation. In particular, LAILA is well-suited to coordinate the process of negotiation aimed at exchanging resources between agents, thus allowing them to execute the plans to achieve their goals.     

On robots and flies: Modeling the visual orientation behavior of flies

Although artificial and biological systems face similar sensorimotor control problems, until today only a few attempts have been made to implement specific biological control structures on robots. Nevertheless, the process of designing the sensorimotor control of a robot can contribute to our understanding of these mechanisms and can provide the basis of a critical evaluation of existing biological models. Flies have developed a specialized visuomotor control for tasks such as course stabilization, fixation and approach towards stationary objects, tracking of moving objects and landing, which are based on the analysis of visual motion information. Theoretical and experimental results suggest that in flies the visuomotor control for course stabilization as well as fixation and approach towards stationary objects may be implemented at least partially by one common sensory circuit. We present agents with a visuomotor controller that regulates the two behaviors of course stabilization and object fixation. To test this controller under real world conditions, we implemented it on a miniature robot. We have been able to show that in addition to course stabilization and object fixation, the robot also approaches stationary objects.