Cooperative requests and replies in a collaborative dialogue model

In this paper, we present a computational model of dialogue, and an underlying theory of action, which supports the representation of, reasoning about and execution of communicative and non-communicative actions. This model rests on a theory of collaborative discourse, and allows for cooperative human–machine communication in written dialogues. We show how cooperative behaviour, illustrated by the analysis of a dialogue corpus and formalized by an underlying theory of cooperation, is interpreted and produced in our model. We describe and illustrate in detail the main algorithms used to model the reasoning processes necessary for interpretation, planning, generation, as well as for determining which actions to perform and when. Finally, we present our implemented system.Our data are drawn from a corpus of human–human dialogues, selected and transcribed from a day-long recording of phone calls at a phone desk in an industrial setting (Castaing, 1993). We present an analysis of this corpus, focusing on dialogues which require, in order to succeed, helpful behaviour on the part of both the caller and the operator.The theoretical framework of our model rests on the theory of collaborative discourse developed by Grosz and Sidner (1986, 1990), Grosz and Kraus (1993, 1996), and further extended by Lochbaum (1994, 1995). An important objective guiding the design of our dialogue model was to allow the agent being modelled to interpret and manifest a type of cooperative behaviour which follows Grosz and Kraus's formalization of the commitment of each collaborative agent towards the actions of the other collaborative agents. The model we propose extends Lochbaum's approach to discourse processing in extending her interpretation algorithm to allow for the treatment of a wider range of dialogues, and in providing an algorithm of task advancement which guides the generation process and allows for the interleaving of execution and planning, thereby facilitating cooperation among agents. The cooperative behaviour of the agent being modelled rests on the use of communicative actions allowing agents to share additional knowledge and assist each other in performing their actions.     

Integrating linguistic primitives in learning context-dependentrepresentation

The paper presents an explicit connectionist-inspired, language learning model in which the process of settling on a particular interpretation for a sentence emerges from the interaction of a set of “soft” lexical, semantic, and syntactic primitives. We address how these distinct linguistic primitives can be encoded from different modular knowledge sources but strongly involved in an interactive processing in such a way as to make implicit linguistic information explicit. The learning of a quasi-logical form called context-dependent representation, is inherently incremental and dynamical in such a way that every semantic interpretation will be related to what has already been presented in the context created by prior utterances. With the aid of the context-dependent representation, the capability of the language learning model in text understanding is strengthened. This approach also shows how the recursive and compositional role of a sentence as conveyed in the syntactic structure can be modeled in a neurobiologically motivated linguistics based on dynamical systems rather on combinatorial symbolic architecture. Experiments with more than 2000 sentences in different languages illustrating the influences of the context-dependent representation on semantic interpretation, among other issues, are included     

Plans, Actions and Dialogues Using Linear Logic

We describe how Intuitionistic Linear Logic can be used to provide a unified logical account for agents to find and execute plans. This account supports the modelling of agent interaction, including dialogue; allows agents to be robust to unexpected events and failures; and supports significant reuse of agent specifications. The framework has been implemented and several case studies have been considered. Further applications include human–computer interfaces as well as agent interaction in the semantic web.     

Introspective and elaborative processes in rational agents

This paper explores the design of rational agent architectures from the perspective of the dynamics of information change. The procedural elements that guide an agent's behavior and that reflect the evolution of pro-attitudes (for example, from desire to intention to plan) are described in terms of McCarthy's notion of a reified mental action. The function of each module of an agent architecture is exactly specified by identifying processes with each module and then describing the effects of those processes or mental actions (such as updating beliefs, elaborating plans, deliberating, reconsidering, revising intentions, filtering intentions, and monitoring) in the same way as one would describe the effects of physical actions. A new semantics for intention is presented that is both dynamic and causal in the sense that it is given in terms of the relation of an intention to both previous and subsequent mental states as well as to the choice of physical action. Desires are given a syntactic analysis while the pro-attitude of intentions-that, which has been proposed in the SharedPlans framework of Grosz and Kraus, is axiomatized in terms of an evolving commitment to certain deliberative, mental actions that evolve as a function of knowledge of the state of the joint activity. This revised version was published online in June 2006 with corrections to the Cover Date.     

Plan inferences in dialogue under Dynamical Constraint Programming

Flexible dialogues involve various inferences on plans. Inferring an agent’s intentions can be regarded as reasoning from both utterances and behavior. Since plan generation and plan recognition involve various inference patterns, which encompass both deduction and abduction, reasoning control raises various problems. To implement control of inferences on plans in a domain-independent fashion, we employ a computational architecture called Dynamical Constraint Programming. Dynamical Constraint Programming accounts for the semantics of first-order clausal programs in terms of dynamics, with potential energy and field of force, from which various heuristics for control of inferences emerge on the basis of the energy minimization principle. We introduce a computational treatment of verbal communication and account for both plan inference and generation in a dialogue.     

Dynamic intention structures I: a theory of intention representation

This article introduces a new theory of intention representation which is based on a structure called a Dynamic Intention Structure (DIS). The theory of DISs was motivated by the problem of how to properly represent incompletely specified intentions and their evolution. Since the plans and intentions of collaborating agents are most often elaborated incrementally and jointly, elaboration processes naturally involve agreements among agents on the identity of appropriate agents, objects and properties that figure into their joint plans. The paper builds on ideas from dynamic logic to present a solution to the representation and evolution of agent intentions involving reference to incompletely specified and, possibly, mutually dependent intentions, as well as the objects referenced within those intentions. It provides a first order semantics for the resulting logic. A companion paper extends further the logical form of DISs and explores the problem of logical consequence and intention revision.     

An Intelligent Man-Machine Dialogue System Based on AI Planning

We describe the modular architecture of a generic dialogue system that assists a user/operator in performing a task with a tool. This coaching system is named CALLIOPE after the Greek goddess of eloquence. It aims at being an active partner in an intelligent man-machine dialogue. The intelligent dimension of the coaching system is reflected by its ability to adapt to the user and the situation at hand. The CALLIOPE system contains an explicit user model and world model to situate its dialogue actions. A plan library allows it to follow loosely predetermined dialogue scenarios.The heart of the coaching system is an AI planning module, which plans a series of dialogue actions. We present a coherent set of three dialogue or speech actions that will make up the physical form of the man-machine communication.The use of the AI planning paradigm as a basis for man-machine interaction is motivated by research in various disciplines, as e.g., AI, Cognitive Science and Social Sciences. Starting from the man-man communication metaphor, we can view the thinking before speaking of a human communication partner as constructing an underlying plan which is responsible for the purposiveness, the organisation and the relevance of the communication.CALLIOPE has been fully implemented and tested on theoretical examples. At present, also three tailored versions of CALLIOPE are in operational use in different industrial application domains: operator support for remedying tasks in chemical process industry, operator support for a combined task of planning, plan execution and process control in the area of chemical process development, and thirdly decision support in production scheduling.     

GRAFLOG: Understanding Drawings through Natural Language

This paper describes an experimental interactive graphics interface, GRAFLOG, in which drawings receive linguistic interpretations. It is possible to emulate linguistic interaction in situations where graphics is thought to be necessary. The paper presents examples of such a kind of dialogue and the architecture of the implementation. The paper explains how representations of drawings can be constructed by treating graphical symbols as "objects", and how a parallel linguistic interpretation for these symbols can be constructed. It highlights the relevance of "deictic expressions" and "spatial prepositions" in building the interface mechanisms between these two kinds of representations. Lastly, it shows how a reasoning component is constructed for making deductions from premises that are found in both the graphical and linguistic domains. Using GRAFLOG, it is possible to represent knowledge through words and pictures. GRAFLOG is implemented, using an object oriented programming style, in PROLOG and GKS.     

基于关系的两维意向结构

从建构agent角度出发,提出了一个基于关系结构的包括agent意向、信念以及目标等认知状态的框架.在此框架中,实现目标的意向形成了两维序结构,其中一维表示意向间的时序关系,另一维表示意向间的相干关系,在此基础上,研究了信念、意向和目标的相互关系.因为摒弃了传统的用模态算子来刻画agent的意向的方法,所以在构建agent时,可以直接采用意向库以及意向间的时序、相干关系来表示agent的意向,从而缩小了agent理论模型与实际agent结构之间的差异,为agent结构的建立提供了必要的理论基础.     

A Language/Action Perspective on cooperative information agents

The communication between different autonomous Information and Communication Systems requires a certain amount of intelligence of each system. One way of supporting this intelligence and autonomy is by means of Cooperative Information Agents (CIA). We show that basing the information contents of these agents on linguistic concepts and furthermore modelling the communications between the agents using the Language/Action Perspective provides for a natural and sound setting for these CIAs. In addition, we describe an agent architecture and agent language particularly suited for implementing communication systems.     

A hybrid agent architecture integrating desire,intention and reinforcement learning

This paper presents a hybrid agent architecture that integrates the behaviours of BDI agents, specifically desire and intention, with a neural network based reinforcement learner known as Temporal Difference-Fusion Architecture for Learning and COgNition (TD-FALCON). With the explicit maintenance of goals, the agent performs reinforcement learning with the awareness of its objectives instead of relying on external reinforcement signals. More importantly, the intention module equips the hybrid architecture with deliberative planning capabilities, enabling the agent to purposefully maintain an agenda of actions to perform and reducing the need of constantly sensing the environment. Through reinforcement learning, plans can also be learned and evaluated without the rigidity of user-defined plans as used in traditional BDI systems. For intention and reinforcement learning to work cooperatively, two strategies are presented for combining the intention module and the reactive learning module for decision making in a real time environment. Our case study based on a minefield navigation domain investigates how the desire and intention modules may cooperatively enhance the capability of a pure reinforcement learner. The empirical results show that the hybrid architecture is able to learn plans efficiently and tap both intentional and reactive action execution to yield a robust performance.     

REACTIVE AGENT DESIGN FOR INTELLIGENT TUTORING SYSTEMS

This paper presents a new approach to the analysis and design of intelligent tutoring systems (ITS), based on reactive principles and cognitive models, this way leading to multiagent architecture. In these kinds of models, the analysis problem is treated bottom-up, as opposed to that of traditional artificial intelligence (AI), i.e., top down. We present one ITS example called Makatsina (meaning tutor in TOTONACA, a Mexican pre-Columbian language), constructed according to this approach, which teaches the skills necessary to solve the truss analysis problem by the method of joints. This learning domain is an integration skill. The classical ITS work is based on explicit goals and an internal representation of the environment. The new approach has reactive agents which have no representation of their environment and act using a stimulus response behavior type. In this way they can respond to the present state of the environment in which they are embedded. With these elements, errors, and teaching plans, each agent behaves as an expert assistant that is able to handle different teaching methods. Reactive agent programming is found to be simple because agents have simple behaviors. The difficulty lies in the interaction mechanism analysis and design between the environment and the intelligent reactive system.     

Towards personality-based user adaptation: psychologically informed stylistic language generation

Conversation is an essential component of social behavior, one of the primary means by which humans express intentions, beliefs, emotions, attitudes and personality. Thus the development of systems to support natural conversational interaction has been a long term research goal. In natural conversation, humans adapt to one another across many levels of utterance production via processes variously described as linguistic style matching, entrainment, alignment, audience design, and accommodation. A number of recent studies strongly suggest that dialogue systems that adapted to the user in a similar way would be more effective. However, a major research challenge in this area is the ability to dynamically generate user-adaptive utterance variations. As part of a personality-based user adaptation framework, this article describes personage, a highly parameterizable generator which provides a large number of parameters to support adaptation to a user’s linguistic style. We show how we can systematically apply results from psycholinguistic studies that document the linguistic reflexes of personality, in order to develop models to control personage’s parameters, and produce utterances matching particular personality profiles. When we evaluate these outputs with human judges, the results indicate that humans perceive the personality of system utterances in the way that the system intended.     

A Generative System for Mamluk Madrasa Form-Making

In this paper, a parametric shape grammar for the derivation of the floor plans of educational buildings (madrasas) in Mamluk architecture is presented. The grammar is constructed using a corpus of sixteen Mamluk madrasas that were built in Egypt, Syria, and Palestine during the Mamluk period. Based on an epistemological premise of structuralism, the morphology of Mamluk madrasas is analyzed to deduce commonalities of the formal and compositional aspects among them. The set of underlying common lexical and syntactic elements that are shared by the study cases is listed. The shape rule schemata to derive Mamluk madrasa floor plans are formulated. The sets of lexical elements and syntactic rules are systematized to form a linguistic framework. The theoretical framework for the formal language of Mamluk architecture is structured to establish a basis for a computerized model for the automatic derivation of Mamluk madrasa floor plans.     

KAZOO: a sign language generation platform based on production rules

This paper describes KAZOO, a web application for sign language (SL) generation using a virtual signer. Firstly, it explains the motivation to this project, which is grounded on an approach designed solely from SL corpus analysis and modelling. Then, various projects conducted in the past few years on linguistic modelling and 3D animation are presented. The platform’s architecture integrates parts of this work and new pieces of software allowing control and linking of all these components. This is an ongoing project, though the current version offers the possibility to animate a virtual signer automatically and synthesize the contents using an abstract representation, the authors’ own linguistic model AZee, providing a means of validating this model.     

A communication model based on the 2-tuple fuzzy linguistic representation for a distributed intelligent agent system on Internet

 Internet users are assisted by means of distributed intelligent agents in the information gathering process to find the fittest information to their needs. In this paper we present a distributed intelligent agent model where the communication of the evaluation of the retrieved information among the agents is carried out by using linguistic operators based on the 2-tuple fuzzy linguistic representation as a way to endow the retrieval process with a higher flexibility, uniformity and precision. The 2-tuple fuzzy linguistic representation model allows to make processes of computing with words without loss of information.     

Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot

Automatic perception of human affective behaviour from facial expressions and recognition of intentions and social goals from dialogue contexts would greatly enhance natural human robot interaction. This research concentrates on intelligent neural network based facial emotion recognition and Latent Semantic Analysis based topic detection for a humanoid robot. The work has first of all incorporated Facial Action Coding System describing physical cues and anatomical knowledge of facial behaviour for the detection of neutral and six basic emotions from real-time posed facial expressions. Feedforward neural networks (NN) are used to respectively implement both upper and lower facial Action Units (AU) analysers to recognise six upper and 11 lower facial actions including Inner and Outer Brow Raiser, Lid Tightener, Lip Corner Puller, Upper Lip Raiser, Nose Wrinkler, Mouth Stretch etc. An artificial neural network based facial emotion recogniser is subsequently used to accept the derived 17 Action Units as inputs to decode neutral and six basic emotions from facial expressions. Moreover, in order to advise the robot to make appropriate responses based on the detected affective facial behaviours, Latent Semantic Analysis is used to focus on underlying semantic structures of the data and go beyond linguistic restrictions to identify topics embedded in the users’ conversations. The overall development is integrated with a modern humanoid robot platform under its Linux C++ SDKs. The work presented here shows great potential in developing personalised intelligent agents/robots with emotion and social intelligence.