Mode and modal transfer effects on performance and discourse organization with an information retrieval dialogue system in natural language

This paper describes an experiment on the effects of learning, mode of interaction (written vs. spoken) and transfer mode on user performance and discourse organization during interaction with a natural language dialogue system. Forty-eight participants took part in a series of 12 dialogues with an information retrieval system presented either in the written or the spoken mode during the first six dialogues. The next six dialogues were then presented either in the same interaction mode or in another mode. The analysis of the results showed that performance (time, number of effective turns) improved throughout the dialogues whatever the mode of interaction. Nevertheless, performance was higher in the written mode. Moreover, mode-specific characteristics were observed. These consisted in greater use of subject pronouns and articles in the spoken mode. Similarly, in the spoken mode, the users found it easier to re-use the formulations presented in the system speech than in the written mode. Furthermore, the analysis also revealed a positive transfer effect on performance and discourse organization when the individuals first interacted in the spoken mode and then in the written mode. Both positive and negative transfer effects were observed when the individuals interacted first in the written mode followed by the spoken mode. The implications of the results are discussed in terms of direct and indirect consequences of modality effects on natural language dialogue interaction.     

User-Tailored Planning of Mixed Initiative Information-Seeking Dialogues

Intelligent dialogue systems usually concentrate on user support at the level of the domain of discourse, following a plan-based approach. Whereas this is appropriate for collaborative planning tasks, the situation in interactive information retrieval systems is quite different: there is no inherent plan-goal hierarchy, and users are known to often opportunistically change their goals and strategies during and through interaction. We need to allow for mixed-initiative retrieval dialogues, where the system evaluates the user's individual dialogue behavior and performs situation-dependent interpretation of user goals, to determine when to take the initiative and to change the control of the dialogue, e.g., to propose (new) problem-solving strategies to the user. In this article, we present the dialogue planning component of a concept- oriented, logic-based retrieval system (MIRACLE). Users are guided through the global stages of the retrieval interaction but may depart, at any time, from this guidance and change the direction of the dialogue. When users submit ambiguous queries or enter unexpected dialogue control acts, abductive reasoning is used to generate interpretations of these user inputs in light of the dialogue history and other internal knowledge sources. Based on these interpretations, the system initiates a short dialogue offering the user suitable options and strategies for proceeding with the retrieval dialogue. Depending on the user's choice and constraints resulting from the history, the system adapts its strategy accordingly.     

Human-computer dialogue design considerations

The three critical elements of quality human-computer dialogue design include a compilation of existing design guidelines, behavioral research to expand the general dialogue principles, and a computer-aided implementation for retrieval of the guidelines by dialogue authors. The need for empirically based dialogue guidelines relating to generic human-computer taks is stressed. The results of two studies dealing with interactive text editing are presented as examples of research directed toward specifying dialogue design principles and developing formal tools for human-computer dialogue research. One of these studies deals with a methodology for developing user models, and the second study deals with the design of HELP information. Computer-aiding in the form of rule-based systems for selecting appropriate dialogue guidelines and software tools for authoring dialogues are discussed as means for implementing these dialogue design considerations.     

Searching information with a natural language dialogue system: a comparison of spoken vs. written modalities

This paper examines the effects of spoken vs. written dialogue modalities on the effectiveness of information search with a computerized retrieval system. Forty-eight adults familiar with the use of computers were asked to carry out six information retrieval tasks, engaging with the system using either spoken or written communication. The written modality was more efficient with regard to the number of dialogue turns, length of interaction with the system and mental workload. Even though the turns lasted longer in the written mode, they appeared to yield less mental workload. Moreover, spoken and written dialogues did not differ as regards the use of pronouns and articles. The implications for the development of natural-language dialogue systems are discussed.     

Shaping user input: a strategy for natural language dialogue design

Traditional approaches to natural language dialogue interface design have adopted ordinary human-human conversation as the model for online human-computer interaction. The attempt to deal with all the subtleties of natural dialogues, such as topic focus, coherence, ellipsis, pronominal reference, etc. has resulted in prototype systems that are enormously complex and computationally expensive.

In a series of experiments^†, we explored ways of minimizing the processing burden of a dialogue system by channeling user input towards a more tractable, though still natural, form of Englishlanguage questions. Through linking a pair of terminals, we presented subjects with two different dialogue styles as a framework for online help in the domain of word-processing. The first dialogue style involved ordinary conversational format. The second style involved a simulation of an automated dialogue system, including apparent processing restrictions and ‘system process messages’ to inform the subject of the steps taken by the system during query analysis. In both cases human tutors played the role of the help system. After each dialogue session, subjects were interviewed to determine their assessments of the naturalness and usability of the dialogue interface.

We found that user input became more tractable to parsing and query analysis as the dialogue style became more formalized, yet the subjective assessment of naturalness and usability remained fairly constant. This suggests that techniques for channeling user input in a dialogue system may be effectively employed to reduce processing demands without compromising the benefits of a natural language interface. Theoretically, this data lends support to the hypothesis that unrestricted human-human conversation is not the most appropriate model for the design of human-computer dialogue interfaces.     

Usefulness,localizability, humanness,and language-benefit: additional evaluation criteria for natural language dialogue systems

Human–computer dialogue systems interact with human users using natural language. We used the ALICE/AIML chatbot architecture as a platform to develop a range of chatbots covering different languages, genres, text-types, and user-groups, to illustrate qualitative aspects of natural language dialogue system evaluation. We present some of the different evaluation techniques used in natural language dialogue systems, including black box and glass box, comparative, quantitative, and qualitative evaluation. Four aspects of NLP dialogue system evaluation are often overlooked: “usefulness” in terms of a user’s qualitative needs, “localizability” to new genres and languages, “humanness” or “naturalness” compared to human–human dialogues, and “language benefit” compared to alternative interfaces. We illustrated these aspects with respect to our work on machine-learnt chatbot dialogue systems; we believe these aspects are worthwhile in impressing potential new users and customers.     

A meta-rule approach to flexible plan recognition in dialogue

Although a number of researchers have demonstrated that reasoning on a model of the user's plans and goals is helpful in language understanding and response generation, current models of plan inference cannot handle naturally occurring dialogue. This paper argues that model building from less than ideal dialogues has a great deal in common with processing ill-formed input. It defines well-formedness constraints for information-seeking dialogues and contends that strategies for interpreting ill-formed input can be applied to the problem of modeling the user's plan during an ill-formed dialogue. It presents a meta-rule approach for hypothesizing the cause of dialogue ill-formedness, and describes meta-rules for relaxing the plan inference process and enabling the consideration of alternative hypotheses. The advantages of this approach are that it provides a unified framework for handling both well-formed and ill-formed dialogue, avoids unnatural interpretations when the dialogue is proceeding smoothly, and facilitates a nonmonotonic plan recognition system.Rhonda Eller is a Ph.D. candidate in Computer Science at the University of Delaware. She received her B.S. in Computer Science from Millersville University of Pennsylvania in 1987, and her M.S. degree in the same field from the University of Delaware. Her primary interests lie in the areas of natural language processing, plan recognition, and user modelling. This paper summarizes the current state of her thesis work on repair of an incorrectly inferred plan for a user.Sandra Carberry is an associate professor of computer science at the University of Delaware. Her research interests include discourse understanding, user modeling, planning and plan recognition, and intelligent natural language interfaces. This paper describes work that is part of an ongoing research project to develop a robust model of plan recognition.     

基于RNN句子编码器的聊天机器人

人机对话是自然语言处理领域衍生的一项现实应用场景,根据现实获取的大量短文本知识数据,构建单轮短对话式智能应答聊天机器人。本文基于传统的信息检索式聊天机器人,引入循环神经网络（RNN）深度表征交互式知识库中短文本的语义向量,重构表达式语义空间。实验表明该编码向量的方法比传统的利用TF-IDF向量的方法效果更好。     

面向用户偏好建模的个性化对话推荐算法

推荐系统的目标是从物品数据库中，选择出与用户兴趣偏好相匹配的子集，缓解用户面临的“信息过载”问题。因而近年来推荐系统越来越多地应用到电商、社交等领域，展现出巨大的商业潜力。传统推荐系统中，系统对用户的认知往往来源于历史交互记录，例如点击率或者购买记录，这是一种隐式用户反馈。对话推荐系统能够通过自然语言与用户进行多轮对话，逐步深入挖掘其兴趣偏好，从而向对方提供高质量的推荐结果。相比于传统推荐系统，对话推荐系统主要有两方面的不同。其一，对话推荐系统能够利用自然语言与用户进行语义上连贯的多轮对话，提升了人机交互中的用户体验；其二，系统能够询问特定的问题直接获取用户的显式反馈，从而更深入地理解用户兴趣偏好，提供更可靠的推荐结果。目前已经有不少工作在不同的问题设定下对该领域进行了探索，然而尽管如此，这些工作仍仅局限于关注当前正在进行的对话，忽视了过去交互记录中蕴涵的丰富信息，导致对用户偏好建模的不充分。为了解决这个问题，本文提出了一个面向用户偏好建模的个性化对话推荐算法框架，通过双线性模型注意力机制与自注意力层次化编码结构进行用户偏好建模，从而完成对候选物品的排序与推荐。本文设计的模型结构能够在充分利用用户历史对话信息的同时，权衡历史对话与当前对话两类数据的重要性。丰富的用户相关信息来源使得推荐结果在契合用户个性化偏好的同时，更具备多样性，从而缓解“信息茧房”等现象带来的不良影响。基于公开数据集的实验表明了本文方法在个性化对话推荐任务上的有效性。     

Structure for user-oriented dialogues in computer-aided telephony

Members of The Speech Project, at Oxford Brookes University, have been experimenting with alternative structures for ‘goal-seeking’ dialogues. A dialogue prototyping system has been developed that can instantiate different dialogue structures kept on file. These become active after being installed using dynamic data structures. Input is achieved using speech recognition for a small range of words, and output is in the form of pre-recorded speech messages. The dialogues thus created employ a new approach, and replace the usual menustructure with a simpler question-answer process.

The nature of dialogues using computer aided telephony is analysed, and then the work done by members of The Speech Project on an alternative approach using a simpler dialogue structure is described.     

Dispel: A run-time debugging language

Dispel is a language designed to aid communication between an interactive user and a run-time, symbolic debugging system. Important attributes of Dispel are that it provides a small set of primitive debugging actions, most traditional debugging aids (such as variable traces and postmortem dumps) are written in terms of these primitives as debugging routines, and Dispel serves both as an interactive debugging command language and as a special-purpose programming language. The syntax and semantics of Dispel are explained and examples of Dispel commands and routines are presented.     

Evaluation of Methods for the Determination of Factors Inducing Fatigue in Man at Work

The aim of the study was to determine the influence of textual feedback on the content and outcome of spoken interaction with a natural language dialogue system. More specifically, the assumption that textual feedback could disrupt spoken interaction was tested in a human–computer dialogue situation. In total, 48 adult participants, familiar with the system, had to find restaurants based on simple or difficult scenarios using a real natural language service system in a speech-only (phone), speech plus textual dialogue history (multimodal) or text-only (web) modality. The linguistic contents of the dialogues differed as a function of modality, but were similar whether the textual feedback was included in the spoken condition or not. These results add to burgeoning research efforts on multimodal feedback, in suggesting that textual feedback may have little or no detrimental effect on information searching with a real system.

Statement of Relevance: The results suggest that adding textual feedback to interfaces for human–computer dialogue could enhance spoken interaction rather than create interference. The literature currently suggests that adding textual feedback to tasks that depend on the visual sense benefits human–computer interaction. The addition of textual output when the spoken modality is heavily taxed by the task was investigated.     

Using speech and dialogue for interactive TV navigation

Interaction techniques for interactive television (iTV) are currently complex and difficult to use for a wide-range of viewers. Few previous studies have dealt with the potential benefits of multimodal dialogue interaction in the context of iTV for the purpose of flexibility, usability, efficiency, and accessibility. This paper investigates the benefits of introducing speech and connected dialogue for iTV interaction, and presents a case study in which a prototype system was built allowing users to navigate the information space and control the operation of the TV by a speech-based natural language interface. The system was evaluated by analysing the user experience in five categories capturing essential aspects of iTV interaction: interaction style, information load, data access, effectiveness and initiative. Design considerations relevant for speech and dialogue information systems for TV interfaces also emerged from the analysis.     

Sisl: Several Interfaces, Single Logic

Modern interactive services such as information and e-commerce services are becoming increasingly more flexible in the types of user interfaces they support. These interfaces incorporate automatic speech recognition and natural language understanding and include graphical user interfaces on the desktop and web-based interfaces using applets and HTML forms. To what extent can the user interface software be decoupled from the service logic software (the code that defines the essential function of a service)? Decoupling of user interface from service logic directly impacts the flexibility of services, or how easy they are to modify and extend. To explore these issues, we have developed Sisl, an architecture and domain-specific language for designing and implementing interactive services with multiple user interfaces. A key principle underlying Sisl is that all user interfaces to a service share the same service logic. Sisl provides a clean separation between the service logic and the software for a variety of interfaces, including Java applets, HTML pages, speech-based natural language dialogue, and telephone-based voice access. Sisl uses an event-based model of services that allows service providers to support interchangeable user interfaces (or add new ones) to a single consistent source of service logic and data. As part of a collaboration between research and development, Sisl is being used to prototype a new generation of call processing services for a Lucent Technologies switching product.     

MeteoBayes: Effective Plan Recognition in a Weather Dialogue System

When you've called a voice portal for any kind of information retrieval, chances are that an automated system guided the entire interaction. It might have correctly identified your goal, but probably only after asking too many questions. MeteoBayes is a meteorological information dialogue system that lets you use natural language to direct the interaction. Based on Bayesian networks, MeteoBayes' inference engine attempts to identify user intentions by consulting its past dialogue repository. For unfamiliar words, MeteoBayes has an unknown-term disambiguation module that learns word similarities from texts to avoid unnecessary system inquiries, thus speeding up the understanding process     

Testing the performance of spoken dialogue systems by means of an artificially simulated user

This paper proposes a new technique to test the performance of spoken dialogue systems by artificially simulating the behaviour of three types of user (very cooperative, cooperative and not very cooperative) interacting with a system by means of spoken dialogues. Experiments using the technique were carried out to test the performance of a previously developed dialogue system designed for the fast-food domain and working with two kinds of language model for automatic speech recognition: one based on 17 prompt-dependent language models, and the other based on one prompt-independent language model. The use of the simulated user enables the identification of problems relating to the speech recognition, spoken language understanding, and dialogue management components of the system. In particular, in these experiments problems were encountered with the recognition and understanding of postal codes and addresses and with the lengthy sequences of repetitive confirmation turns required to correct these errors. By employing a simulated user in a range of different experimental conditions sufficient data can be generated to support a systematic analysis of potential problems and to enable fine-grained tuning of the system.     

Semantic interpretation of a database query language

In this paper, we will discuss a system that semantically interprets a formal database accessing language and generates natural language from this interpretation. In the past, the major way of communication between a user and a database was by means of a formal language. One such language is the SQL query language. Even though constructed as a user friendly language, SQL exemplifies the same difficulties for users as do other formal languages, namely a fairly rigid syntax, the necessity of variable binding, the lack of pronouns, and in the case of erroneous queries error messages that do not provide much insight. To alleviate some of the formal language problems, yet utilize the power of the formal language, we set out to build a natural language ‘umbrella’ for the SQL user. Our goal was not to build a natural language query system, but rather to use semantic knowledge and natural language for paraphrasing the formal language (SQL) and producing error messages as a feedback mechanism. In this way we build a genuine help facility, which would not only aid the user in dealing with SQL, but also trap erroneous queries.