Data-driven user simulation for automated evaluation of spoken dialog systems

This paper proposes a novel integrated dialog simulation technique for evaluating spoken dialog systems. A data-driven user simulation technique for simulating user intention and utterance is introduced. A novel user intention modeling and generating method is proposed that uses a linear-chain conditional random field, and a two-phase data-driven domain-specific user utterance simulation method and a linguistic knowledge-based ASR channel simulation method are also presented. Evaluation metrics are introduced to measure the quality of user simulation at intention and utterance. Experiments using these techniques were carried out to evaluate the performance and behavior of dialog systems designed for car navigation dialogs and a building guide robot, and it turned out that our approach was easy to set up and showed similar tendencies to real human users.     

A domain-independent statistical methodology for dialog management in spoken dialog systems

This paper proposes a domain-independent statistical methodology to develop dialog managers for spoken dialog systems. Our methodology employs a data-driven classification procedure to generate abstract representations of system turns taking into account the previous history of the dialog. A statistical framework is also introduced for the development and evaluation of dialog systems created using the methodology, which is based on a dialog simulation technique. The benefits and flexibility of the proposed methodology have been validated by developing statistical dialog managers for four spoken dialog systems of different complexity, designed for different languages (English, Italian, and Spanish) and application domains (from transactional to problem-solving tasks). The evaluation results show that the proposed methodology allows rapid development of new dialog managers as well as to explore new dialog strategies, which permit developing new enhanced versions of already existing systems.     

A framework for improving error detection and correction in spoken dialog systems

Despite the recent improvements in performance and reliably of the different components of dialog systems, it is still crucial to devise strategies to avoid error propagation from one another. In this paper, we contribute a framework for improved error detection and correction in spoken conversational interfaces. The framework combines user behavior and error modeling to estimate the probability of the presence of errors in the user utterance. This estimation is forwarded to the dialog manager and used to compute whether it is necessary to correct possible errors. We have designed an strategy differentiating between the main misunderstanding and non-understanding scenarios, so that the dialog manager can provide an acceptable tailored response when entering the error correction state. As a proof of concept, we have applied our proposal to a customer support dialog system. Our results show the appropriateness of our technique to correctly detect and react to errors, enhancing the system performance and user satisfaction.     

DaisyViz: A model-based user interface toolkit for interactive information visualization systems

While information visualization technologies have transformed our life and work, designing information visualization systems still faces challenges. Non-expert users or end-users need toolkits that allow for rapid design and prototyping, along with supporting unified data structures suitable for different data types (e.g., tree, network, temporal, and multi-dimensional data), various visualization, interaction tasks. To address these issues, we designed DaisyViz, a model-based user interface toolkit, which enables end-users to rapidly develop domain-specific information visualization applications without traditional programming. DaisyViz is based on a user interface model for information (UIMI), which includes three declarative models: data model, visualization model, and control model. In the development process, a user first constructs a UIMI with interactive visual tools. The results of the UIMI are then parsed to generate a prototype system automatically. In this paper, we discuss the concept of UIMI, describe the architecture of DaisyViz, and show how to use DaisyViz to build an information visualization system. We also present a usability study of DaisyViz we conducted. Our findings indicate DaisyViz is an effective toolkit to help end-users build interactive information visualization systems.     

A multi-formalism approach for model-based dynamic distribution of user interfaces of critical interactive systems

Evolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. User Centered Development promotes reactive answers to this kind of evolutions either by software evolutions through iterative development approaches or at runtime by providing additional information to the operators such as contextual help for instance. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. The approach proposed handles both concentration and distribution of user interfaces requiring both fusion of information into a single UI or fission of information into several ones. This generic model-based approach is exemplified on a safety critical system from space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new procedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus+context way improving usability by increasing both efficiency and effectiveness.     

The RavenClaw dialog management framework: Architecture and systems

In this paper, we describe RavenClaw, a plan-based, task-independent dialog management framework. RavenClaw isolates the domain-specific aspects of the dialog control logic from domain-independent conversational skills, and in the process facilitates rapid development of mixed-initiative systems operating in complex, task-oriented domains. System developers can focus exclusively on describing the dialog task control logic, while a large number of domain-independent conversational skills such as error handling, timing and turn-taking are transparently supported and enforced by the RavenClaw dialog engine. To date, RavenClaw has been used to construct and deploy a large number of systems, spanning different domains and interaction styles, such as information access, guidance through procedures, command-and-control, medical diagnosis, etc. The framework has easily adapted to all of these domains, indicating a high degree of versatility and scalability.     

The canonical processes of a dramatized approach to information presentation

This paper describes the application “Carletto the spider” in terms of the mapping with the canonical processes of media production. “Carletto the spider” is a character-based guide to a historical site and implements the Dramatour approach for the design of drama-based interactive presentations. Dramatization makes presentations more engaging, thus improving the reception of the content by the user. The major technical issue of the approach is the segmentation of the presentation into audiovisual units that are edited on-the-fly in a way that guarantees dramatic continuity while adapting to the user response. We describe the workflow of the application and its mapping to the canonical processes of media production, envisaging possible standardizations for the application portability.     

User preferences for adaptive user interfaces in health information systems

An adaptive user interface requires identification of user requirements. Interface designers and engineers must understand end-user interaction with the system to improve user interface design. A combination of interviews and observations is applied for user requirement analysis in health information systems (HIS). Then, user preferences are categorized in this paper as either data entry, language and vocabulary, information presentation, or help, warning and feedback. The user preferences in these categories were evaluated using the focus group method. Focus group sessions with different types of HIS users comprising medical staff (with and without computer skills) and system administrators identified each user group’s preference for the initial adaptation of the HIS user interface. User needs and requirements must be identified to adapt the interface to users during data entry into the system. System designers must understand user interactions with the system to identify their needs and preferences. Without this, interface design cannot be adapted to users and users will not be comfortable using the system and eventually abandon its use.     

Level of interest sensing in spoken dialog using decision-level fusion of acoustic and lexical evidence

Automatic detection of a user's interest in spoken dialog plays an important role in many applications, such as tutoring systems and customer service systems. In this study, we propose a decision-level fusion approach using acoustic and lexical information to accurately sense a user's interest at the utterance level. Our system consists of three parts: acoustic/prosodic model, lexical model, and a model that combines their decisions for the final output. We use two different regression algorithms to complement each other for the acoustic model. For lexical information, in addition to the bag-of-words model, we propose new features including a level-of-interest value for each word, length information using the number of words, estimated speaking rate, silence in the utterance, and similarity with other utterances. We also investigate the effectiveness of using more automatic speech recognition (ASR) hypotheses (n-best lists) to extract lexical features. The outputs from the acoustic and lexical models are combined at the decision level. Our experiments show that combining acoustic evidence with lexical information improves level-of-interest detection performance, even when lexical features are extracted from ASR output with high word error rate.     

基于模型法应对企业信息系统工程中不确定性需求

在软件工程或具体的需求工程中,用户需求通常具有不确定性.这成为了企业信息化实践中的主要问题之一;在企业信息系统工程中这尤其是一关键问题.通过扩展模型的概念、分析企业领域中可用模型的情况,提出了一种基于模型来应对用户需求之不确定性的方法.描述了应用基于模型的方法确定企业信息系统需求的基本逻辑与主要活动过程,并给出了一个应用ARIS(集成信息系统结构)参考模型库解决需求问题的实例.研究表明,基于模型的方法可用于有效地应对企业信息系统工程中的不确定性需求.     

User modeling in dialog systems: Potentials and hazards

In order to be capable of exhibiting a wide range of cooperative behavior, a computer-based dialog system must have available assumptions about the current user's goals, plans, background knowledge and (false) beliefs, i.e., maintain a so-called user model. Apart from cooperativity aspects, such a model is also necessary for intelligent coherent dialog behavior in general. This article surveys recent research on the problem of how such a model can be constructed, represented and used by a system during its interaction with the user. Possible applications, as well as potential problems concerning the advisability of application, are then discussed. Finally, a number of guidelines are presented which should be observed in future research to reduce the risk of a potential misuse of user modeling technology.     

The metadatabase approach to integrating and managing manufacturing information systems

Integration has become a self-evident goal in today's manufacturing enterprises. Since the late 1970s, numerous efforts have been launched world-wide to develop, employ, and deploy integration technology as a strategic weapon to compete in the global market-place. Important results have been obtained, including computer integrated manufacturing (CIM), concurrent engineering, and enterprise re-engineering. The field is poised to tackle some fundamental barriers of integration and thereby effect a new quantum jump in overall productivity, where information will become a fourth basic factor of production along with land, labor and capital. Among the barriers are the lack of information technology enabling multiple systems operating concurrently over different geographical regions. This paper discusses a unique approach to the integration problem regarding information resources management, global query and concurrent systems administration. This approach is based on the metadatabase model which incorporates both data and knowledge in order to accomplish information integration across manufacturing functions. Basic concepts, methods and techniques have been developed for the approach and verified via a prototype metadatabase system. The system's primary components and major functionalities are discussed and illustrated in detail through the use of an extended example of a pilot manufacturing system comprising order processing, process design and shop floor control. It will also provide the reader with a simulated system demonstration.A preliminary version of this paper was presented to the Third International Conference on Computer Integrated Manufacturing, May 1992.     

A methodology which requires user participation in the design of information systems

User participation in the design of computerized information systems is an accepted philosophy. The goal is to enhance the appreciation for and thus utility of the new system. In reality, however, system design procedures tend not to achieve this goal. The reason is that participation is implemented half heartedly, allowing the user to ask and answer questions, but not get involved in the design. This paper presents an adaptation of Nadler's “IDEALS” approach to the design of information systems. The approach presented is a five step procedure which removes the user from his daily information related activities and allows him to hypothesize an idealized use of information in his job. The approach then proceeds to allow the user and analyst to creatively implement a close approximation to the ideal design target.     

The neglected user in music information retrieval research

Personalization and context-awareness are highly important topics in research on Intelligent Information Systems. In the fields of Music Information Retrieval (MIR) and Music Recommendation in particular, user-centric algorithms should ideally provide music that perfectly fits each individual listener in each imaginable situation and for each of her information or entertainment needs. Even though preliminary steps towards such systems have recently been presented at the “International Society for Music Information Retrieval Conference” (ISMIR) and at similar venues, this vision is still far away from becoming a reality. In this article, we investigate and discuss literature on the topic of user-centric music retrieval and reflect on why the breakthrough in this field has not been achieved yet. Given the different expertises of the authors, we shed light on why this topic is a particularly challenging one, taking computer science and psychology points of view. Whereas the computer science aspect centers on the problems of user modeling, machine learning, and evaluation, the psychological discussion is mainly concerned with proper experimental design and interpretation of the results of an experiment. We further present our ideas on aspects crucial to consider when elaborating user-aware music retrieval systems.