Data-driven user simulation for automated evaluation of spoken dialog systems

This paper proposes a novel integrated dialog simulation technique for evaluating spoken dialog systems. A data-driven user simulation technique for simulating user intention and utterance is introduced. A novel user intention modeling and generating method is proposed that uses a linear-chain conditional random field, and a two-phase data-driven domain-specific user utterance simulation method and a linguistic knowledge-based ASR channel simulation method are also presented. Evaluation metrics are introduced to measure the quality of user simulation at intention and utterance. Experiments using these techniques were carried out to evaluate the performance and behavior of dialog systems designed for car navigation dialogs and a building guide robot, and it turned out that our approach was easy to set up and showed similar tendencies to real human users.     

A domain-independent statistical methodology for dialog management in spoken dialog systems

This paper proposes a domain-independent statistical methodology to develop dialog managers for spoken dialog systems. Our methodology employs a data-driven classification procedure to generate abstract representations of system turns taking into account the previous history of the dialog. A statistical framework is also introduced for the development and evaluation of dialog systems created using the methodology, which is based on a dialog simulation technique. The benefits and flexibility of the proposed methodology have been validated by developing statistical dialog managers for four spoken dialog systems of different complexity, designed for different languages (English, Italian, and Spanish) and application domains (from transactional to problem-solving tasks). The evaluation results show that the proposed methodology allows rapid development of new dialog managers as well as to explore new dialog strategies, which permit developing new enhanced versions of already existing systems.     

A framework for improving error detection and correction in spoken dialog systems

Despite the recent improvements in performance and reliably of the different components of dialog systems, it is still crucial to devise strategies to avoid error propagation from one another. In this paper, we contribute a framework for improved error detection and correction in spoken conversational interfaces. The framework combines user behavior and error modeling to estimate the probability of the presence of errors in the user utterance. This estimation is forwarded to the dialog manager and used to compute whether it is necessary to correct possible errors. We have designed an strategy differentiating between the main misunderstanding and non-understanding scenarios, so that the dialog manager can provide an acceptable tailored response when entering the error correction state. As a proof of concept, we have applied our proposal to a customer support dialog system. Our results show the appropriateness of our technique to correctly detect and react to errors, enhancing the system performance and user satisfaction.     

DaisyViz: A model-based user interface toolkit for interactive information visualization systems

While information visualization technologies have transformed our life and work, designing information visualization systems still faces challenges. Non-expert users or end-users need toolkits that allow for rapid design and prototyping, along with supporting unified data structures suitable for different data types (e.g., tree, network, temporal, and multi-dimensional data), various visualization, interaction tasks. To address these issues, we designed DaisyViz, a model-based user interface toolkit, which enables end-users to rapidly develop domain-specific information visualization applications without traditional programming. DaisyViz is based on a user interface model for information (UIMI), which includes three declarative models: data model, visualization model, and control model. In the development process, a user first constructs a UIMI with interactive visual tools. The results of the UIMI are then parsed to generate a prototype system automatically. In this paper, we discuss the concept of UIMI, describe the architecture of DaisyViz, and show how to use DaisyViz to build an information visualization system. We also present a usability study of DaisyViz we conducted. Our findings indicate DaisyViz is an effective toolkit to help end-users build interactive information visualization systems.     

A multi-formalism approach for model-based dynamic distribution of user interfaces of critical interactive systems

Evolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. User Centered Development promotes reactive answers to this kind of evolutions either by software evolutions through iterative development approaches or at runtime by providing additional information to the operators such as contextual help for instance. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. The approach proposed handles both concentration and distribution of user interfaces requiring both fusion of information into a single UI or fission of information into several ones. This generic model-based approach is exemplified on a safety critical system from space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new procedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus+context way improving usability by increasing both efficiency and effectiveness.     

The information center approach for developing computer-based information systems

Organizations which have used computers for a number of years are well acquainted with the software maintenance problem and the new-application development backlog. The first exists because computer programs require various types of changes throughout their life. The second has been created because so many systems development staff have to be assigned to software maintenance, and because many computer users have recognized the value of additional computer applications in their organization.It is believed that the concept of end-user computing has been used to relieve the new application development backlog in many organizations. In some, information centers have been implemented partly to support the development of computer applications by end-users. A survey was conducted to determine to what extent the information center approach is being used. Over one-half of the responding organizations reported their use. The survey included an identification of the functions provided by information centers to end-users and the responsibilities of both information center consultants and end-users. Recommendations are presented.     

The relation between user satisfaction,usage of information systems and performance

This study investigates the validity of two commonly used measures for the success of information systems (is): usage and user satisfaction (us). A questionnaire survey among Dutch managers was used to assess the mutual relation between both measures and performance. The results indicate that us is significantly related to performance (r=0.42). The relation between usage and performance is not significant. A partial correlation after correction for us is not significant either. This study provides empirical evidence for the popular assumption that us is the most appropriate measure for is success available.     

The RavenClaw dialog management framework: Architecture and systems

In this paper, we describe RavenClaw, a plan-based, task-independent dialog management framework. RavenClaw isolates the domain-specific aspects of the dialog control logic from domain-independent conversational skills, and in the process facilitates rapid development of mixed-initiative systems operating in complex, task-oriented domains. System developers can focus exclusively on describing the dialog task control logic, while a large number of domain-independent conversational skills such as error handling, timing and turn-taking are transparently supported and enforced by the RavenClaw dialog engine. To date, RavenClaw has been used to construct and deploy a large number of systems, spanning different domains and interaction styles, such as information access, guidance through procedures, command-and-control, medical diagnosis, etc. The framework has easily adapted to all of these domains, indicating a high degree of versatility and scalability.     

The canonical processes of a dramatized approach to information presentation

This paper describes the application “Carletto the spider” in terms of the mapping with the canonical processes of media production. “Carletto the spider” is a character-based guide to a historical site and implements the Dramatour approach for the design of drama-based interactive presentations. Dramatization makes presentations more engaging, thus improving the reception of the content by the user. The major technical issue of the approach is the segmentation of the presentation into audiovisual units that are edited on-the-fly in a way that guarantees dramatic continuity while adapting to the user response. We describe the workflow of the application and its mapping to the canonical processes of media production, envisaging possible standardizations for the application portability.     

User preferences for adaptive user interfaces in health information systems

An adaptive user interface requires identification of user requirements. Interface designers and engineers must understand end-user interaction with the system to improve user interface design. A combination of interviews and observations is applied for user requirement analysis in health information systems (HIS). Then, user preferences are categorized in this paper as either data entry, language and vocabulary, information presentation, or help, warning and feedback. The user preferences in these categories were evaluated using the focus group method. Focus group sessions with different types of HIS users comprising medical staff (with and without computer skills) and system administrators identified each user group’s preference for the initial adaptation of the HIS user interface. User needs and requirements must be identified to adapt the interface to users during data entry into the system. System designers must understand user interactions with the system to identify their needs and preferences. Without this, interface design cannot be adapted to users and users will not be comfortable using the system and eventually abandon its use.     

Measuring the privacy of user profiles in personalized information systems

Personalized information systems are information-filtering systems that endeavor to tailor information-exchange functionality to the specific interests of their users. The ability of these systems to profile users is, on the one hand, what enables such intelligent functionality, but on the other, the source of innumerable privacy risks. In this paper, we justify and interpret KL divergence as a criterion for quantifying the privacy of user profiles. Our criterion, which emerged from previous work in the domain of information retrieval, is here thoroughly examined by adopting the beautiful perspective of the method of types and large deviation theory, and under the assumption of two distinct adversary models. In particular, we first elaborate on the intimate connection between Jaynes’ celebrated method of entropy maximization and the use of entropies and divergences as measures of privacy; and secondly, we interpret our privacy metric as false positives and negatives in a binary hypothesis testing.     

The effects of user participation on the design of accounting information systems

The roles of user participation (UP) in the design of Management Accounting Information System(s) (MAS) have long received the attention of MAS researchers. By surveying business firms through a structured questionnaire, this study investigated the interactions among contextual variables (Task uncertainty and organizational structure), information characteristics (Scope, timeliness and aggregation) and UP. The results suggest that under high task uncertainty, aggregated and timely information with high UP is positively associated with increased MAS performance. However, when the task uncertainty is low, the UP makes no impact on the relation between performance and information characteristics. In a less structured organization, broad scope, timely and aggregated information with high UP has a positive influence on the performance. In mechanistic organization, narrow scope and disaggregated information through the high UP can lead to higher MAS performance.     

Level of interest sensing in spoken dialog using decision-level fusion of acoustic and lexical evidence

Automatic detection of a user's interest in spoken dialog plays an important role in many applications, such as tutoring systems and customer service systems. In this study, we propose a decision-level fusion approach using acoustic and lexical information to accurately sense a user's interest at the utterance level. Our system consists of three parts: acoustic/prosodic model, lexical model, and a model that combines their decisions for the final output. We use two different regression algorithms to complement each other for the acoustic model. For lexical information, in addition to the bag-of-words model, we propose new features including a level-of-interest value for each word, length information using the number of words, estimated speaking rate, silence in the utterance, and similarity with other utterances. We also investigate the effectiveness of using more automatic speech recognition (ASR) hypotheses (n-best lists) to extract lexical features. The outputs from the acoustic and lexical models are combined at the decision level. Our experiments show that combining acoustic evidence with lexical information improves level-of-interest detection performance, even when lexical features are extracted from ASR output with high word error rate.     

基于模型法应对企业信息系统工程中不确定性需求

在软件工程或具体的需求工程中,用户需求通常具有不确定性.这成为了企业信息化实践中的主要问题之一;在企业信息系统工程中这尤其是一关键问题.通过扩展模型的概念、分析企业领域中可用模型的情况,提出了一种基于模型来应对用户需求之不确定性的方法.描述了应用基于模型的方法确定企业信息系统需求的基本逻辑与主要活动过程,并给出了一个应用ARIS(集成信息系统结构)参考模型库解决需求问题的实例.研究表明,基于模型的方法可用于有效地应对企业信息系统工程中的不确定性需求.     

User modeling in dialog systems: Potentials and hazards

In order to be capable of exhibiting a wide range of cooperative behavior, a computer-based dialog system must have available assumptions about the current user's goals, plans, background knowledge and (false) beliefs, i.e., maintain a so-called user model. Apart from cooperativity aspects, such a model is also necessary for intelligent coherent dialog behavior in general. This article surveys recent research on the problem of how such a model can be constructed, represented and used by a system during its interaction with the user. Possible applications, as well as potential problems concerning the advisability of application, are then discussed. Finally, a number of guidelines are presented which should be observed in future research to reduce the risk of a potential misuse of user modeling technology.