Real user evaluation of a POMDP spoken dialogue system using automatic belief compression

This article describes an evaluation of a POMDP-based spoken dialogue system (SDS), using crowdsourced calls with real users. The evaluation compares a “Hidden Information State” POMDP system which uses a hand-crafted compression of the belief space, with the same system instead using an automatically computed belief space compression. Automatically computed compressions are a way of introducing automation into the design process of statistical SDSs and promise a principled way of reducing the size of the very large belief spaces which often make POMDP approaches intractable. This is the first empirical comparison of manual and automatic approaches on a problem of realistic scale (restaurant, pub and coffee shop domain) with real users. The evaluation took 2193 calls from 85 users. After filtering for minimal user participation the two systems were compared on more than 1000 calls.     

Reinforcement learning for parameter estimation in statistical spoken dialogue systems

Reinforcement techniques have been successfully used to maximise the expected cumulative reward of statistical dialogue systems. Typically, reinforcement learning is used to estimate the parameters of a dialogue policy which selects the system's responses based on the inferred dialogue state. However, the inference of the dialogue state itself depends on a dialogue model which describes the expected behaviour of a user when interacting with the system. Ideally the parameters of this dialogue model should be also optimised to maximise the expected cumulative reward.This article presents two novel reinforcement algorithms for learning the parameters of a dialogue model. First, the Natural Belief Critic algorithm is designed to optimise the model parameters while the policy is kept fixed. This algorithm is suitable, for example, in systems using a handcrafted policy, perhaps prescribed by other design considerations. Second, the Natural Actor and Belief Critic algorithm jointly optimises both the model and the policy parameters. The algorithms are evaluated on a statistical dialogue system modelled as a Partially Observable Markov Decision Process in a tourist information domain. The evaluation is performed with a user simulator and with real users. The experiments indicate that model parameters estimated to maximise the expected reward function provide improved performance compared to the baseline handcrafted parameters.     

Evaluation of a hierarchical reinforcement learning spoken dialogue system

We describe an evaluation of spoken dialogue strategies designed using hierarchical reinforcement learning agents. The dialogue strategies were learnt in a simulated environment and tested in a laboratory setting with 32 users. These dialogues were used to evaluate three types of machine dialogue behaviour: hand-coded, fully-learnt and semi-learnt. These experiments also served to evaluate the realism of simulated dialogues using two proposed metrics contrasted with ‘Precision-Recall’. The learnt dialogue behaviours used the Semi-Markov Decision Process (SMDP) model, and we report the first evaluation of this model in a realistic conversational environment. Experimental results in the travel planning domain provide evidence to support the following claims: (a) hierarchical semi-learnt dialogue agents are a better alternative (with higher overall performance) than deterministic or fully-learnt behaviour; (b) spoken dialogue strategies learnt with highly coherent user behaviour and conservative recognition error rates (keyword error rate of 20%) can outperform a reasonable hand-coded strategy; and (c) hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of optimized dialogue behaviours in larger-scale systems.     

How to make and defend a proposal in a deliberation dialogue

In this paper it is shown how tools developed in argumentation theory and artificial intelligence can be applied to the development of a new dialectical analysis of the speech act of making a proposal in a deliberation dialogue. These tools are developed, modified and used to formulate dialogue pre-conditions, defining conditions and post-conditions for the speech act of making a proposal in a deliberation dialogue. The defining conditions set out what is required for a move in a dialogue to count as the making of a proposal by one of the parties. What is required are the conditions that (1) the move fit the requirements of the argumentation scheme for practical reasoning, and (2) the premises are propositions describing common goals of both parties or propositions that they reasonably consider means to achieve these goals. The analysis goes beyond the standard speech act approach by specifying not only the normative requirements for making a well-formed proposal, but also the requirements for responding to it by questioning or criticizing it, and the requirements for defending it.     

A fault tolerant single-chip intelligent agent with feature extraction capability

Autonomy and adaptability are key features of intelligent agents. Many applications of intelligent agents, such as the control of ambient intelligence environments and autonomous intelligent robotic systems, require the processing of information coming in from many available sensors to produce adequate output responses in changing scenarios. Autonomy, in these cases, applies not only to the ability of the agent to produce correct outputs without human guidance, but also to its ubiquity and/or portability, low-power consumption and integrability. In this sense, an embedded electronic system implementation paradigm can be applied to the design of autonomous intelligent agents in order to satisfy the above mentioned characteristics. However, processing complex computational intelligence algorithms with tight delay constraints in resource-constrained and low power embedded systems is a challenging engineering problem. In this paper a single-chip intelligent agent based on a computationally efficient neuro-fuzzy information processing core is described. The system has been endowed with an information preprocessing module based on Principal Component Analysis (PCA) that permits a substantial reduction of the input space dimensionality with little loss of modeling capability. Moreover, the PCA module has been tested as a means to achieve deep adaptability in changing environment dynamics and to endow the agent with fault tolerance in the presence of sensor failures. For data driven trials and research, a data set obtained from an experimental intelligent inhabited environment has been used as a benchmark system.     

Testing the performance of spoken dialogue systems by means of an artificially simulated user

This paper proposes a new technique to test the performance of spoken dialogue systems by artificially simulating the behaviour of three types of user (very cooperative, cooperative and not very cooperative) interacting with a system by means of spoken dialogues. Experiments using the technique were carried out to test the performance of a previously developed dialogue system designed for the fast-food domain and working with two kinds of language model for automatic speech recognition: one based on 17 prompt-dependent language models, and the other based on one prompt-independent language model. The use of the simulated user enables the identification of problems relating to the speech recognition, spoken language understanding, and dialogue management components of the system. In particular, in these experiments problems were encountered with the recognition and understanding of postal codes and addresses and with the lengthy sequences of repetitive confirmation turns required to correct these errors. By employing a simulated user in a range of different experimental conditions sufficient data can be generated to support a systematic analysis of potential problems and to enable fine-grained tuning of the system.     

Design of a real-time knowledge-based controller with applications in hydraulic turbine generator systems

Issues in the design and implementation of a real-time Knowledge-Based Controller (KBC) have been investigated. The design objective of such a controller is to maintain overall system stability and performance not only during normal, but also contingency, conditions. A general structure for such controllers is proposed. It consists of four major components: a knowledge base, a real-time inference engine, information processing algorithms, and a distributed controller. The functions of each individual component, as well as the relationship among them, are considered. The proposed design methodology is applied to synthesize a real-time knowledge-based controller for a hydraulic turbine governor.