Wizard of Oz studies — why and how

Current approaches to the development of natural language dialogue systems are discussed, and it is claimed that they do not sufficiently consider the unique qualities of man-machine interaction as distinct from general human discourse. It is concluded that empirical studies of this unique communication situation are required for the development of user-friendly interactive systems. One way of achieving this is through the use of so-called Wizard of Oz studies. The focus of the work described in the paper is on the practical execution of the studies and the methodological conclusions drawn on the basis of the authors' experience. While the focus is on natural language interfaces, the methods used and the conclusions drawn from the results obtained are of relevance also to other kinds of intelligent interfaces.     

A user interface evaluation environment using synchronized video,visualizations and event trace data

This paper presents a simple but very powerful technique to support user interface evaluation along with a prototype open environment — I-Observe, the Interface OBServation, Evaluation, Recording, and Visualization Environment. I-Observe's technique requires recording user interface sessions in multiple modalities, both as a trace of interesting events and through video images. It provides tools to allow the user interface evaluator either to analyze and visualize the event tracer data, or to combine the event trace and video modalities. The analyst using I-Observe can search the event stream for patterns of interesting or important user actions, then use the recorded timestamps associated with these actions to present only the sections of the video recording of interest. This allows the analyst to study, for example, all places where the user invokes a help system or a particular command to be observed, without manually searching the recording or sitting through long sessions of unrelated interactions. By combining the precise recording of automatic event trace capture with the rich contextual information that can be captured in a video and audio recording, this technique allows analyses to be performed that would not be practical with either media alone.     

Evaluation of user interfaces: EVADIS II— a comprehensive evaluation approach

As a result of the importance of the usability approach in system development and the EC's 'Directive concerning the minimum safety and health requirements for VDT workers' (EWG 1990), there is an accepted need for practical evaluation methods for user interfaces. The usability approach and the EC Directive are not restricted to user interface design, as they include the design of appropriate hardware and software, as well as organization, job, and task design. Therefore system designers are faced with many, often conflicting, requirements and need to address the question, 'How can usability requirements comprehensively be considered and evaluated in system development?' Customers buying hardware and software and introducing them into their organization ask, (How can I select easy-to-use hardware and software?' Both designers and customers need an evaluation procedure that covers all the organizational, user, hard- and software requirements. The evaluation method, EVADIS.II, we present in this paper overcomes characteristic deficiencies of previous evaluation methods. In particular, it takes the tasks, the user, and the organizational context into consideration during the evaluation process, and provides computer support for the use of the evaluation procedure.     

The promise of zoomable user interfaces

Zoomable user interfaces (ZUIs) have received a significant amount of attention in the 18 years since they were introduced. They have enjoyed some success, and elements of ZUIs are widely used in computers today, although the grand vision of a zoomable desktop has not materialised. This paper describes the premise and promise of ZUIs along with their challenges. It describes design guidelines, and offers a cautionary tale about research and innovation.     

Designing the user interface of the computer-based speech training system ARTUR based on early user tests

This study has been performed in order to evaluate a prototype for the human - computer interface of a computer-based speech training aid named ARTUR. The main feature of the aid is that it can give suggestions on how to improve articulations. Two user groups were involved: three children aged 9 - 14 with extensive experience of speech training with therapists and computers, and three children aged 6, with little or no prior experience of computer-based speech training. All children had general language disorders. The study indicates that the present interface is usable without prior training or instructions, even for the younger children, but that more motivational factors should be introduced. The granularity of the mesh that classifies mispronunciations was satisfactory, but the flexibility and level of detail of the feedback should be developed further.     

A multi-formalism approach for model-based dynamic distribution of user interfaces of critical interactive systems

Evolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. User Centered Development promotes reactive answers to this kind of evolutions either by software evolutions through iterative development approaches or at runtime by providing additional information to the operators such as contextual help for instance. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. The approach proposed handles both concentration and distribution of user interfaces requiring both fusion of information into a single UI or fission of information into several ones. This generic model-based approach is exemplified on a safety critical system from space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new procedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus+context way improving usability by increasing both efficiency and effectiveness.     

一种新的笔式交互范式的可用性评估

笔式用户界面是Post—WIMP界面的一种重要形态,具有自然、和谐的交互特性。尽管笔式用户界面广泛应用于无处不在计算环境中,但很少专门对它的可用性进行评估。针对笔式用户界面的特征,提出了一种新的笔式用户界面范式,基于此范式设计并实现了一个面向教学的笔式交互系统(PenOffice),利用此系统对范式从易学性、效率、出错率和用户满意度这几个方面进行了可用性评估,从中分析了笔式用户界面以及这种新的界面范式的优点和不足。     

Towards the design of usable multimodal interaction languages

This paper presents novel recommendations for the design of usable multimodal command or query languages. These recommendations have been inferred from the results of three empirical studies focused on the use of spontaneous speech (first study) and the synergic use of spontaneous versus controlled speech and gestures for interacting with current application software (second and third studies). In particular, we propose a method for designing multimodal languages that can be considered as an appropriate substitute for direct manipulation in all contexts precluding the use of mouse and keyboard, and for all standard categories of users, especially the general public.     

WAMM in the framework of graphical user interfaces for metacomputing management

Current environments for metacomputing generally have tools for managing the resources of a metacomputer but often lack adequate tools for designing, writing, and executing programs. Building an application for a metacomputer typically involves writing source codes on a local node, transferring and compiling codes on every node, and starting their execution. Without such tools, the application development phases can come up against considerable difficulties. In order to alleviate these problems, some graphical user interfaces (GUIs) based on PVM, such as XPVM, Parallel Application Development Environment (PADE) and Wide Area Metacomputing Manager (WAMM) have been implemented. These GUIs integrate a programming environment which facilitates the user in performing the application development phases and the application execution.

This paper outlines the general requirements for designing GUIs for metacomputing management, and compares WAMM, a graphical user interface, with some related works.     

Investigating the effects of multimodal feedback through tracking state in pen-based interfaces

A tracking state increases the bandwidth of pen-based interfaces. However, this state is difficult to detect with default visual feedback. This paper reports on two experiments that are designed to evaluate multimodal feedback for pointing tasks (both 1D and 2D) in tracking state. In 1D pointing experiments, results show that there is a significant effect for input types on movement time while feedback type and the use of different hands for receiving feedback (i.e. the dominant or non-preferred hand) do not affect movement time significantly. We also report that there is a significant effect for feedback types and input device types on error rate while the choice of hand (used for detecting feedback vibrations) does not affect the error rate significantly. In the 2D pointing experiment, results show that there are no significant effects for either input type or the use of different hands on movement time while feedback type affects movement time significantly. Results for both the 1D and 2D pointing tasks show that tactile plus visual feedback can improve accuracy and audio is not efficient to give user feedback in tracking state. This paper proposes several guidelines for feedback design in tracking state. We believe these results can aid designers of pen-based interfaces.     

学生综合素质评价的用户画像构建研究

提升职业院校学生综合素质评价的科学性与准确性,是建设现代职业教育体系的必然要求。文章按照学生综合素质评价体系的要素,采集学生在校期间产生的海量数据,并对海量数据清洗、交换和集成。通过标签定义和模型特征构建,实现基于学生思想品德、身心素质、学业水平、实践能力、职业能力和指标评价的用户画像模型。该模型的构建为学生综合素质评价的方法、过程、反馈提供了新思路和新方法。     

Design and evaluation of an adaptive icon toolbar

As information systems become increasingly important in many different domains, the potential to adapt them to individual users and their needs also becomes more important. Adaptive user interfaces offer many possible ways to adjust displays and improve procedures for a user's individual patterns of work. This paper describes an attempt to design an adaptive user interface in a computer environment familiar to many users. According to one classification of adaptive user interfaces, the adaptive bar described in this paper would be classified as a user-controlled self-adaptation system.At the user's convenience, the adaptive bar offers suggestions for adding or removing command icons, based on the frequency and probability of specific commands. It also implements these changes once the user has agreed to them. Beyond the adaptive bar, the general behavior of the whole user interface does not change, thereby allowing the user to maintain a clear general model of the system. This paper describes the decision-making algorithm implemented in the bar. It also describes the bar's self-adaptive behavior of displaying the frequency of each icon's use through the icon's size. Finally, we present some encouraging preliminary results of evaluations by users.     

3M: a user modelling interface of an expert system for X-ray topographic image interpretation

This paper discusses user modelling techniques and presents the design and implementation of the 3M user modelling interface of INTEREX. INTEREX is an expert system for X-ray topographic image interpretation which assists its users in identifying and analysing a number of defects that can occur in high-quality crystals. 3M consists of a monitor, a model and a modifier. It is used to adapt the consultation route and the explanations provided by INTEREX to three categories of users. It demonstrates the use-of an implicit, individual, dynamic and long-term user model to enable an expert system to accommodate users with different levels of expertise.     

Towards a practical method of user interface evaluation

This paper describes a practical method for evaluating the usability of human-computer interfaces. The paper specifies the requirements of such a method, and then outlines our work in developing a method to meet this specification. The method is based on the conduct of realistic tasks with an interactive system and the subsequent systematic elicitation of end-users' and designers' reactions to the interface using a criterion-based evaluation checklist.

Two practical examples are used to illustrate development of the method: (a) evaluation of a prototype production scheduling system, and (b) comparative assessment of the usability of three prototype user interfaces to a public-access database. The paper discusses some issues raised by the method and considers how it can be furthe developed.     

The design of the user interface for a large physics experiment

The paper describes the experience in developing and supporting the user interface of RFX, one of the large nuclear fusion experiments of the co-ordinated nuclear fusion experiment programme of the European Community. The aim of this work is to present the problems and some possible solutions when developing user interfaces in a scientific environment, especially in large physics experiments. An overview of the current state of interface technology in such an environment is first provided. The control and data acquisition system of the RFX experiment is then introduced and its user interface described in greater detail. Finally, our experience both in maintaining the system interface and in training its users is described.     

Effects of compatibility and turning biases on arrowhead cursor placement in graphical user interfaces

Arrowhead cursors have the potential to generate response conflict, as they are used to signify location but also implicitly cue direction. This study considered the time course of the resolution of any response conflict. The interplay between object properties and action was addressed through consideration of the effects of arrowhead cursor orientation upon action. Twelve participants moved arrowhead cursors of four possible orientations with respect to direction of movement (compatible, incompatible, port, starboard) towards a square (4 mm or 8 mm) target in a graphical user interface. Kinematic analysis determined if and how an interplay existed between object properties and action. Cursor orientation primarily influenced cursor movement, with the effects manifesting during terminal guidance. The blunt end of an arrowhead cursor was moved faster by users, and there appeared to be turning biases during cursor placement. Arrowheads have a number of properties (vertex, orientation, width) that influence cursor placement in graphical user interfaces.     

On the role of individual human abilities in the design of adaptive user interfaces for scientific problem solving environments

A scientific problem solving environment should be built in such a way that users (scientists) might exploit underlying technologies without a specialised knowledge about available tools and resources. An adaptive user interface can be considered as an opportunity in addressing this challenge. This paper explores the importance of individual human abilities in the design of adaptive user interfaces for scientific problem solving environments. In total, seven human factors (gender, learning abilities, locus of control, attention focus, cognitive strategy and verbal and nonverbal IQs) have been evaluated regarding their impact on interface adjustments done manually by users. People’s preferences for different interface configurations have been investigated. The experimental study suggests criteria for the inclusion of human factors into the user model guiding and controlling the adaptation process. To provide automatic means of adaptation, the Intelligent System for User Modelling has been developed. Elena Zudilova-Seinstra is a Senior Researcher at the Scientific Visualisation and Virtual Reality Group of the University of Amsterdam. Previously, she worked for the Corning Scientific Centre. Apart from being a researcher, in 1999–2002 she was a part-time Assistant Professor at the St. Petersburg Academy of Management Methods and Techniques. She received her M.S. degree in technical engineering in 1993 and Ph.D. in computer science in 1998 from the St. Petersburg State Technical University. In 1996, she received an award for R&D from the Welles-Johnson Foundation of Maryland. She is a Program Committee Member of several International Conferences and Workshops. Her current research interests include multi-modal and adaptive interaction, scientific visualisation, virtual and augmented reality, ambient intelligence and usability studies. She has more than 40 research publications and three editorials in these areas. Also, she has been an INTAS evaluator since February 2005.     

The effects of zoomable user interfaces and user age in searching for a target with a mouse on a two-dimensional information space

A zoomable user interface (ZUI) is a useful function to help users deal with large information spaces displayed within a screen. Although used in many applications, ZUIs have not been sufficiently studied in terms of usability. Usability problems may be more crucial for older people than younger ones. The objective of this study is to examine the effects of a combination of two zoom factors (three zoom focus methods: original-center, re-center, and aiming-point; and two zoom scales: +100% and +400%) for two age groups (younger and older adults) on mouse-based information searching tasks in a map-type two-dimensional information space. Twenty four volunteers (twelve users per age group) participated in the experiments by answering a pair of questions about fifty movie information topics. Task completion time, number of operations, and number of errors were selected as performance measures, and a subjective assessment of satisfaction was collected. Older adults used the tested ZUIs less efficiently and precisely than younger adults. The effects of zoom focus methods vary with zoom scale levels regardless of the age groups. The participants also preferred using the aiming-point focus regardless of zoom scale and using the re-center focus with the +400% scale. We discuss potential implications of the age-related performance differences and the effects of ZUI functions, and suggest some ZUI design guidelines in conclusion. We expect that the findings can be used as basic resources in designing various web services and applications for older computer users.

Relevance to industry

Given that a zoomable user interface is widely used in basic geographic information systems, web services, and various applications, using a selective-focus method such as the aiming-point focus can improve computer users’ usability; in particular, the zooming functions will greatly help older people to explore large information spaces quickly and easily.