Towards end-user development of distributed user interfaces

Universal Access in the Information Society - Over the last decade, we have seen an increasing number of solutions for distributed user interfaces (DUIs). This paper provides a detailed overview of...     

Towards automatic evaluation of multimodal user interfaces

The evaluation of the usability and the learnability of a computer system may be performed with predictive models during the design phase. It may be done on the executable code as well as by observing the user in action. In the latter case, data collected in vivo must be processed. The goal is to provide software supports for performing this difficult and time consuming task.

The paper presents an early analysis of, and experience relating to, the automatic evaluation of multimodal user interfaces. With this end in view, a generic Wizard of Oz platform has been designed to allow the observation and automatic recording of subjects' behavior while they interact with a multimodal interface. It is then shown how recorded data can be analyzed to detect behavioral patterns, and how deviations of such patterns from a data-flow-oriented task model can be exploited by a software usability critic.     

Towards portable natural language interfaces based on case-based reasoning

Natural Language Interfaces allow non-technical people to access information stored in Knowledge Bases keeping them unaware of the particular structure of the model or the underlying formal query language. Early research in the field was devoted to improve the performance of a particular system for a given Knowledge Base. Since adapting the system to new domains usually entailed considerable effort, investigating how to bring Portability to NLI became a new challenge. In this article, we investigate how Case-Based Reasoning could serve to assist the expert in porting the system so as to improve its retrieval performance. Our method HITS is based on a novel grammar learning algorithm combined with language acquisition techniques that exploit structural analogies. The learner (system) is able to engage the teacher (expert) with clarification dialogues to validate conjectures (hypotheses and deductions) about the language. Our method presents the following advantages: (i) the customization is naturally defined in the case-based cycle, (ii) the types of questions the system can deal with are not delimited in advance, and (iii) the system ‘reasons’ about precedent cases to deal with unseen questions.     

Towards a model-driven approach for multiexperience AI-based user interfaces

Software and Systems Modeling - Software systems start to include other types of interfaces beyond the “traditional” Graphical-User Interfaces (GUIs). In particular, Conversational User...     

Designing for augmented attention: Towards a framework for attentive user interfaces

Attentive user interfaces are user interfaces that aim to support the user’s attentional capacities. By sensing the users’ attention for objects and people in their everyday environment, and by treating user attention as a limited resource, these interfaces avoid today’s ubiquitous patterns of interruption. Focusing upon attention as a central interaction channel allows development of more sociable methods of communication and repair with ubiquitous devices. Our methods are analogous to human turn taking in group communication. Turn taking improves the user’s ability to conduct foreground processing of conversations. Attentive user interfaces bridge the gap between the foreground and periphery of user activity in a similar fashion, allowing users to move smoothly in between.We present a framework for augmenting user attention through attentive user interfaces. We propose five key properties of attentive systems: (i) to sense attention; (ii) to reason about attention; (iii) to regulate interactions; (iv) to communicate attention and (v) to augment attention.     

Fostering trust in virtual project teams: Towards a design framework grounded in a TrustWorthiness ANtecedents (TWAN) schema

Several collaboration problems in virtual project teams that work in knowledge-intensive contexts can be attributed to a hampered process of interpersonal trust formation. Solutions to trust formation problems need to be based on an understanding of how interpersonal trust forms in face-to-face project teams as well as on insight into how this process differs in virtual teams. Synthesizing literature from various disciplines, we propose a model for the formation of interpersonal trust between project team members. Taking this model as a starting point, we analyse how virtual settings may alter or even obstruct the process of trust formation. One method to improve the formation of interpersonal trust in virtual settings is to facilitate the assessment of trustworthiness. This can be done by making information available about individual virtual project team members. Previous research in virtual project teams focussed principally on the medium by which information is spread, for example, by phone, mail, or videoconferencing. Most researchers failed to take the specific content of the information into account, although there is general agreement that personal, non-task-related information is important to foster trust. For this, we propose to use the antecedents of trustworthiness, which until now have mainly been used as a framework to measure trust, as a design framework instead. This framework of antecedents can also be used to determine which type of information is relevant to assess each other’s trustworthiness. We review existing literature on the antecedents of trustworthiness and extend the well-accepted antecedents of ‘ability’, ‘benevolence’ and ‘integrity’ with several other antecedents, such as ‘communality’ and ‘accountability’. Together, these form the TrustWorthiness ANtecedents (TWAN) schema. We describe how these antecedents can be used to determine which information is relevant for team members assessing others’ trustworthiness. In future research we will first verify this extended cognitive schema of trustworthiness (TWAN) empirically and then apply it to the design of artefacts or guidelines, such as a personal identity profile to support the assessment of trustworthiness in virtual project teams.     

Towards building intelligent speech interfaces through the use of more flexible,robust and natural dialogue management solutions

In this paper a Bayesian Networks-based solution for dialogue modelling is presented. This solution is combined with carefully designed contextual information handling strategies. With the purpose of validating these solutions, and introducing a spoken dialogue system for controlling a Hi-Fi audio system as the selected prototype, a real-user evaluation has been conducted. Two different versions of the prototype are compared. Each version corresponds to a different implementation of the algorithm for the management of the actuation order, the algorithm for deciding the proper order to carry out the actions required by the user. The evaluation is carried out in terms of a battery of both subjective and objective metrics collected from speakers interacting with the Hi-Fi audio box through predefined scenarios. Defined metrics have been specifically adapted to measure: first, the usefulness and the actual relevance of the proposed solutions, and, secondly, their joint performance through their intelligent combination mainly measured as the level achieved with regard to the user satisfaction. A thorough and comprehensive study of the main differences between both approaches is presented. Two-way analysis of variance (ANOVA) tests are also included to measure the effects of both: the system used and the type of scenario factors, simultaneously. Finally, the effect of bringing this flexibility, robustness and naturalness into our home dialogue system is also analyzed through the results obtained. These results show that the intelligence of our speech interface has been well perceived, highlighting its excellent ease of use and its good acceptance by users, therefore validating the approached dialogue management solutions and demonstrating that a more natural, flexible and robust dialogue is possible thanks to them.     

Multimodal interfaces

In this paper, we present an overview of research in our laboratories on Multimodal Human Computer Interfaces. The goal for such interfaces is to free human computer interaction from the limitations and acceptance barriers due to rigid operating commands and keyboards as the only/main I/O-device. Instead we move to involve all available human communication modalities. These human modalities include Speech, Gesture and Pointing, Eye-Gaze, Lip Motion and Facial Expression, Handwriting, Face Recognition, Face Tracking, and Sound Localization.     

Body-based interfaces

This research explores different ways to use features of one's own body for interacting with computers. Such "body-based" interfaces may find good uses in wearable computing or virtual reality systems as part of a 3D multi-modal interface in the future, freeing the user from holding interaction devices. Four types of body-based interfaces have been identified: Body-inspired metaphor (BIM); Body-as-interaction-surface (BAIS); Mixed mode (MM); and Object mapping (OM). These four body-based interfaces were applied to a few different applications (and associated tasks) and were tested for their performance and preference. It was generally found that, among the four, the BIM exhibited low error rates, but produced relatively longer task completion times and significant fatigue. The BAIS method had the contrasting character of higher error rates, but shorter task completion times and lower intuitiveness. The OM method exhibited high error rates, longer completion times, and much fatigue. Overall, the MM was superior in terms of both performance and preference as it combined the merits of the above three methods. Thus, it is expected, for applications with many associated tasks, a careful division of tasks among those that have natural semantic links to body parts and those that do not, is necessary to design the most performing body-based interface.     

Shape-changing interfaces

The design of physical interfaces has been constrained by the relative akinesis of the material world. Current advances in materials science promise to change this. In this paper, we present a foundation for the design of shape-changing surfaces in human–computer interaction. We provide a survey of shape-changing materials and their primary dynamic properties, define the concept of soft mechanics within an HCI context, and describe a soft mechanical alphabet that provides the kinetic foundation for the design of four design probes: Surflex, SpeakCup, Sprout I/O, and Shutters. These probes explore how individual soft mechanical elements can be combined to create large-scale transformable surfaces, which can alter their topology, texture, and permeability. We conclude by providing application themes for shape-changing materials in HCI and directions for future work.     

Expressive interfaces

Analysis of expressiveness in human gesture can lead to new paradigms for the design of improved human-machine interfaces, thus enhancing users participation and experience in mixed reality applications and context-aware mediated environments. The development of expressive interfaces decoding the highly affective information gestures convey opens novel perspectives in the design of interactive multimedia systems in several application domains: performing arts, museum exhibits, edutainment, entertainment, therapy, and rehabilitation. This paper describes some recent developments in our research on expressive interfaces by presenting computational models and algorithms for the real-time analysis of expressive gestures in human full-body movement. Such analysis is discussed both as an example and as a basic component for the development of effective expressive interfaces. As a concrete result of our research, a software platform named EyesWeb was developed (). Besides supporting research, EyesWeb has also been employed as a concrete tool and open platform for developing real-time interactive applications.     

On interfaces

The nature of interfaces is discussed. It is recommended that modern software approaches to interfaces (such as abstract data types) be extended to interfaces of all kinds. These interfaces can and should be developed at the level of abstraction appropriate to the interacting elements, rather than at the level of their lowest common denominator, which is frequently the case. It is proposed that for an interface to exist at any level of abstraction, there must be an operational framework that implements that level of abstraction and within which the interfacing elements interact. Four tasks required for the development of interfaces are described. The traditional classification of interfaces as hardware/software, software/software, and hardware/hardware is found wanting and a new partitioning is suggested—structural or interpretive interfaces, symbolic interfaces, and physical interfaces. Standards and scientific and engineering theories are shown to define levels of abstraction for general interfaces in much the same way that programming languages do for software interfaces. Implications for software engineering and concurrent engineering are explored.     

Intelligent interfaces

The exchange of data and control information between man and machine or between various modules in a complex manufacturing system can either be done through standardized or specially defined interfaces. For either of the two ways applied, all data channels should be well defined and rigorously implemented otherwise proper functional operation cannot be secured. In the present practice this is the most formidable obstacle when creating flexible control systems.This paper surveys four approaches to establish more flexible, more intelligent interfaces. The self-adaptive interface is independent from the user and from the system as well. The extended man-machine interface is based on the roles of human communication. The interfaces in a data base environment provide indirect links through common media. The control and communication functions can be realized by a distributed architecture of an artificial intelligence system.     

Spatial learning with perceptually grounded representations

The goal of this paper is to develop the foundation for a spatial navigation without objective representations. Rather than building the spatial representations on a Euclidean space, a weaker conception of space is used. A type of spatial representation is described that uses perceptual information directly to define the regions in space. By combining such regions, it is possible to derive a number of useful spatial representations such as place-fields, paths and topological maps. Compared to other methods, the representations of the present approach have the advantage that they are always grounded in the perceptual abilities of the robot.     

Reengineering user interfaces

Most developers would like to avoid redesigning a system around a new interface. But turning a character-based interface into a graphical one requires significant time and resources. The authors describe how this process can be partially automated, giving the results of their own reverse-engineering effort     

Development of a gaze-controlled support system for a person in an advanced stage of multiple sclerosis: a case study

Universal Access in the Information Society - A gaze-controlled system was developed to address the specific needs of a person in an advanced stage of Multiple Sclerosis. This patient scored 9 on...     

Standardized protocol interfaces

A traditional protocol implementation typically consists of at least two distinct parts, a sender and a receiver. Each part runs on a distinct machine, with the implementation provided by a local expert. At best, the two machines are of the same type and the protocol implementations are provided by the same person. More likely, however, the machines are not of the same type and the implementations of the two halves of the protocol are provided by two different people, working from an often loosely defined protocol specification. It seems almost unavoidable that the two implementations are not quite compatible. In this paper we consider an alternative technique. With this method, one of the two implementors can design, formally validate, and implement all the relevant protocol parts, including those parts that are to be executed remotely. Each communication channel is now terminated on the receiving side, by a single standard protocol interface, which can be called a universal asynchronous protocol interface, or UAPI. Though it is likely that the UAPI is most efficiently implemented in hardware, it can also trivially run as a software module, e.g. under a standard UNIX® operating system (in our case under 10th Edition Research Unix). This paper introduces the concept of a UAPI and explains how the sample software controller was constructed.