User modelling for adaptive computer systems: a survey of recent developments

User modelling is becoming an important sub-area of Artificial Intelligence with both theoretical and practical consequences. The theoretical foundations of user modelling are to be found in key areas of AI, such as knowledge representation and plan recognition, while its practical applications impinge on the construction of intelligent user interfaces and adaptive systems. This paper provides a survey of current work in user modelling. The paper begins by distinguishing between AI approaches, which are the subject of this survey, and those of HCI (Human-Computer Interaction) and then considers the major issues in user modelling such as: types of user modelling system, the sorts of information modelled, how the information is acquired, represented and used. The paper concludes by examining some of the more problematic aspects of user modelling as well as indicating areas for future research.     

User-centered indexing for adaptive information access

We are focusing on information access tasks characterized by large volume of hypermedia connected technical documents, a need for rapid and effective access to familiar information, and long-term interaction with evolving information. The problem for technical users is to build and maintain a personalized task-oriented model of the information to quickly access relevant information. We propose a solution which provides user-centered adaptive information retrieval and navigation. This solution supports users in customizing information access over time. It is complementary to information discovery methods which provide access to new information, since it lets users customize future access to previously found information. It relies on a technique, called Adaptive Relevance Network, which creates and maintains a complex indexing structure to represent personal user's information access maps organized by concepts. This technique is integrated within the Adaptive HyperMan system, which helps NASA Space Shuttle flight controllers organize and access large amount of information. It allows users to select and mark any part of a document as interesting, and to index that part with user-defined concepts. Users can then do subsequent retrieval of marked portions of documents. This functionality allows users to define and access personal collections of information, which are dynamically computed. The system also supports collaborative review by letting users share group access maps. The adaptive relevance network provides long-term adaptation based both on usage and on explicit user input. The indexing structure is dynamic and evolves over time. Learning and generalization support flexible retrieval of information under similar concepts. The network is geared towards more recent information access, and automatically manages its size in order to maintain rapid access when scaling up to large hypermedia space. We present results of simulated learning experiments.Dr. Mathé and Dr. Chen are contractors with Recom Technologies, Inc.     

Adaptive systems: from intelligent tutoring to autonomous agents

Computer systems which can automatically alter aspects of their functionality or interface to suit the needs of individuals or groups of users have appeared over the years in a variety of guises. Most recently, attention has focused on intelligent interface agents, which are seen as specialised, knowledge-based systems acting on behalf of the user in some aspect of the interaction. Similar requirements for automatic adaptation have been noted in intelligent tutoring systems, natural-language systems and intelligent interfaces. The paper brings together the research which has emanated from a number of backgrounds, and provides a unifying perspective on adaptive systems in general. An architecture for adaptive systems and a methodology for their development are presented. The paper also describes software support for producing adaptive systems, and offers some experimental evidence to justify both the desirability and feasibility of exploiting an adaptive system approach to human-computer interaction     

Applying user modeling to human-computer interaction design

Since the early 1980's, intelligent and adaptive systems have appeared and have been written about in a variety of guises. Although there are many differences between such systems as adaptive user interfaces, intelligent help systems and explanation systems, there are many similarities as well. The most notable of these is that they all require user models to be maintained and exploited as part of their designs. Furthermore, they share the same high level architecture. This paper discusses the use of models in human-computer interaction design and offers a common architecture for these adaptive systems. A methodology for the development of these systems is presented.This is based on our experimental work which is reported fully in Jennings, Benyon and Murray, 1991 and Jennings and Benyon, in press)     

On the interaction between plan recognition and intelligent interfaces

Plan recognition is an active research area in automatic reasoning, as well as a promising approach to engineering interfaces that can exploit models of user's plans and goals. Much research in the field has focused on the development of plan recognition algorithms to support particular user/system interactions, such as found in naturally occurring dialogues. However, two questions have typically remained unexamined: 1) exactly what kind of interface tasks can knowledge of a user's plans be used to support across communication modalities, and 2) how can such tasks in turn constrain development of plan recognition algorithms? In this paper we present a concrete exploration of these issues. In particular, we provide an assessment of plan recognition, with respect to the use of plan recognition in enhancing user interfaces. We clarify how use of a user model containing plans makes interfaces more intelligent and interactive (by providing an intelligent assistant that supports such tasks as advice generation, task completion, context-sensitive responses, error detection and recovery). We then show how interface tasks in turn provide constraints that must be satisfied in order for any plan recognizer to construct and represent a plan in ways that efficiently support these tasks. Finally, we survey how interfaces are fundamentally limited by current plan recognition approaches, and use these limitations to identify and motivate current research. Our research is developed in the context of CHECS, a plan-based design interface.     

Design and evaluation of an adaptive icon toolbar

As information systems become increasingly important in many different domains, the potential to adapt them to individual users and their needs also becomes more important. Adaptive user interfaces offer many possible ways to adjust displays and improve procedures for a user's individual patterns of work. This paper describes an attempt to design an adaptive user interface in a computer environment familiar to many users. According to one classification of adaptive user interfaces, the adaptive bar described in this paper would be classified as a user-controlled self-adaptation system.At the user's convenience, the adaptive bar offers suggestions for adding or removing command icons, based on the frequency and probability of specific commands. It also implements these changes once the user has agreed to them. Beyond the adaptive bar, the general behavior of the whole user interface does not change, thereby allowing the user to maintain a clear general model of the system. This paper describes the decision-making algorithm implemented in the bar. It also describes the bar's self-adaptive behavior of displaying the frequency of each icon's use through the icon's size. Finally, we present some encouraging preliminary results of evaluations by users.     

A glass box approach to adaptive hypermedia

Utilising adaptive interface techniques in the design of systems introduces certain risks. An adaptive interface is not static, but will actively adapt to the perceived needs of the user. Unless carefully designed, these changes may lead to an unpredictable, obscure and uncontrollable interface. Therefore the design of adaptive interfaces must ensure that users can inspect the adaptivity mechanisms, and control their results. One way to do this is to rely on the user's understanding of the application and the domain, and relate the adaptivity mechanisms to domain-specific concepts. We present an example of an adaptive hypertext help system POP, which is being built according to these principles, and discuss the design considerations and empirical findings that lead to this design.     

From user interface design to the support of intelligent and adaptive interfaces: an overhaul of user interface software infrastracture

Research on adaptive interfaces in the past has lacked support from user interface tools which allow interfaces to be easily created and modified. Also, current user interface tools provide no support for user models which can collect task-oriented information about users. Developing an adaptive interface requires a user model and an adaptation strategy. It also, however, requires a user interface which can be adapted. The latter task is often time-consuming, especially in relation to more sophisticated user interfaces.

The paper presents a user interface design environment, UIDE, which has a different software infrastracture. Designers use high-level specifications to create a model of an application and links from the application to various interface components. The model is the heart of all the design and run-time support in UIDE, including automatic dialog sequencing and help generation. UIDE provides automatic support for collecting task-oriented information about users, by the use of its high-level specifications in its application model as a basic construct for a user model. Some examples of adaptive interfaces and adaptive help are presented that use the information that is collectable in UIDE.     

The promise of zoomable user interfaces

Zoomable user interfaces (ZUIs) have received a significant amount of attention in the 18 years since they were introduced. They have enjoyed some success, and elements of ZUIs are widely used in computers today, although the grand vision of a zoomable desktop has not materialised. This paper describes the premise and promise of ZUIs along with their challenges. It describes design guidelines, and offers a cautionary tale about research and innovation.     

Modeling decisions under uncertainty in adaptive user interfaces

The present work proposes a methodological approach for modeling adaptation decisions and for solving the problem of integrating existing as well as acquired knowledge in the decision module of an adaptive interface. So far, most applications do not exploit in full the value of data originating from user models or knowledge acquisition engines that monitor the user and the context. The proposed decision theoretic model is represented through specifically structured influence diagrams. It provides to designers and developers a specific method to encode user and context information, as well as other crucial decision factors, to be subsequently used in the decision making process regarding user interface adaptation actions. Such a process is driven by the definition of relevant utilities referring to the design of a user interface. The proposed model guides designers and developers of an adaptive or intelligent interface to integrate, without conflicts and incoherence, design strategies, design goals, user goals, alternative constituents, user profile, context and application domain knowledge. An illustrative example of the analyzed modeling method is presented.

Neuro-cognitively inspired haptic user interfaces

Haptic systems and devices are a recent addition to multimodal systems. These devices have widespread applications such as surgical simulations, medical and procedural training, scientific visualizations, assistive and rehabilitative devices for individuals who have physical or neurological impediments and assistive devices for individuals who are blind. While the potential of haptics in natural human machine interaction is undisputable, the realization of such means is still a long way ahead. There are considerable research challenges to development of natural haptic interfaces. The study of human tactile abilities is a recent endeavor and many of the available systems still do not incorporate the domain knowledge of psychophysics, biomechanics and neurological elements of haptic perception. Development of smart and effective haptic interfaces and devices requires extensive studies that link perceptual phenomena with measurable parameters and incorporation of such domain knowledge in the engineering of haptic interfaces. This paper presents design, development and usability testing of a neuro-cognitively inspired haptic user interface for individuals who are blind. The proposed system design is inspired by neuro-cognitive basis of haptic perception and incorporates the computational aspects and requirements of multimodal information processing system. Usability testing of the system suggests that a biologically inspired haptic user interfaces may form a powerful paradigm for haptic user interface design.

Affectively intelligent and adaptive car interfaces

In this article, we describe a new approach to enhance driving safety via multi-media technologies by recognizing and adapting to drivers’ emotions with multi-modal intelligent car interfaces. The primary objective of this research was to build an affectively intelligent and adaptive car interface that could facilitate a natural communication with its user (i.e., the driver). This objective was achieved by recognizing drivers’ affective states (i.e., emotions experienced by the drivers) and by responding to those emotions by adapting to the current situation via an affective user model created for each individual driver. A controlled experiment was designed and conducted in a virtual reality environment to collect physiological data signals (galvanic skin response, heart rate, and temperature) from participants who experienced driving-related emotions and states (neutrality, panic/fear, frustration/anger, and boredom/sleepiness). k-Nearest Neighbor (KNN), Marquardt-Backpropagation (MBP), and Resilient Backpropagation (RBP) Algorithms were implemented to analyze the collected data signals and to find unique physiological patterns of emotions. RBP was the best classifier of these three emotions with 82.6% accuracy, followed by MBP with 73.26% and by KNN with 65.33%. Adaptation of the interface was designed to provide multi-modal feedback to the users about their current affective state and to respond to users’ negative emotional states in order to decrease the possible negative impacts of those emotions. Bayesian Belief Networks formalization was employed to develop the user model to enable the intelligent system to appropriately adapt to the current context and situation by considering user-dependent factors, such as personality traits and preferences.     

A multi-formalism approach for model-based dynamic distribution of user interfaces of critical interactive systems

Evolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. User Centered Development promotes reactive answers to this kind of evolutions either by software evolutions through iterative development approaches or at runtime by providing additional information to the operators such as contextual help for instance. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. The approach proposed handles both concentration and distribution of user interfaces requiring both fusion of information into a single UI or fission of information into several ones. This generic model-based approach is exemplified on a safety critical system from space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new procedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus+context way improving usability by increasing both efficiency and effectiveness.     

Dynamically reconfigurable vision-based user interfaces

We describe a system that supports practical, vision-based user interfaces, addressing the issues of a usable interaction paradigm, support for application developers, and support for application deployment in real-world environments. Interfaces are defined as configurations of predefined interactive widgets that can be moved from one surface to another. Complex interfaces can be dynamically reconfigured, changing both form and location on the fly, because the functional definition of the interface is decoupled from the specification of its location in the environment. We illustrate the power of such an architecture in the context of projected interactive displays.Published online: 13 July 2004     

Formal Reduction of Interfaces to Large-scale Process Control Systems

A formal methodology is proposed to reduce the amount of information displayed to remote human operators at interfaces to large-scale process control plants of a certain type.The reduction proceeds in two stages.In the first stage,minimal reduced subsets of components,which give full information about the state of the whole system,are generated by determining functional dependencies between components.This is achieved by using a temporal logic proof obligation to check whether the state of all components can be inferred from the state of components in a subset in specified situations that the human operator needs to detect,with respect to a finite state machine model of the system and other human operator behavior.Generation of reduced subsets is automated with the help of a temporal logic model checker.The second stage determines the interconnections between components to be displayed in the reduced system so that the natural overall graphical structure of the system is maintained.A formal definition of an aesthetic for the required subgraph of a graph representation of the full system,containing the reduced subset of components,is given for this purpose. The methodology is demonstrated by a case study.     

A formal model for plastic human computer interfaces

The considerable and significant progress achieved in the design and development of new interaction devices between man and machine has enabled the emergence of various powerful and efficient input and/or output devices. Each of these new devices brings specific interaction modes.With the emergence of these devices, new interaction techniques and modes arise and new interaction capabilities are offered. New user interfaces need to be designed or former ones need to evolve. The design of so called plastic user interfaces contributes to handling such evolutions. The key requirement for the design of such a user interface is that the new obtained user interface shall be adapted to the application and have, at least, the same behavior as the previous (adapted) one. This paper proposes to address the problem of user interface evolution due to the introduction of new interaction devices and/or new interaction modes. More, precisely, we are interested by the study of the design process of a user interface resulting from the evolution of a former user interface due to the introduction of new devices and/or new interaction capabilities. We consider that interface behaviors are described by labelled transition systems and comparison between user interfaces is handled by an extended definition of the bi-simulation relationship to compare user interface behaviors when interaction modes are replaced by new ones.     

Methods and techniques of adaptive hypermedia

Adaptive hypermedia is a new direction of research within the area of adaptive and user model-based interfaces. Adaptive hypermedia (AH) systems build a model of the individual user and apply it for adaptation to that user, for example, to adapt the content of a hypermedia page to the user's knowledge and goals, or to suggest the most relevant links to follow. AH systems are used now in several application areas where the hyperspace is reasonably large and where a hypermedia application is expected to be used by individuals with different goals, knowledge and backgrounds. This paper is a review of existing work on adaptive hypermedia. The paper is centered around a set of identified methods and techniques of AH. It introduces several dimensions of classification of AH systems, methods and techniques and describes the most important of them.     

Augmented reality agents for user interface adaptation

Most augmented reality (AR) applications are primarily concerned with letting a user browse a 3D virtual world registered with the real world. More advanced AR interfaces let the user interact with the mixed environment, but the virtual part is typically rather finite and deterministic. In contrast, autonomous behavior is often desirable in ubiquitous computing (Ubicomp), which requires the computers embedded into the environment to adapt to context and situation without explicit user intervention. We present an AR framework that is enhanced by typical Ubicomp features by dynamically and proactively exploiting previously unknown applications and hardware devices, and adapting the appearance of the user interface to persistently stored and accumulated user preferences. Our framework explores proactive computing, multi‐user interface adaptation, and user interface migration. We employ mobile and autonomous agents embodied by real and virtual objects as an interface and interaction metaphor, where agent bodies are able to opportunistically migrate between multiple AR applications and computing platforms to best match the needs of the current application context. We present two pilot applications to illustrate design concepts. Copyright © 2007 John Wiley & Sons, Ltd.     

On the role of individual human abilities in the design of adaptive user interfaces for scientific problem solving environments

A scientific problem solving environment should be built in such a way that users (scientists) might exploit underlying technologies without a specialised knowledge about available tools and resources. An adaptive user interface can be considered as an opportunity in addressing this challenge. This paper explores the importance of individual human abilities in the design of adaptive user interfaces for scientific problem solving environments. In total, seven human factors (gender, learning abilities, locus of control, attention focus, cognitive strategy and verbal and nonverbal IQs) have been evaluated regarding their impact on interface adjustments done manually by users. People’s preferences for different interface configurations have been investigated. The experimental study suggests criteria for the inclusion of human factors into the user model guiding and controlling the adaptation process. To provide automatic means of adaptation, the Intelligent System for User Modelling has been developed. Elena Zudilova-Seinstra is a Senior Researcher at the Scientific Visualisation and Virtual Reality Group of the University of Amsterdam. Previously, she worked for the Corning Scientific Centre. Apart from being a researcher, in 1999–2002 she was a part-time Assistant Professor at the St. Petersburg Academy of Management Methods and Techniques. She received her M.S. degree in technical engineering in 1993 and Ph.D. in computer science in 1998 from the St. Petersburg State Technical University. In 1996, she received an award for R&D from the Welles-Johnson Foundation of Maryland. She is a Program Committee Member of several International Conferences and Workshops. Her current research interests include multi-modal and adaptive interaction, scientific visualisation, virtual and augmented reality, ambient intelligence and usability studies. She has more than 40 research publications and three editorials in these areas. Also, she has been an INTAS evaluator since February 2005.     

Distributed user interfaces in public spaces using RFID-based panels

The combination and integration of services between mobile computing and context-aware applications responds to the use of mobile devices defining a wide range of distributed user interfaces to support social activities. In this paper, we propose a novel solution that combines social software with context awareness to improve users' interaction in public spaces. This approach is based on the concept of collaborative interactive panels where users share their opinions and ideas about environmental issues by performing natural gestures. And so, taking advantage of physical resources already available in public spaces combined with the use of well-known technologies, such as mobile devices and RFID, we extend the concept of social software from the Web to physical public scenarios, such as bus stations, squares, etc. As an example, we present a case of study that encourage citizens' participation in decisions related to the community environmental issues reducing the gap between the social software and users.