THE EMPATHIC COMPANION: A CHARACTER-BASED INTERFACE THAT ADDRESSES USERS' AFFECTIVE STATES

ABSTRACT

In this paper, we report on our efforts in developing affective character-based interfaces, i.e., interfaces that recognize and measure affective information of the user and address user affect by employing embodied characters. In particular, we describe the Empathic Companion, an animated interface agent that accompanies the user in the setting of a virtual job interview. This interface application takes physiological data (skin conductance and electromyography) of a user in realtime, interprets them as emotions, and addresses the user's affective states in the form of empathic feedback. The Empathic Companion is conceived as an educational agent that supports job seekers preparing for a job interview.

We also present results from an exploratory study that aims to evaluate the impact of the Empathic Companion by measuring users' skin conductance and heart rate. While an overall positive effect of the Empathic Companion could not be shown, the outcome of the experiment suggests that empathic feedback has a positive effect on the interviewee's stress level while hearing the interviewer question.     

The effects of 3D interface metaphor on older adults’ mobile navigation performance and subjective evaluation

Navigating vast information spaces through mobile interfaces has become a common activity in older adults' everyday lives. Studies suggested that interface metaphors could be used to facilitate users' metal model development and information processing when using mobile technologies. However, we know little about how metaphors affect older adults' mobile navigation behavior, and which user characteristics matter during this perceptual and cognitive process. To investigate this, a card interface with a 3D metaphor and a list interface without 3D metaphors were compared among twenty-two participants when performing four navigation tasks. User characteristics including demographic factors, technology experience, and user capabilities were examined. The participants' navigation performance and subjective evaluations were measured as the dependent variables. From the results, we recommend the list interface without 3D metaphors as a beneficial choice for older adults. It performed better in navigation performance, although the differences are not statistically significant. Moreover, navigation performance using the card interface with a 3D metaphor was significantly associated with participants' perceptual speed, thus this interface may be more sensitive to capability declines. Valuable insights into the older adults’ mobile navigation performance and preferences are discussed and important implications for the design of mobile navigation user interfaces are proposed based on the results.Relevance to industryThe experimental results propose a more beneficial way to present contents on a mobile user interface for older adults and provide valuable insights for the designers and industry to help them understand the older adults’ usage and perceptions towards the application of 3D metaphors when navigating with mobile interfaces.     

A Dynamic Gesture Language and Graphical Feedback for Interaction in a 3D User Interface

In user interfaces of modern systems, users get the impression of directly interacting with application objects. In 3D based user interfaces, novel input devices, like hand and force input devices, are being introduced. They aim at providing natural ways of interaction. The use of a hand input device allows the recognition of static poses and dynamic gestures performed by a user's hand. This paper describes the use of a hand input device for interacting with a 3D graphical application. A dynamic gesture language, which allows users to teach some hand gestures, is presented. Furthermore, a user interface integrating the recognition of these gestures and providing feedback for them, is introduced. Particular attention has been spent on implementing a tool for easy specification of dynamic gestures, and on strategies for providing graphical feedback to users' interactions. To demonstrate that the introduced 3D user interface features, and the way the system presents graphical feedback, are not restricted to a hand input device, a force input device has also been integrated into the user interface.     

A study of usability principles and interface design for mobile e-books

This study examined usability principles and interface designs in order to understand the relationship between the intentions of mobile e-book interface designs and users' perceptions. First, this study summarised 4 usability principles and 16 interface attributes, in order to conduct usability testing and questionnaire survey by referring to Nielsen (1993), Norman (2002), and Yeh (2010), who proposed the usability principles. Second, this study used the interviews to explore the perceptions and behaviours of user operations through senior users of multi-touch prototype devices. The results of this study are as follows: (1) users' behaviour of operating an interactive interface is related to user prior experience; (2) users' rating of the visibility principle is related to users' subjective perception but not related to user prior experience; however, users' ratings of the ease, efficiency, and enjoyment principles are related to user prior experience; (3) the interview survey reveals that the key attributes affecting users' behaviour of operating an interface include aesthetics, achievement, and friendliness.     

Visual-haptic feedback interaction in automotive touchscreens

Touchscreen interfaces offer benefits in terms of flexibility and ease of interaction and as such their use has increased rapidly in a range of devices, from mobile phones to in-car technology. However, traditional touchscreens impose an inevitable visual workload demand that has implications for safety, especially in automotive use. Recent developments in touchscreen technology have enabled feedback to be provided via the haptic channel. A study was conducted to investigate the effects of visual and haptic touchscreen feedback on visual workload, task performance and subjective response using a medium-fidelity driving simulator. Thirty-six experienced drivers performed touchscreen ‘search and select’ tasks while engaged in a motorway driving task. The study utilised a 3 × 2 within-subjects design, with three levels of visual feedback: ‘immediate’, ‘delayed’, ‘none’; and two levels of haptic feedback: ‘visual only’, ‘visual + haptic’. Results showed that visual workload was increased when visual feedback was delayed or absent; however, introducing haptic feedback counteracted this effect, with no increases observed in glance time and count. Task completion time was also reduced when haptic feedback was enabled, while driving performance showed no effect due to feedback type. Subjective responses indicated that haptic feedback improved the user experience and reduced perceived task difficulty.     

Perceived interactivity leading to e-loyalty: Development of a model for cognitive–affective user responses

Novel applications of website interactivity are important to attract and retain online users. In this empirical study five designs for interactivity are examined using different web-poll interfaces. The goal of the investigation is to examine perceived interactivity in a model which includes most commonly tested cognitive elements such as efficiency and effectiveness, but augments this model with the inclusion of a cognitive–affective element for trust, and an affective element of enjoyment. More specifically, a model is created to validate the relationship of perceived interactivity (comprised of user control, user connectedness, and responsiveness of the web-poll application) to efficiency, effectiveness, trust and enjoyment, of the website. In turn, efficiency, effectiveness, trust, and enjoyment are tested for their influence on user behavioral intentions for e-loyalty. All relationships in the model are supported. In addition, exploratory evaluation of qualitative comments is conducted to investigate additional insights between the five web-poll treatments in this investigation. The research confirms the complexity of a model in which cognitive, cognitive–affective and affective elements are present, and advances knowledge on the consequences of perceived interactivity. In additional to theoretical advancements, the research has merit for web designers and online marketers regarding how to enhance interactive online web applications.     

Mobile health monitoring for the elderly: Designing for diversity

In the field of mobile health monitoring the current most important user groups are those aged 50 +. In our project senSAVE^® we developed a user interface for monitoring vital personal parameters that is specifically adapted to the needs of this group. The group is said to show less perception and control capability and has less experience in the use of information technology. More realistically, the group of 50 + users shows more diversity in their cognitive, sensory and motor skills than younger people. User interfaces for elderly people should therefore be designed for different capabilities and needs. For a mobile health monitoring system three design types were developed and evaluated in this study: three basic interfaces, two professional interfaces and an interface in between as a compromise of the two former types. Beyond monitoring the vital parameters of the user during mobile phases a stationary module for the inspection of aggregated data was included on a standard TV set together with a remote control device. The paper describes the user interface design and a comparative evaluation of the interfaces during and after the development, i.e. formative and summative evaluation. It also describes first user feedback about the stationary component.     

A home-page overhaul using other Web sites

The best prototype for designing a new user interface is your old user interface. The second best prototype is a competing product. Your competitors have invested significant resources in designing and implementing what they believe to be good user interfaces. You can glean much of what you need to create a new interface by examining products designed to solve similar problems. As with your own old user interface, you can analyze competing interfaces to see what works and what doesn't. You can also watch how users interact with competing products, and thus learn how they approach tasks. This, in essence, is competitive usability analysis. I recommend performing it very early in the usability engineering life-cycle-after you have visited the customer, gathered requirements, and defined the product vision, but before you design and prototype your new user interface     

Exploring the Critical Factors Influencing the Quality of Blog Interfaces Using the Decision-Making Trial and Evaluation Laboratory (DEMATEL) Method

Many studies have shown the significance of a good user interface in affecting a blogger's decision to select a platform for their blog and a reader's decision to read one, and so it is vital that both developers and bloggers need a greater understanding of how they can improve user experience through perfecting their blog interfaces. The aim of this research is to explore the critical factors influencing the quality of blog interfaces and the causal relationships between these factors, enabling blog interfaces to be designed more effectively. Using an approach combining a focus group and the Decision-Making Trial and Evaluation Laboratory (DEMATEL), this study defined eight factors in four dimensions that influence blog interface quality. The results of the DEMATEL analysis identify the key causal factors and effect factors, as well as the causal relationships between the eight factors via the impact-relations map. The research also indicates the most critical causal factors that bloggers and developers should focus on, in order to most effectively improve the quality and attractiveness of blog interfaces.     

Head gestures for perceptual interfaces: The role of context in improving recognition

Head pose and gesture offer several conversational grounding cues and are used extensively in face-to-face interaction among people. To accurately recognize visual feedback, humans often use contextual knowledge from previous and current events to anticipate when feedback is most likely to occur. In this paper we describe how contextual information can be used to predict visual feedback and improve recognition of head gestures in human–computer interfaces. Lexical, prosodic, timing, and gesture features can be used to predict a user's visual feedback during conversational dialog with a robotic or virtual agent. In non-conversational interfaces, context features based on user–interface system events can improve detection of head gestures for dialog box confirmation or document browsing. Our user study with prototype gesture-based components indicate quantitative and qualitative benefits of gesture-based confirmation over conventional alternatives. Using a discriminative approach to contextual prediction and multi-modal integration, performance of head gesture detection was improved with context features even when the topic of the test set was significantly different than the training set.     

Haptic User Interfaces for the Visually Impaired: Implications for Haptically Enhanced Science Learning Systems

The overall quality of haptic user interfaces designed to support visually impaired students' science learning through sensorial feedback was systematically studied to investigate task performance and user behavior. Fourteen 6th- to 11th-grade students with visual impairments recruited from a state-funded blind school were asked to perform three main tasks (i.e., menu selection, structure exploration, and force recognition) using haptic user interfaces and a haptic device. This study used several dependent measures that are categorized into three types of variables: (a) task performance including success rate, workload, and task completion time; (b) user behavior defined as cursor movements proportionately represented from the user's cursor positional data; and (c) user preference. Results showed that interface type has significant effects on task performance, user behavior, and user preference, with varying degree of impact to participants with severe visual impairments performing the tasks. The results of this study as well as a set of refined design guidelines and principles should provide insights to the future research of haptic user interfaces that can be used when developing haptically enhanced science learning systems for the visually impaired.     

Modeling users' task performance on the mobile device: PC convergence system

This study aims to establish a model-based approach for user interface design that simultaneously considers the system's information hierarchy, users' task procedure knowledge, and system interfaces. The approach is based on a framework that contains multiple interaction models to express both system elements and users' knowledge. The framework evaluates system interface through the interaction between user's knowledge on interface, task procedure and information structure perceived by the user in the system. The interface is evaluated by its contribution to the users' task performance and system navigation.These three factors were defined as design factors that affect users' task performance. Through the crosscheck process of models, the relation between information, interface, and task procedure is calculated into combined difficulty index (CDI) that expresses the difficulty of a system interface that users would experience during the use of system. A user test was conducted for the validation of the CDI. The difficulties of the interface of a mobile healthcare system were predicted with the CDI, and the predictions were compared with the experimental results, where the users' performance showed consistence with the prediction.     

Measuring perceived interactivity of mobile advertisements

Interactivity is considered a prominent characteristic of mobile advertising; however, a properly developed tool to measure the concept has not been developed. This study focusses on the development and validation of an instrument for measuring the interactivity perceived by individuals who interact with mobile advertisements on handheld devices. The confirmatory factor analysis results show that perceived interactivity in mobile advertising is comprised of six correlated but distinct constructs: user control, synchronicity, two-way communication, connectedness, playfulness and interpersonal communication. The result from a series of tests and examinations indicated that the multidimensional scale exhibits content validity and discriminant validity. The internal consistency of the scale reaches an acceptable level. The relationship with personal variables was also analysed. Gender, education, monthly income, years using cell phones, monthly cell phone charges, monthly cost for data services on cell phones and previous experiences with mobile advertisements have no influences on respondent interactivity perception with mobile advertisements. But user age is negatively correlated with perceived interactivity and years using the Internet are positively correlated with perceived interactivity. Young people with more Internet experience are more likely to perceive interactivity of mobile advertisements.     

How does Parallax Scrolling Influence User Experience? A Test of TIME (Theory of Interactive Media Effects)

Parallax scrolling is a popular web technique used widely in product presentation websites, but its effect on user experience is unknown. Parallax scrolling allows the object displayed in the foreground to move faster than the object displayed in the background, so as to create a 3D effect. Dimensionality and scrolling (modality interactivity) are two major characteristics of this technique. A 2 (dimension) × 2 (modality interactivity) between-subjects (N = 295) online experiment revealed that parallax scrolling influences user engagement through two routes: the cue route (perceived coolness, perceived vividness) and the action route (natural mapping, perceived ease of use). User engagement is in turn associated with positive attitudes and behavioral intentions toward the website and the product. The theoretical contribution of the study is it examined interface feature of parallax scrolling on both user perceptions and user actions using the theory of interactive media effects (TIME). The practical contribution of the study is it provides a roadmap to web developers to emphasize affective factors (i.e., vividness, coolness) rather than cognitive factors (i.e., natural mapping, perceived usefulness) when it comes to improving the user experience of the product presentation websites.     

Tangible user interfaces for physically-based deformation: design principles and first prototype

We present design principles for conceiving tangible user interfaces for the interactive physically-based deformation of 3D models. Based on these design principles, we developed a first prototype using a passive tangible user interface that embodies the 3D model. By associating an arbitrary reference material with the user interface, we convert the displacements of the user interface into forces required by physically-based deformation models. These forces are then applied to the 3D model made out of any material via a physical deformation model. In this way, we compensate for the absence of direct haptic feedback, which allows us to use a force-driven physically-based deformation model. A user study on simple deformations of various metal beams shows that our prototype is usable for deformation with the user interface embodying the virtual beam. Our first results validate our design principles, plus they have a high educational value for mechanical engineering lectures.     

A novel mobile device user interface with integrated social networking services

Modern mobile devices support accessing Web-based social networking services from the user interface (UI) of Web browsers, applications, and mobile widgets. While effectively accessing these services, people may find it tedious to switch between multiple user interfaces in order to be aware of the latest content. Aiming for an improved user experience, we experimented with integration of these services into mobile devices' main user interface. The integrated content is presented beyond application silos and automatically filtered to highlight the relevant elements. A mobile system called LinkedUI was developed and deployed in one lab test and one field study. Three findings emerge from these studies. Firstly, it is feasible to construct an alternative device UI that supports integration of Web content across applications and services via hyperlinking. Time, publisher (e.g., contacts), content types, and geographical locations are key dimensions for association of content. Secondly, the alternative device UI enables better usability of accessing social networking services than accessing them from individual Web sites on mobile devices. It helps people to be aware of the latest content during microbreaks. Thirdly, automatic filtering, on the basis of one user's data, is one promising approach to identifying relevant content. Given filtered content, most people using the automatic filtering approved the functionality and experienced a better sense of control that is arguably due to the reduced information volume.     

The effect of visual feedback types on the wait indicator interface of a mobile application

This study examines the essential effects of visual feedback on users’ perceptions of time and on their evaluations of waiting time for a mobile application user interface. The two variables adopted in this experiment are wait indicator visual feedback and waiting time. Three types of visual feedback were used: bar indicator, pie indicator, and cartoon indicator. Three waiting times were used: 2 s, 5 s, and 10 s. This study was conducted based on the attentional-gate model of Zakay and Block to help investigate user time perceptions. The experiment of this study was a 3 × 3 mixed factorial design to measure the impact of the research variables on the users’ perceptions of waiting time, wait evaluations, and preferences. Results showed that the users’ time perceptions and wait evaluations were affected by the feedback types and waiting times, and the results are consistent with Vierordt's law and the attentional-gate model. Furthermore, there was a positive impact on the participants' preferences when they perceived a shorter waiting time. The cartoon type feedback can help reduce the perception of waiting time, but it does not necessarily improve user experience regarding short waiting time.     

Sisl: Several Interfaces, Single Logic

Modern interactive services such as information and e-commerce services are becoming increasingly more flexible in the types of user interfaces they support. These interfaces incorporate automatic speech recognition and natural language understanding and include graphical user interfaces on the desktop and web-based interfaces using applets and HTML forms. To what extent can the user interface software be decoupled from the service logic software (the code that defines the essential function of a service)? Decoupling of user interface from service logic directly impacts the flexibility of services, or how easy they are to modify and extend. To explore these issues, we have developed Sisl, an architecture and domain-specific language for designing and implementing interactive services with multiple user interfaces. A key principle underlying Sisl is that all user interfaces to a service share the same service logic. Sisl provides a clean separation between the service logic and the software for a variety of interfaces, including Java applets, HTML pages, speech-based natural language dialogue, and telephone-based voice access. Sisl uses an event-based model of services that allows service providers to support interchangeable user interfaces (or add new ones) to a single consistent source of service logic and data. As part of a collaboration between research and development, Sisl is being used to prototype a new generation of call processing services for a Lucent Technologies switching product.     

Emotion recognition from physiological signals using wireless sensors for presence technologies

In this article we describe a new approach to enhance presence technologies. First, we discuss the strong relationship between cognitive processes and emotions and how human physiology is uniquely affected when experiencing each emotion. Secondly, we introduce our prototype multimodal affective user interface. In the remainder of the paper we describe the emotion elicitation experiment we designed and conducted and the algorithms we implemented to analyse the physiological signals associated with emotions. These algorithms can then be used to recognise the affective states of users from physiological data collected via non-invasive technologies. The affective intelligent user interfaces we plan to create will adapt to user affect dynamically in the current context, thus providing enhanced social presence.     

Affective experience of physical user interfaces: Similarities and differences among control types

This study aims to determine similarities and differences in affective experience when manipulating 3 physical user interfaces (PUIs): button, touch screen, and jog dial inputs. First, we conducted literature and expert reviews to identify key affective experiences of PUI manipulation. As a result, seven affective experiences (Softness, Elasticity, Heaviness, Smoothness, Sharpness, Clarity of Feedback, and Rattle) were identified, and they were categorized into two dimensions (Force and Feedback Vibration). Three experiments, including 51 types of buttons, 25 synthesized stimuli for touch screen, and 51 types of jog dials, were then conducted to determine the relationship between affective experiences, satisfaction, and physical parameters of each PUI. According to the results, it was common for all three PUIs that Clarity of Feedback was the most important factor for satisfaction. However, there were also differences between the effects of affective experiences and physical parameters. User satisfaction on manipulating jog dials was found to be less affected by affective experiences than buttons and touch screens. Among physical parameters, the strength of feedback was the most important factor for the affective experience of buttons and touch screens, whereas the period of feedback was the most important factor for jog dials. These similarities and differences could be explained by cognitive and manipulative factors (e.g., attention level, types of feedback, and hand posture of manipulation). The results of this study can help researchers and practitioners better understand and enhance the affective aspects of PUI manipulations.