The effects of multimodal feedback on older adults' task performance given varying levels of computer experience

This experiment examines the effect that computer experience and various combinations of feedback (auditory, haptic, and/or visual) have on the performance of older adults completing a drag-and-drop task on a computer. Participants were divided into three computer experience groups, based on their frequency of use and breadth of computer knowledge. Each participant completed a series of drag-and-drop tasks under each of seven feedback conditions (three unimodal, three bimodal, one trimodal). Performance was assessed using measures of efficiency and accuracy. Experienced users responded well to all multimodal feedback while users without experience responded well to auditory-haptic bimodal, but poorly to haptic-visual bimodal feedback. Based on performance benefits for older adults seen in this experiment, future research should extend investigations to effectively integrate multimodal feedback into GUI interfaces in order to improve usability for this growing and diverse user group.     

Effects of haptic feedback,stereoscopy, and image resolution on performance and presence in remote navigation

Traditionally, the main goal of teleoperation has been to successfully achieve a given task as if performing the task in local space. An emerging and related requirement is to also match the subjective sensation or the user experience of the remote environment, while maintaining reasonable task performance. This concept is often called “presence” or “(experiential) telepresence,” which is informally defined as “the sense of being in a mediated environment.” In this paper, haptic feedback is considered as an important element for providing improved presence and reasonable task performance in remote navigation. An approach for using haptic information to “experientially” teleoperate a mobile robot is described. Haptic feedback is computed from the range information obtained by a sonar array attached to the robot, and delivered to a user's hand via a haptic probe. This haptic feedback is provided to the user, in addition to stereoscopic images from a forward-facing stereo camera mounted on the mobile robot. The experiment with a user population in a real-world environment showed that haptic feedback significantly improved both task performance and user-felt presence. When considering user-felt presence, no interaction among haptic feedback, image resolution, and stereoscopy was observed, that is, haptic feedback was effective, regardless of the fidelity of visual elements. Stereoscopic images also significantly improved both task performance and user-felt presence, but high-resolution images only significantly improved user-felt presence. When considering task performance, however, it was found that there was an interaction between haptic feedback and stereoscopy, that is, stereoscopic images were only effective when no force feedback was applied. According to the multiple regression analysis, haptic feedback was a higher contributing factor to the improvement in performance and presence than image resolution and stereoscopy.     

The influence of different sensory cues as selection feedback and co-location in presence and task performance

For some applications based on virtual reality technology, presence and task performance are important factors to validate the experience. Different approaches have been adopted to analyse the extent to which certain aspects of a computer-generated environment may enhance these factors, but mainly in 2D graphical user interfaces. This study explores the influence of different sensory modalities on performance and the sense of presence experienced within a 3D environment. In particular, we have evaluated visual, auditory and active haptic feedback for indicating selection of virtual objects. The effect of spatial alignment between proprioceptive and visual workspaces (co-location) has also been analysed. An experiment has been made to evaluate the influence of these factors in a controlled 3D environment based on a virtual version of the Simon game. The main conclusions obtained indicate that co-location must be considered in order to determine the sensory needs during interaction within a virtual environment. This study also provides further evidence that the haptic sensory modality influences presence to a higher extent, and that auditory cues can reduce selection times. Conclusions obtained provide initial guidelines that will help designers to set out better selection techniques for more complex environments, such as training simulators based on VR technology, by highlighting different optimal configurations of sensory feedback.     

Pedestrian navigation using the sense of touch

Haptics is a feedback technology that takes advantage of the human sense of touch by applying forces, vibrations, and/or motions to a haptic-enabled user device such as a mobile phone. Historically, human–computer interaction has been visual, data, or images on a screen. Haptic feedback can be an important modality in Mobile Location-Based Services like – knowledge discovery, pedestrian navigation and notification systems. In this paper we describe a methodology for the implementation of haptics in four distinct prototypes for pedestrian navigation. Prototypes are classified based on the user’s navigation guidance requirements, the user type (based on spatial skills), and overall system complexity. Here haptics is used to convey location, orientation, and distance information to users using pedestrian navigation applications. Initial user trials have elicited positive responses from the users who see benefit in being provided with a “heads up” approach to mobile navigation. We also tested the spatial ability of the user to navigate using haptics and landmark images based navigation. This was followed by a test of memory recall about the area. Users were able to successfully navigate from a given origin to a Destination Point without the use of a visual interface like a map. Results show the users of haptic feedback for navigation prepared better maps (better memory recall) of the region as compared to the users of landmark images based navigation.     

A user study of command strategies for mobile robot teleoperation

This article presents a user study of mobile robot teleoperation. Performance of speed, position and combined command strategies in combination with text, visual and haptic feedback information were evaluated by experiments. Two experimental tasks were designed as follows: positioning of mobile robot and navigation in complex environment. Time for task completion and motion accuracy were measured and compared for different command strategies and types of feedback. Role of haptic, text and visual feedback information in combination with described command strategies is outlined.     

Improving virtual reality navigation tasks using a haptic vest and upper body tracking

Haptic feedback is an important component of immersive virtual reality (VR) applications that is often suggested to complement visual information through the sense of touch. This paper investigates the use of a haptic vest in navigation tasks. The haptic vest produces a repulsive vibrotactile feedback from nearby static virtual obstacles that augments the user spatial awareness. The tasks require the user to perform complex movements in a 3D cluttered virtual environment, like avoiding obstacles while walking backwards and pulling a virtual object. The experimental setup consists of a room-scale environment. Our approach is the first study where a haptic vest is tracked in real time using a motion capture device so that proximity-based haptic feedback can be conveyed according to the actual movement of the upper body of the user.User study experiments have been conducted with and without haptic feedback in virtual environments involving both normal and limited visibility conditions. A quantitative evaluation was carried out by measuring task completion time and error (collision) rate. Multiple haptic rendering techniques have also been tested. Results show that under limited visibility conditions proximity-based haptic feedback generated by a wearable haptic vest can significantly reduce the number of collisions with obstacles in the virtual environment.     

Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user’s sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application’s Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.     

Auditory feedback in haptic collaborative interfaces

The combined effect of haptic and auditory feedback in shared interfaces on the cooperation between visually impaired and sighted persons is under-investigated. A central challenge for cooperating group members lies in obtaining a common understanding of the elements of the workspace and maintaining awareness of the other members' actions, as well as one's own, during the group work process. The aim of the experimental study presented here was to investigate if adding audio cues in a haptic and visual interface makes collaboration between a sighted and a blindfolded person more efficient. Results showed that task performance was significantly faster in the audio, haptic and visual feedback condition compared to the haptic and visual feedback condition. One special focus was also to study how participants utilize the auditory and haptic force feedback in order to obtain a common understanding of the workspace and to maintain an awareness of the group members' actions. Results from a qualitative analysis showed that the auditory and haptic feedback was used in a number of important ways to support the group members' action awareness and in the participants' grounding process.     

Effects of auditory,haptic and visual feedback on performing gestures by gaze or by hand

Modern interaction techniques like non-intrusive gestures provide means for interacting with distant displays and smart objects without touching them. We were interested in the effects of feedback modality (auditory, haptic or visual) and its combined effect with input modality on user performance and experience in such interactions. Therefore, we conducted two exploratory experiments where numbers were entered, either by gaze or hand, using gestures composed of four stroke elements (up, down, left and right). In Experiment 1, a simple feedback was given on each stroke during the motor action of gesturing: an audible click, a haptic tap or a visual flash. In Experiment 2, a semantic feedback was given on the final gesture: the executed number was spoken, coded by haptic taps or shown as text. With simultaneous simple feedback in Experiment 1, performance with hand input was slower but more accurate than with gaze input. With semantic feedback in Experiment 2, however, hand input was only slower. Effects of feedback modality were of minor importance; nevertheless, semantic haptic feedback in Experiment 2 showed to be useless at least without extensive training. Error patterns differed between both input modes, but again not dependent on feedback modality. Taken together, the results show that in designing gestural systems, choosing a feedback modality can be given a low priority; it can be chosen according to the task, context and user preferences.     

Visual-haptic feedback interaction in automotive touchscreens

Touchscreen interfaces offer benefits in terms of flexibility and ease of interaction and as such their use has increased rapidly in a range of devices, from mobile phones to in-car technology. However, traditional touchscreens impose an inevitable visual workload demand that has implications for safety, especially in automotive use. Recent developments in touchscreen technology have enabled feedback to be provided via the haptic channel. A study was conducted to investigate the effects of visual and haptic touchscreen feedback on visual workload, task performance and subjective response using a medium-fidelity driving simulator. Thirty-six experienced drivers performed touchscreen ‘search and select’ tasks while engaged in a motorway driving task. The study utilised a 3 × 2 within-subjects design, with three levels of visual feedback: ‘immediate’, ‘delayed’, ‘none’; and two levels of haptic feedback: ‘visual only’, ‘visual + haptic’. Results showed that visual workload was increased when visual feedback was delayed or absent; however, introducing haptic feedback counteracted this effect, with no increases observed in glance time and count. Task completion time was also reduced when haptic feedback was enabled, while driving performance showed no effect due to feedback type. Subjective responses indicated that haptic feedback improved the user experience and reduced perceived task difficulty.     

Haptic User Interfaces for the Visually Impaired: Implications for Haptically Enhanced Science Learning Systems

The overall quality of haptic user interfaces designed to support visually impaired students' science learning through sensorial feedback was systematically studied to investigate task performance and user behavior. Fourteen 6th- to 11th-grade students with visual impairments recruited from a state-funded blind school were asked to perform three main tasks (i.e., menu selection, structure exploration, and force recognition) using haptic user interfaces and a haptic device. This study used several dependent measures that are categorized into three types of variables: (a) task performance including success rate, workload, and task completion time; (b) user behavior defined as cursor movements proportionately represented from the user's cursor positional data; and (c) user preference. Results showed that interface type has significant effects on task performance, user behavior, and user preference, with varying degree of impact to participants with severe visual impairments performing the tasks. The results of this study as well as a set of refined design guidelines and principles should provide insights to the future research of haptic user interfaces that can be used when developing haptically enhanced science learning systems for the visually impaired.     

Age differences in the control of a precision reach to grasp task within a desktop virtual environment

The purpose of this experiment is to investigate the fine motor performance of young and older adults on a reach to grasp task in a desktop virtual environment with increasing precision requirements. Aging brings about potential loss of an individual's function due to disease, injury, or the degenerative nature of aging itself. Three-dimensional virtual environments have been identified as systems with good potential to ameliorate such problems in older individuals, and precise fine motor skills represent an important class of functional skills. Two groups of participants (Young, n=10, mean age 21.3 years, range 20–24, senior, n=10, mean age 70.7 years, range 60–85) performed a reach to grasp in a desktop virtual environment with simple, low contrast graphics. Results indicate that visual feedback of the hand for sensory guidance of movement did not improve motor performance for either group, and that as precision requirements of the task increased, age group differences in movement time and peak grasp aperture also increased. These findings extend the literature on age group differences in human motor control across the lifespan and differ from previous studies which showed presence of visual feedback of the hand improved motor performance in young adults. Differences in luminance contrast levels in past studies and the current one suggest that control over this feature of the visual scene is an important design consideration for all end-users and warrants additional investigation. Additional recommendations for age-specific design of three dimensional user interfaces include usage of tangibles that are sufficient in size to limit detrimental effects for older adults.     

Foundation for improved interaction by individuals with visual impairments through multimodal feedback

Through an investigation of how the performance of people who have normal visual capabilities is affected by unimodal, bimodal, and trimodal feedback, this research establishes a foundation for presenting effective feedback to enhance the performance of individuals who have visual impairments. Interfaces that employ multiple feedback modalities, such as auditory, haptic, and visual, can enhance user performance for individuals with barriers limiting one or more channels of perception, such as a visual impairment. Results obtained demonstrate the effects of different feedback combinations on mental workload, accuracy, and performance time. Future, similar studies focused on participants with visual impairments will be grounded in this work. Published online: 6 November 2002     

Multimodal feedback: an assessment of performance and mental workload

Multimodal interfaces offer great potential to humanize interactions with computers by employing a multitude of perceptual channels. This paper reports on a novel multimodal interface using auditory, haptic and visual feedback in a direct manipulation task to establish new recommendations for multimodal feedback, in particular uni-, bi- and trimodal feedback. A close examination of combinations of uni-, bi- and trimodal feedback is necessary to determine which enhances performance without increasing workload. Thirty-two participants were asked to complete a task consisting of a series of 'drag-and-drops' while the type of feedback was manipulated. Each participant was exposed to three unimodal feedback conditions, three bimodal feedback conditions and one trimodal feedback condition that used auditory, visual and haptic feedback alone, and in combination. Performance under the different conditions was assessed with measures of trial completion time, target highlight time and a self-reported workload assessment captured by the NASA Task Load Index (NASA-TLX). The findings suggest that certain types of bimodal feedback can enhance performance while lowering self-perceived mental demand.     

The Effects of Haptic Feedback and Visual Distraction on Pointing Task Performance

Previous research has not fully examined the effect of additional sensory feedback, particularly delivered through the haptic modality, in pointing task performance with visual distractions. This study examined the effect of haptic feedback and visual distraction on pointing task performance in a 3D virtual environment. Results indicate a strong positive effect of haptic feedback on performance in terms of task time and root mean square error of motion. Level of similarity between distractor objects and the target object significantly reduced performance, and subjective ratings indicated a sense of increased task difficulty as similarity increased. Participants produced the best performance in trials where distractor objects had a different color but the same shape as the target object and constant haptic assistive feedback was provided. Overall, this study provides insight toward the effect of object features and similarity and the effect of haptic feedback on pointing task performance.     

How can we best use landmarks to support older people in navigation?

Although landmarks are an integral part of navigation, they have rarely been used explicitly within electronic pedestrian navigation aids. We describe a two-part study into the use of landmarks in such aids, using a set of field experiments. The first part investigated whether such devices can be effective for older adults (over 60 years old), who might particularly benefit from them due to declines in sensory, cognitive and motor abilities. The second part compared the effectiveness of different methods of presenting landmark information. We show that a pedestrian navigation aid based around landmarks is particularly useful for older people and demonstrate that text, speech and photographs are all effective ways of presenting landmark information, although speech on its own has some drawbacks. We found that different people prefer information to be presented in different modalities, indicating a need for personalisation, although multi-modality may also help to address this issue.     

Haptic Seat Interfaces for Driver Information and Warning Systems

The implementation of haptic interfaces in vehicles has important safety and flexibility implications for lessening visual and auditory overload during driving. The present study aims to design and evaluate haptic interfaces with vehicle seats. Three experiments were conducted by testing a haptic seat in a simulator with a total of 20 participants. The first experiment measured reaction time, subjective satisfaction, and subject workloads of the haptic, visual, and auditory displays for the four signals primarily used by vehicle navigation systems. The second experiment measured reaction time, subjective satisfaction, and subjective workloads of the haptic, auditory, and multimodal (haptic + auditory) displays for the ringing signal used by in-vehicle Bluetooth hands-free systems. The third experiment measured drivers' subjective awareness, urgency, usefulness, and disturbance levels at various vibration intensities and positions for a haptic warning signal used by a driver drowsiness warning system. The results indicated that haptic seat interfaces performed better than visual and auditory interfaces, but the unfamiliarity of the haptic interface caused a lower subjective satisfaction for some criteria. Generally, participants showed high subjective satisfaction levels and low subjective workloads toward haptic seat interfaces. This study provided guidance for implementing haptic seat interfaces and identified the possible benefits of their use. It is expected that haptic seats implemented in vehicles will improve safety and the interaction between driver and vehicle.     

Haptic teleoperation of a multirotor aerial robot using path planning with human intention estimation

Classical haptic teleoperation systems heavily rely on operators’ intelligence and efforts in aerial robot navigation tasks, thereby posing significantly users’ workloads. In this paper, a novel shared control scheme is presented facilitating a multirotor aerial robot haptic teleoperation system that exhibits autonomous navigation capability. A hidden Markov model filter is proposed to identify the intention state of operator based on human inputs from haptic master device, which is subsequently adopted to derive goal position for a heuristic sampling based local path planner. The human inputs are considered as commanded velocity for a trajectory servo controller to drive the robot along the planned path. In addition, vehicle velocity is perceived by the user via haptic feedback on master device to enhance situation awareness and navigation safety of the user. An experimental study was conducted in a simulated and a physical environment, and the results verify the effectiveness of the novel scheme in safe navigation of aerial robots. A user study was carried out between a classical haptic teleoperation system and the proposed approach in the identical simulated complex environment. The flight data and task load index (TLX) are acquired and analyzed. Compared with the conventional haptic teleoperation scheme, the proposed scheme exhibits superior performance in safe and fast navigation of the multirotor vehicle, and is also of low task and cognitive loads.

     

Effects of feedback and dwell time on eye typing speed and accuracy

Eye typing provides a means of communication that is especially useful for people with disabilities. However, most related research addresses technical issues in eye typing systems, and largely ignores design issues. This paper reports experiments studying the impact of auditory and visual feedback on user performance and experience. Results show that feedback impacts typing speed, accuracy, gaze behavior, and subjective experience. Also, the feedback should be matched with the dwell time. Short dwell times require simplified feedback to support the typing rhythm, whereas long dwell times allow extra information on the eye typing process. Both short and long dwell times benefit from combined visual and auditory feedback. Six guidelines for designing feedback for gaze-based text entry are provided.