Tactile interruption management: tactile cues as task-switching reminders

Tactile cuing has been suggested as a method of interruption management for busy visual environments. This study examined the effectiveness of tactile cues as an interruption management strategy in a multi-tasking environment. Sixty-four participants completed a continuous aircraft monitoring task with periodic interruptions of a discrete gauge memory task. Participants were randomly assigned to two groups; one group had to remember to monitor for interruptions while the other group received tactile cues indicating an interruption’s arrival and location. As expected, the cued participants evidenced superior performance on both tasks. The results are consistent with the notion that tactile cues transform the resource-intensive, time-based task of remembering to check for interruptions into a simpler, event-based task, where cues assume a portion of the workload, permitting the application of valuable resources to other task demands. This study is discussed in the context of multiple resource theory and has practical implications for systems design in environments consisting of multiple, visual tasks and time-sensitive information.     

Interruption management: A comparison of auditory and tactile cues for both alerting and orienting

Tactile and auditory cues have been suggested as methods of interruption management for busy visual environments. The current experiment examined attentional mechanisms by which cues might improve performance. The findings indicate that when interruptive tasks are presented in a spatially diverse task environment, the orienting function of tactile cues is a critical component, which directs attention to the location of the interruption, resulting in superior interruptive task performance. Non-directional tactile cues did not degrade primary task performance, but also did not improve performance on the secondary task. Similar results were found for auditory cues. The results support Posner and Peterson's [1990. The attention system of the human brain. Annual Review of Neuroscience 13, 25–42] theory of independent functional networks of attention, and have practical applications for systems design in work environments that consist of multiple, visual tasks and time-sensitive information.     

Effectiveness of auditory and tactile crossmodal cues in a dual-task visual and auditory scenario

Abstract

In this study, we examined how spatially informative auditory and tactile cues affected participants’ performance on a visual search task while they simultaneously performed a secondary auditory task. Visual search task performance was assessed via reaction time and accuracy. Tactile and auditory cues provided the approximate location of the visual target within the search display. The inclusion of tactile and auditory cues improved performance in comparison to the no-cue baseline conditions. In comparison to the no-cue conditions, both tactile and auditory cues resulted in faster response times in the visual search only (single task) and visual–auditory (dual-task) conditions. However, the effectiveness of auditory and tactile cueing for visual task accuracy was shown to be dependent on task-type condition. Crossmodal cueing remains a viable strategy for improving task performance without increasing attentional load within a singular sensory modality.

Practitioner Summary: Crossmodal cueing with dual-task performance has not been widely explored, yet has practical applications. We examined the effects of auditory and tactile crossmodal cues on visual search performance, with and without a secondary auditory task. Tactile cues aided visual search accuracy when also engaged in a secondary auditory task, whereas auditory cues did not.     

Good vibrations: tactile feedback in support of attention allocation and human-automation coordination in event-driven domains

Observed breakdowns in human-machine communication can be explained, in part, by the nature of current automation feedback, which relies heavily on focal visual attention. Such feedback is not well suited for capturing attention in case of unexpected changes and events or for supporting the parallel processing of large amounts of data in complex domains. As suggested by multiple-resource theory, one possible solution to this problem is to distribute information across various sensory modalities. A simulator study was conducted to compare the effectiveness of visual, tactile, and redundant visual and tactile cues for indicating unexpected changes in the status of an automated cockpit system. Both tactile conditions resulted in higher detection rates for, and faster response times to, uncommanded mode transitions. Tactile feedback did not interfere with, nor was its effectiveness affected by, the performance of concurrent visual tasks. The observed improvement in task-sharing performance indicates that the introduction of tactile feedback is a promising avenue toward better supporting human-machine communication in event-driven, information-rich domains.     

Cross-modal links among vision, audition, and touch in complex environments

OBJECTIVES: This study sought to determine whether performance effects of cross-modal spatial links that were observed in earlier laboratory studies scale to more complex environments and need to be considered in multimodal interface design. It also revisits the unresolved issue of cross-modal cuing asymmetries. BACKGROUND: Previous laboratory studies employing simple cues, tasks, and/or targets have demonstrated that the efficiency of processing visual, auditory, and tactile stimuli is affected by the modality, lateralization, and timing of surrounding cues. Very few studies have investigated these cross-modal constraints in the context of more complex environments to determine whether they scale and how complexity affects the nature of cross-modal cuing asymmetries. METHOD: Amicroworld simulation of battlefield operations with a complex task set and meaningful visual, auditory, and tactile stimuli was used to investigate cuing effects for all cross-modal pairings. RESULTS: Significant asymmetric performance effects of cross-modal spatial links were observed. Auditory cues shortened response latencies for collocated visual targets but visual cues did not do the same for collocated auditory targets. Responses to contralateral (rather than ipsilateral) targets were faster for tactually cued auditory targets and each visual-tactile cue-target combination, suggesting an inhibition-of-return effect. CONCLUSIONS: The spatial relationships between multimodal cues and targets significantly affect target response times in complex environments. The performance effects of cross-modal links and the observed cross-modal cuing asymmetries need to be examined in more detail and considered in future interface design. APPLICATION: The findings from this study have implications for the design of multimodal and adaptive interfaces and for supporting attention management in complex, data-rich domains.     

Interfaces That Heal: Coupling Real and Virtual Objects to Treat Spider Phobia

Tactile augmentation is a simple, safe, inexpensive interaction technique for adding physical texture and force feedback cues to virtual objects. This study explored whether virtual reality (VR) exposure therapy reduces fear of spiders and whether giving patients the illusion of physically touching the virtual spider increases treatment effectiveness. Eight clinically phobic students were randomly assigned to one of 3 groups-(a) no treatment, (b) VR with no tactile cues, or (c) VR with a physically "touchable" virtual spider-as were 28 nonclinically phobic students. Participants in the 2 VR treatment groups received three 1-hr exposure therapy sessions resulting in clinically significant drops in behavioral avoidance and subjective fear ratings. The tactile augmentation group showed the greatest progress on behavioral measures. On average, participants in this group, who only approached to 5.5 ft of a live spider on the pretreatment Behavioral Avoidance Test (Garcia-Palacios, 2002), were able to approach to 6 in. of the spider after VR exposure treatment and did so with much less anxiety (see www.vrpain.com for details). Practical implications are discussed.     

Effect of tactile feedback on movement speed and precision during work-related tasks using a computer mouse

Effects of tactile feedback on movement accuracy and speed were studied. Younger and older participants performed three tasks (1, select and drag word; 2, menu navigation; 3, select and drag cell) using commercial software and a mouse with or without tactile feedback. Task time and error number were recorded. Tasks were divided according to presence or absence of tactile feedback, and participants were divided into subgroups (high, average, low) based on Task 1 performance. Overall, older participants took longer (p < .0001) and made more errors (p < .001) than younger participants. There was an effect of feedback by task in younger participants for all six outcomes (p < 0.02). At the task level, with feedback, younger participants reduced performance time (13%) and errors (24%) on Task 1. Low- and average-performance younger participants benefited most from feedback for Task 1. Older low-performance participants also benefited from feedback for Task 1. For Task 3, older participants tended to take more time and make more errors with feedback. Tactile feedback may enhance performance when feedback is event related. Older people may not integrate sensation as well as younger individuals to enhance performance. Potential applications of this research include the development of tactile feedback interfaces to facilitate computer use.     

Effects of tactile cueing on concurrent performance of military and robotics tasks in a simulated multitasking environment

This study examined the concurrent performance of military gunnery, robotics control and communication tasks in a simulated environment. More specifically, the study investigated how aided target recognition (AiTR) capabilities (delivered either through tactile or tactile + visual cueing) for the gunnery task might benefit overall performance. Results showed that AiTR benefited not only the gunnery task, but also the concurrent robotics and communication tasks. The participants' spatial ability was found to be a good indicator of their gunnery and robotics task performance. However, when AiTR was available to assist their gunnery task, those participants of lower spatial ability were able to perform their robotics tasks as well as those of higher spatial ability. Finally, participants' workload assessment was significantly higher when they teleoperated (i.e. remotely operated) a robot and when their gunnery task was unassisted. These results will further understanding of multitasking performance in military tasking environments. These results will also facilitate the implementation of robots in military settings and will provide useful data to military system designs.     

Design and evaluation of virtual environments mechanisms to support remote collaboration on complex process diagrams

Many organizational analysis tasks are solved by collaborating teams. In technology-mediated collaborations, enabling relevant visual cues is a core issue with existing technology. We explore whether avatars can provide relevant cues in collaborative virtual environments. To do so, we develop a research prototype for a collaborative virtual environment that utilises avatars to improve workspace awareness in collaborative tasks. We test this solution through two studies, qualitative and quantitative, in which participants have to collaborate to jointly validate and correct a diagrammatic model of operational procedures.Our evaluations provide both positive and negative results about the proposed prototype. Collaboration patterns changed and became significantly easier, but, task performance was only slightly improved. Together, these findings inform both the ongoing development of collaborative 3D virtual environments and the role of technology-mediated collaboration for validating and fixing models of processes.     

A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving

OBJECTIVE: This study examined the effectiveness of rear-end collision warnings presented in different sensory modalities as a function of warning timing in a driving simulator. BACKGROUND: The proliferation of in-vehicle information and entertainment systems threatens driver attention and may increase the risk of rear-end collisions. Collision warning systems have been shown to improve inattentive and/or distracted driver response time (RT) in rear-end collision situations. However, most previous rear-end collision warning research has not directly compared auditory, visual, and tactile warnings. METHOD: Sixteen participants in a fixed-base driving simulator experienced four warning conditions: no warning, visual, auditory, and tactile. The warnings activated when the time-to-collision (TTC) reached a critical threshold of 3.0 or 5.0 s. Driver RT was captured from a warning below critical threshold to brake initiation. RESULTS: Drivers with a tactile warning had the shortest mean RT. Drivers with a tactile warning had significantly shorter RT than drivers without a warning and had a significant advantage over drivers with visual warnings. CONCLUSION: Tactile warnings show promise as effective rear-end collision warnings. APPLICATION: The results of this study can be applied to the future design and evaluation of automotive warnings designed to reduce rear-end collisions.     

Not Now! Supporting interruption management by indicating the modality and urgency of pending tasks

Operators in complex event-driven domains must coordinate competing attentional demands in the form of multiple tasks and interactions. This study examined the extent to which this requirement can be supported more effectively through informative interruption cueing (in this case, partial information about the nature of pending tasks). The 48 participants performed a visually demanding air traffic control (ATC) task. They were randomly assigned to 1 of 3 experimental groups that differed in the availability of information (not available, available upon request, available automatically) about the urgency and modality of pending interruption tasks. Within-subject variables included ATC-related workload and the modality, frequency, and priority of interruption tasks. The results show that advance knowledge about the nature of pending tasks led participants to delay visual interruption tasks the longest, which allowed them to avoid intramodal interference and scanning costs associated with performing these tasks concurrently with ATC tasks. The 3 experimental groups did not differ significantly in terms of their interruption task performance; however, the group that automatically received task-related information showed better ATC performance, thus experiencing a net performance gain. Actual or potential applications of this research include the design of interfaces in support of attention and interruption management in a wide range of event-driven environments.     

Evaluation of automobile navigation systems with multisensory information channels

Although automobile navigation systems have conventionally utilised visual and auditory modalities to deliver information, recent advancement in tactile technology has introduced the possibility of integrating tactile actuators into navigation systems. To empirically test the effect of tactile navigational cues, we conducted simulated driving experiments in which participants (N?=?96) were exposed to four sets of information modalities (visual, visual?+?auditory, visual?+?tactile, and visual?+?auditory?+?tactile). The results indicate that multisensory systems including tactile cues allow participants to respond faster to unexpected road events and impart greater satisfaction with the overall driving experience. The implications and limitations of these findings are discussed.     

Effects of tactile cueing on concurrent performance of military and robotics tasks in a simulated multitasking environment

This study examined the concurrent performance of military gunnery, robotics control and communication tasks in a simulated environment. More specifically, the study investigated how aided target recognition (AiTR) capabilities (delivered either through tactile or tactile + visual cueing) for the gunnery task might benefit overall performance. Results showed that AiTR benefited not only the gunnery task, but also the concurrent robotics and communication tasks. The participants' spatial ability was found to be a good indicator of their gunnery and robotics task performance. However, when AiTR was available to assist their gunnery task, those participants of lower spatial ability were able to perform their robotics tasks as well as those of higher spatial ability. Finally, participants' workload assessment was significantly higher when they teleoperated (i.e. remotely operated) a robot and when their gunnery task was unassisted. These results will further understanding of multitasking performance in military tasking environments. These results will also facilitate the implementation of robots in military settings and will provide useful data to military system designs.     

Maintaining and modifying pace through tactile and multimodal feedback

Older adults are recommended to remain physically active to reduce the risk of chronic diseases and to maintain psychological well-being. At the same time, research also suggests that levels of fitness can be raised among this group. This paper describes the development and evaluation of a mobile technology, which enables older adults to monitor and modify their walking habits, with the long-term aim of sustaining appropriate levels of physical activity. An empirical study was conducted with twenty older adults to determine the feasibility of the proposed solution, with results indicating that tactile signals could be perceived while in motion and could support participants in walking at a range of paces. However, the effects were difficult to discern due to limitations of the hardware. In response, a novel low-cost prototype was developed to amplify vibrations, and effectiveness of redundant auditory information was investigated with the goal of enhancing the perception of the cues. A second study was conducted to determine the impact of multimodal feedback on walking behavior. Findings revealed that participants were able to maintain a desired level of pace more consistently when redundant auditory information was presented alongside the tactile feedback. When the visual channel is not available, these results suggest that tactile cues presented via a mobile device should be augmented with auditory feedback. Our research also suggests that mobile devices could be made more effective for alternative applications if they are designed to allow for stronger tactile feedback.     

Multisensory teamwork: using a tactile or an auditory display to exchange gaze information improves performance in joint visual search

In joint tasks, adjusting to the actions of others is critical for success. For joint visual search tasks, research has shown that when search partners visually receive information about each other’s gaze, they use this information to adjust to each other’s actions, resulting in faster search performance. The present study used a visual, a tactile and an auditory display, respectively, to provide search partners with information about each other’s gaze. Results showed that search partners performed faster when the gaze information was received via a tactile or auditory display in comparison to receiving it via a visual display or receiving no gaze information. Findings demonstrate the effectiveness of tactile and auditory displays for receiving task-relevant information in joint tasks and are applicable to circumstances in which little or no visual information is available or the visual modality is already taxed with a demanding task such as air-traffic control. Practitioner Summary: The present study demonstrates that tactile and auditory displays are effective for receiving information about actions of others in joint tasks. Findings are either applicable to circumstances in which little or no visual information is available or when the visual modality is already taxed with a demanding task.     

Towards developing perceivable tactile feedback for mobile devices

As mobile technologies such as cellular telephones reduce in both size and cost, and improve in fidelity, they become a more attractive option for performing tasks such as surfing the Web and accessing applications while on-the-go. The small size of the visual display limits the amount of information that can be presented, which may lead to cluttered interfaces. Tactile feedback (e.g. vibrations) provides one solution to reducing the burden on the visual channel. This paper describes a series of studies conducted with the goal of developing perceivable tactile icons (tactons) to aid in non-visual interactions with mobile applications. In contrast to previous work, our research addresses the development of pairs of tactons, rather than individual tactons, with the goal of conveying two-state signals such as ‘on/off’, ‘slower/faster’, or ‘left/right’. Such communication can help reduce visual demands associated with using mobile applications, allowing the device to convey important information while the users’ hands and eyes are otherwise occupied. Realistic conditions were simulated in a laboratory-based environment to determine how auditory distracters could affect the perception of tactons. Findings show that recognition rates differed depending on the design of the vibration pair parameters, and type of auditory distracter. This research culminated in a set of guidelines, which tactile interface designers can integrate as they design mobile applications to improve access, as well as insights which can guide future research on tactile feedback for mobile devices.     

Enhancing virtual environment spatial awareness training and transfer through tactile and vestibular cues

Haptic interaction has been successfully incorporated into a variety of virtual environment (VE) systems, yet designing multimodal VE training systems remains challenging as each cue incorporated during training should maximise learning and training transfer. This study examined the impact of incorporating two independent, spatialised tactile cues and vestibular cues into a military VE training environment with the goal of empirically examining whether such cues could enhance performance within the training environment and also that knowledge and skills gained during training could transfer to another environment. The results showed that tactile cues enhanced spatial awareness and performance during both repeated training and within a transfer environment, yet there were costs associated when two independent tactile cues were presented during training. In addition, results suggest that spatial awareness benefits from a tactile point indicator may be impacted by vestibular cues, as performance benefits were seen when tactile cues were paired with head tracking. To fully realise training potential, it is essential to determine how best to leverage multimodal capacity of VE training systems by identifying how multimodal training cues may advance knowledge, skills and attitudes of trainees. Results from this study provide design guidelines for incorporating tactile cues in VE training environments to enhance spatial awareness.     

An empirical evaluation of multi-media based learning of a procedural task

The present study investigated the effects of multi-media modules and their combinations on the learning of procedural tasks. In the experiment, 72 participants were classified as having either low- or high spatial ability based on their spatial ability test. They were randomly assigned to one of the six experimental conditions in a 2 × 3 factorial design with verbal modality (on-screen text procedure vs. auditory procedure) and the format of visual representation (static visual representation vs. static visual representation with motion cues vs. animated visual representation). After they completed their learning session, the ability to perform the procedural task was directly measured in a realistic setting. The results revealed that: (1) in the condition of static visual representation, the high spatial ability group outperformed the low spatial ability group, (2) for the low spatial ability participants, the animated visual representation group outperformed the static visual representation group, however, the static visual representation with motion cues group did not outperform the static visual representation group, (3) the use of animated visual representation helped participants with low spatial ability more than those with high spatial ability, and (4) a modality effect was found for the measure of satisfaction when viewing the animated visual representation. Since the participants with low spatial ability benefited from the use of animation, the results might support an idea that people are better able to retrieve the procedural information by viewing animated representation. The findings also might reflect a preference for the auditory mode of presentation with greater familiarity with the type of visual representation.     

A tactile cockpit instrument supports the control of self-motion during spatial disorientation

OBJECTIVE: We investigated the effectiveness of a tactile torso display as a countermeasure to spatial disorientation (SD) and compared inside-out and outside-in codings. BACKGROUND: SD is a serious threat to military as well as civilian pilots and aircraft. Considerable effort has been put into SD countermeasures such as training programs and advanced cockpit displays. Tactile displays have been considered a promising technology. METHOD: Twenty-four participants were assigned to the two coding groups (12 per group and matched for age and gender). We used a rotating chair to build up a state of SD by rotating participants around their yaw axis followed by a sudden stop. During the following recovery phase a random disturbance signal was added to the chair's orientation. Participants actively controlled their orientation and were instructed to maintain a stable orientation. RESULTS: Statistical analyses revealed that recovery from SD was improved with support of the tactile instrument, but tracking performance was reduced. The effects were the same whether the instrument was available full time or during the recovery phase only. There were no differences between outside-in and inside-out coding. CONCLUSION: The present study demonstrates the potential of tactile cockpit instruments in controlling SD, even in the presence of strong but misleading self-motion information from the vestibular sense. APPLICATION: Actual or potential applications of this research include spatial disorientation countermeasures for pilots, divers, and astronauts.     

Tactile feedback enhanced hand gesture interaction at large, high-resolution displays

Human beings perceive their surroundings based on sensory information from diverse channels. However, for human–computer interaction we mostly restrict the user on visual perception. In this paper, we contribute to the investigation of tactile feedback as an additional perception modality. Therefore, we will first discuss existing user studies and provide a classification scheme for tactile feedback techniques. We will then present and discuss a comparative evaluation study based on the ISO 9241-9 [Ergonomic requirements for office work with visual display terminals (VDTs) – Part 9: requirements for non-keyboard input devices, 2000]. The 20 participants performed horizontal and vertical one-directional tapping tasks with hand gesture input with and without tactile feedback in front of a large, high-resolution display. In contrast to previous research, we cannot confirm a benefit of tactile feedback on user performance. Our results show no significant effect in terms of throughput (effective index of performance (IPe)) and even a significant higher error rate for horizontal target alignment when using tactile feedback. Based on these results, we suggest that tactile feedback can interfere with other senses in a negative way, resulting in the observed higher error rate for horizontal targets. Therefore, more systematic research is needed to clarify the influencing factors on the usefulness of tactile feedback. Besides these results, we found a significant difference in favor of the horizontal target alignment compared with the vertical one in terms of the effective index of performance (IPe), confirming the work by Dennerlein et al. [Force feedback improves performance for steering and combined steering–targeting tasks, in: CHI ’00: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 2000, pp. 423–429].