Head up versus head down: the costs of imprecision, unreliability, and visual clutter on cue effectiveness for display signaling

We conducted 2 experiments to investigate the clutter-scan trade-off between the cost of increasing clutter by overlaying complex information onto the forward field of view using a helmet-mounted display (HMD) and the cost of scanning when presenting this information on a handheld display. In the first experiment, this trade-off was examined in terms of the spatial accuracy of target cuing data in a relatively sparse display; in the second, the spatial accuracy of the cue was varied more radically in an information-rich display. Participants were asked to detect and identify targets hidden in the far domain while performing a monitoring task in the near domain using either an HMD or a handheld display. The results revealed that on a sparse display, the reduced scanning from the HMD presentation of cuing out-weighed the costs of clutter for cued targets, regardless of cue precision, but no benefit was found for uncued targets. When the HMD displayed task-irrelevant information, however, target detection was hindered by the extraneous clutter in the forward field of view relative to the handheld display condition, and this cost of clutter increased as the amount of data that needed to be inspected increased. Potential applications of this research include the development of design considerations for head-up displays for aviation and military applications.     

Verbal interface design: Do verbal directional cues automatically orient visual spatial attention?

The last few years have seen a rapid growth of interest in the use of verbal information displays in many applied interface settings. However, to date, it is unclear what effect the presentation of verbal cues, such as the words ‘left’ or ‘right’, has on the spatial distribution of an interface operator’s attention. In the present study, we addressed this issue by investigating whether centrally-presented spatially-nonpredictive verbal directional cues elicit an automatic shift of visual spatial attention in the direction indicated by the cue. Participants performed a digit discrimination task for targets presented on either the left or right. Prior to target presentation, the directional word cues ‘left’ or ‘right’ were presented auditorily or visually from the centre of the display at cue-target stimulus onset asynchronies (SOAs) of 200, 400, or 600 ms. Visual discrimination performance was assessed both under conditions where the target digits were unmasked (auditory and visual cuing), and when the targets were masked (auditory cuing only). The results showed that unmasked visual target discrimination performance was facilitated on the cued (relative to the uncued) side at the shortest SOA following visual cuing, but was unaffected by auditorily-presented directional cues. Interestingly, our results also indicated improved visual sensitivity on the auditorily-cued side in the masked target condition. These findings are discussed in relation to previous laboratory-based and applied symbolic cuing studies that have investigated the consequences of the presentation of arrow, gaze direction, and/or head orientation directional cues on the spatial distribution of attention.     

Target acquisition with UAVs: vigilance displays and advanced cuing interfaces

Vigilance and threat detection are critical human factors considerations in the control of unmanned aerial vehicles (UAVs). Utilizing a vigilance task in which threat detections (critical signals) led observers to perform a subsequent manual target acquisition task, this study provides information that might have important implications for both of these considerations in the design of future UAV systems. A sensory display format resulted in more threat detections, fewer false alarms, and faster target acquisition times and imposed a lighter workload than did a cognitive display format. Additionally, advanced visual, spatial-audio, and haptic cuing interfaces enhanced acquisition performance over no cuing in the target acquisition phase of the task, and they did so to a similar degree. Thus, in terms of potential applications, this research suggests that a sensory format may be the best display format for threat detection by future UAV operators, that advanced cuing interfaces may prove useful in future UAV systems, and that these interfaces are functionally interchangeable.     

Superimposition, symbology, visual attention, and the head-up display

In two experiments we examined a number of related factors postulated to influence head-up display (HUD) performance. We addressed the benefit of reduced scanning and the cost of increasing the number of elements in the visual field by comparing a superimposed HUD with an identical display in a head-down position in varying visibility conditions. We explored the extent to which the characteristics of HUD symbology support a division of attention by contrasting conformal symbology (which links elements of the display image to elements of the far domain) with traditional instrument landing system (ILS) symbology. Together the two experiments provide strong evidence that minimizing scanning between flight instruments and the far domain contributes substantially to the observed HUD performance advantage. Experiment 1 provides little evidence for a performance cost attributable to visual clutter. In Experiment 2 the pattern of differences in lateral tracking error between conformal and traditional ILS symbology supports the hypothesis that, to the extent that the symbology forms an object with the far domain, attention may be divided between the superimposed image and its counterpart in the far domain.     

Display signaling in augmented reality: effects of cue reliability and image realism on attention allocation and trust calibration

This experiment seeks to examine the relationships among three advanced technology features (presentation of target cuing, reliability of target cuing, and level of image reality and the attention and trust given to that information). The participants were 16 military personnel who piloted an unmanned air vehicle and searched for targets camouflaged in terrain, which was presented at two levels of image realism. Cuing was available for some targets, and the reliability of this information was manipulated at two levels (100% and 75%). The results showed that the presence of cuing aided target detection for expected targets but drew attention away from the presence of unexpected targets. Cuing benefits and attentional tunneling were both reduced when cuing became less reliable. Increasing image realism was compelling but increased reliance on the cuing information when those data were reliable. Potential applications include a cost-benefit analysis of how trust modulates attention in the use of automated target recognition systems and the extent to which increased realism may influence this trust.     

Unpacking the temporal advantage of distributing complex visual displays

Spatial arrangement of information can have large effects on problem solving. Although such effects have been observed in various domains (e.g., instruction and interface designs), little is known about the cognitive processing mechanisms underlying these effects, nor its applicability to complex visual problem solving. In three experiments, we showed that the impact of spatial arrangement of information on problem solving time can be surprisingly large for complex real world tasks. It was also found that the effect can be caused by large increases in slow, external information searches (Experiment 1), that the spatial arrangement itself is the critical factor and the effect is domain-general (Experiment 2a), and that the underlying mechanism can involve micro-strategy selection for information encoding in a response to differing information access cost (Experiment 2b). Overall, these studies show a large slowdown effect (i.e., approximately 30%) that stacking information produces over spatially distributed information, and multiple paths by which this effect can be produced.     

Cross-modal links among vision, audition, and touch in complex environments

OBJECTIVES: This study sought to determine whether performance effects of cross-modal spatial links that were observed in earlier laboratory studies scale to more complex environments and need to be considered in multimodal interface design. It also revisits the unresolved issue of cross-modal cuing asymmetries. BACKGROUND: Previous laboratory studies employing simple cues, tasks, and/or targets have demonstrated that the efficiency of processing visual, auditory, and tactile stimuli is affected by the modality, lateralization, and timing of surrounding cues. Very few studies have investigated these cross-modal constraints in the context of more complex environments to determine whether they scale and how complexity affects the nature of cross-modal cuing asymmetries. METHOD: Amicroworld simulation of battlefield operations with a complex task set and meaningful visual, auditory, and tactile stimuli was used to investigate cuing effects for all cross-modal pairings. RESULTS: Significant asymmetric performance effects of cross-modal spatial links were observed. Auditory cues shortened response latencies for collocated visual targets but visual cues did not do the same for collocated auditory targets. Responses to contralateral (rather than ipsilateral) targets were faster for tactually cued auditory targets and each visual-tactile cue-target combination, suggesting an inhibition-of-return effect. CONCLUSIONS: The spatial relationships between multimodal cues and targets significantly affect target response times in complex environments. The performance effects of cross-modal links and the observed cross-modal cuing asymmetries need to be examined in more detail and considered in future interface design. APPLICATION: The findings from this study have implications for the design of multimodal and adaptive interfaces and for supporting attention management in complex, data-rich domains.     

The complex role of perceptual organization in visual display design theory

Two experiments were performed to test and extend the current ‘emergent features’ approach to display design for dynamic failure detection tasks. It was predicted that a display whose emergent features were well-mapped to goal-relevant task invariants would lead to better failure detection performance than either of two alternative displays. Contrary to prediction, Experiment 1 showed no differences in failure detection speed or accuracy across displays. The reason for this may have been that subjects did not discover the mapping between the mathematical properties of the task and the emergent feature, so in Experiment 2 subjects were explicitly instructed about the mapping and were advised on how to use the display geometry to help their performance. A significant difference in failure detection speed emerged, but the display supporting fastest performance was not the one with the well-mapped emergent feature. These results suggest that alternative perceptual organizational factors were at work which overpowered the intended effect. The results also underscore the difficulty of developing a theory of display design, and their impact on current theories is outlined.     

The complex role of perceptual organization in visual display design theory.

Two experiments were performed to test and extend the current 'emergent features' approach to display design for dynamic failure detection tasks. It was predicted that a display whose emergent features were well-mapped to goal-relevant task invariants would lead to better failure detection performance than either of two alternative displays. Contrary to prediction. Experiment 1 showed no differences in failure detection speed or accuracy across displays. The reason for this may have been that subjects did not discover the mapping between the mathematical properties of the task and the emergent feature, so in Experiment 2 subjects were explicitly instructed about the mapping and were advised on how to use the display geometry to help their performance. A significant difference in failure detection speed emerged, but the display supporting fastest performance was not the one with the well-mapped emergent feature. These results suggest that alternative perceptual organizational factors were at work which overpowered the intended effect. The results also underscore the difficulty of developing a theory of display design, and their impact on current theories is outlined.     

Eye movement and pupillary response indices of mental workload during visual search of symbolic displays

Eye movement and pupillary response measures (in addition to search time and accuracy) were collected as indices of visual workload during two experiments designed to evaluate the addition of colour coding to a symbolic tactical display. Displays also varied with regard to symbol density and the type of information participants were required to abstract from the display. These variables were factorially manipulated to examine the effects of colour coding in conditions of varying difficulty. In Experiment 1 (n = 8), search time and the number of eye fixations were affected by all variables and in a similar manner; fixation dwell time and the pupillary response dissociated from the other measures. Compared to monochrome displays, colour coding facilitated search (reduced search time, but not accuracy) during exhaustive search, but had no effect during self-terminating search. Experiment 2 (n = 8) was a replication of Experiment 1 with a pseudo-search control condition added to examine further the pupillary response measures: in particular, to assess the effects of the physical parameters of the displays, and to verify the findings of Experiment 1. Pupillary response measures were sensitive to the information processing demands of the search task, not merely to the physical parameters of the display. Further, the search time, accuracy, and eye movement results from the active search condition generally replicated Experiment 1, but the fixation dwell time data did not. These between-study differences were interpreted as indicating the importance of participant search strategy.     

Improving Window Manipulation and Content Interaction on High-Resolution,Wall-Sized Displays

Interaction with high-resolution wall-sized (Powerwall) displays can be a tedious and difficult task due to large display areas and small target sizes. To overcome this, we developed techniques that reduce the precision required to manipulate windows and select data. The manipulation layer speeds up the common tasks of moving and resizing application windows by overlaying them with large, transparent target areas. The Power-Lens magnifies target sizes by automatically appearing once the cursor reaches the region of interest. Two experiments evaluated these techniques against conventional desktop-style interfaces. Experiment 1 showed the window manipulation layer to speed up the tasks of moving and resizing a window by 24% and 27%, respectively. Experiment 2 showed the Power-Lens to speed up the selection of 5?×?5 pixel targets by 18%. Together, our new techniques help to make interaction more fluid on Powerwall displays.     

The differential effect of vibrotactile and auditory cues on visual spatial attention

Previous research has shown that the presentation of spatially predictive auditory and vibrotactile warning signals can facilitate driver responses to driving events seen through the windscreen or rearview mirror. The present study investigated whether this facilitation reflects the priming of the appropriate response (i.e. braking vs. accelerating) or an attentional cuing effect (i.e. a perceptual benefit that facilitates subsequent behavioural responding). In the experiments reported here, participants had to discriminate the colour of a number plate (red vs. blue) following the presentation of either spatially predictive vibrotactile (experiment 1) or auditory (experiment 2) warning signals that indicated the likely location (front or back) of the visual target, while simultaneously performing a highly attention-demanding rapid serial visual presentation task. Numberplate discrimination performance was facilitated following the presentation of valid auditory cues, but not following the presentation of equally informative vibrotactile cues. The use of an orthogonal spatial cuing design enabled with us to rule out of a potential response priming account of these data. The results suggest that whilst directional congruency between a warning signal and a target event may be sufficient to facilitate performance due to the priming of the appropriate response, attentional facilitation effects may also require the co-location of the cue and target within the same functional region of space.     

Effect of display resolution and antialiasing on the discrimination of simulated-aircraft orientation

In Experiment 1, antialiasing was found to improve performance on an orientation-discrimination task, whereas increasing display pixel-count did not. The latter finding was attributed to a decrease in image contrast associated with driving the CRT beyond its effective bandwidth. In Experiment 2, it was found that display resolution is the primary determinant of orientation-discrimination performance. This performance was not significantly improved by increasing antialiasing beyond a minimal level, suggesting that greater image detail can be substituted for antialias filtering. Finally, data obtained from an objective target-size calibration showed that nominal target size often does not accurately reflect the size (and hence distance) of simulated targets.     

Integrated displays and the perception of graphical data

Three experiments are presented that apply a principle of compatibility of proximity to the perception of graphical data. These experiments demonstrate that the merits and costs of integral displays relative to separable displays depend in part on the task being performed. Experiment 1 employs a data extrapolation task in which information must be integrated. Here an integral display is found to be superior to a separable display. Experiment 2 employed a data perception task that requires focused attention. This reveals superior performance with separable displays. Experiment 3 employed a multiple cue judgement task in which information integration could be manipulated. Data from Experiment 3 were analysed using the Brunswick lens procedures, and indicated that knowledge of the task structure was influenced by the display type. When selective attention was required, knowledge of the task was superior with separable displays. The results are discussed with reference to the principle of compatibility of proximity, and the implications of the results for the perception of scientific data are noted.     

Smooth rotation of 2-D and 3-D representations of terrain: an investigation into the utility of visual momentum

OBJECTIVE: The potential advantage of visual momentum in the form of smooth rotation between two-dimensional (2-D) and three-dimensional (3-D) displays of geographic terrain was examined. BACKGROUND: The relative effectiveness of 2-D and 3-D displays is task dependent, leading to the need for multiple frames of reference as users switch tasks. The use of smooth rotation to provide visual momentum has received little scrutiny in the task-switching context. A cognitive model of the processes involved in switching viewpoints on a set of spatial elements is proposed. METHODS: In three experiments, participants judged the properties of two points placed on terrain depicted as 2-D or 3-D displays. Participants indicated whether Point A was higher than Point B, or whether Point B could be seen from Point A. Participants performed the two tasks in pairs of trials, switching tasks and displays within the pair. In the continuous transition condition the display dynamically rotated in depth from one display format to the other. In the discrete condition there was an instantaneous viewpoint shift that varied across experiments (Experiment 1: immediate; Experiment 2: delay; Experiment 3: preview). RESULTS: Performance after continuous transition was superior to that after discrete transition. CONCLUSION: The visual momentum provided by smooth rotation helped users switch tasks. Application: The use of dynamic transition is recommended when observers examine multiple views of terrain over time. The model may serve as a useful heuristic for designers. The results are pertinent to command and control, geological engineering, urban planning, and imagery analysis domains.     

An evaluation of a "time tunnel" display format for the presentation of temporal information

The time tunnel display design technique combines the benefits of configural displays (salient visual properties corresponding to critical domain semantics) with the benefits of temporal information (i.e., the value of variables and properties over time). In Experiment 1 a baseline configural display and a time tunnel display were evaluated using real-time measures of system control, fault detection, and state estimation in a simulated process control task. The results provided little evidence in support of the time tunnel format. In Experiment 2 access to the temporal context was limited: Participants performed the detection and estimation tasks with static "snapshots" of system states that had been generated in Experiment 1. The overall pattern of results indicates that the time tunnel display was more effective for state estimation tasks than was the baseline configural display and or a trend display. Issues in the design of temporal displays are discussed, including representational formats and the choice of temporal time frames. Issues in the evaluation of temporal displays are also discussed, including the role of temporal information and the critical nature of participants' access to this information. Actual or potential applications of this research include design techniques for improving graphical displays and methodological insights to guide future evaluations.     

Pathway HUDs: are they viable?

We describe two experiments that examine 3D pathway displays in a head-up location for aircraft landing and taxi. We address both guidance performance and pilot strategies in dividing, focusing, and allocating attention between flight path information and event monitoring. In Experiment 1 the 3D pathway head-up display (HUD) was compared with a conventional 2D HUD. The former was found to produce better guidance, with few costs to event detection. Some evidence was provided that attentional tunneling of the pathway HUD inhibits the detection of unexpected traffic events. In Experiment 2, the pathway display was compared in a head-up versus a head-down location. Excellent guidance was achieved in both locations. A slight HUD cost for vertical tracking in the air was offset by a HUD benefit for event detection and for lateral tracking during taxi (i.e., on the ground). The results of both experiments are interpreted within the framework of object- and space-based theories of visual attention and point to the conclusion that pathway HUDs combine the independent advantages of pathways and HUDs, particularly during ground operations. Actual or potential applications include understanding the costs and benefits of positioning a 3D pathway display in a head-up location.     

Effects of interface and spatial ability on manipulation of virtual models in a STEM domain

Virtual models are increasingly employed in STEM education to foster learning about spatial phenomena. However, the roles of the computer interface and students’ cognitive abilities in moderating learning and performance with virtual models are not yet well understood. In two experiments students solved spatial organic chemistry problems using a virtual model system. Two aspects of the virtual model interface were manipulated: display dimensionality (stereoscopic vs. monoscopic displays) and the location of the hand-held device used to manipulate the virtual molecules (co-located with the visual display vs. displaced). The experimental task required participants to interpret the spatial structure of organic molecules and to manipulate the models to align them with orientations and configurations depicted by diagrams in Experiment 1 and three-dimensional models in Experiment 2. Co-locating the interaction device with the virtual image led to better performance in both experiments and stereoscopic viewing led to better performance in Experiment 2. The effect of co-location on performance was moderated by spatial ability in Experiment 1, and the effect of providing stereo viewing was moderated by spatial ability in Experiment 2. The results are in line with the ability-as-compensator hypothesis: participants with lower ability uniquely benefited from the treatment, while those with higher ability were not affected by stereo or co-location. The findings suggest that increased fidelity in a virtual model system may be one way of alleviating difficulties of low-spatial participants in learning spatially demanding content in STEM domains.     

Proximity compatibility and information display: effects of color, space, and object display on information integration

We report two experiments in which different features of the display of multiple channels of information are varied in their proximity to one another. The display presents three indicators of aircraft stall danger (airspeed, flaps, and bank). On some trials the stall danger is estimated, requiring information integration. On other trials the value of individual indicators is estimated, requiring focused attention. In Experiment 1 the spatial proximity between indicators and their distinctiveness in color were manipulated. Spatial proximity had little effect on either focused attention or integration, whereas a distinct color code improved focused attention and disrupted integration. In Experiment 2 the three indicators were presented as a bar graph or were combined as features of an object display, either with or without color coding of the separate dimension. Relative to the bar graph display, the object display improved information integration but disrupted focused attention. The presence of color borders restored the focused attention accuracy, with a slight cost to response time.     

Temporal limitations in multiple target detection in a dynamic monitoring task

OBJECTIVE: Two experiments examined the detectability of multiple transient changes within a cluttered and dynamic display (a simulated sonar display). BACKGROUND: Research suggests that there are severe limitations when multiple targets must be detected within close temporal proximity. The present research explored whether these limitations influence performance in a dynamic monitoring task. METHOD: Participants monitored a cluttered and dynamic display and reported the number of new objects that appeared (one to four objects). The time between onset events was varied. A blinking cue sometimes accompanied each new object, giving observers multiple opportunities to detect it. RESULTS: A large decrease in performance was observed when participants were asked to detect multiple targets within a short period of time. Performance was worse than predicted based on the attention literature. Performance suffered when observers were asked to detect more than two or three targets. The blinking cue greatly attenuated this performance deficit, even for short-duration blinking cues (one blink). CONCLUSION: Operators can easily become overwhelmed when asked to respond to even a small number of events when these events occur close in time. Extending transient events in time improves performance, but some attentional limitations may be difficult or impossible to overcome. APPLICATION: These results have important implications for systems in which important events may occur within close temporal proximity (e.g., when a sonar operator is tasked with detecting threats in the battle space). Situations in which these limitations may or may not influence performance are discussed.