Superimposition, symbology, visual attention, and the head-up display

In two experiments we examined a number of related factors postulated to influence head-up display (HUD) performance. We addressed the benefit of reduced scanning and the cost of increasing the number of elements in the visual field by comparing a superimposed HUD with an identical display in a head-down position in varying visibility conditions. We explored the extent to which the characteristics of HUD symbology support a division of attention by contrasting conformal symbology (which links elements of the display image to elements of the far domain) with traditional instrument landing system (ILS) symbology. Together the two experiments provide strong evidence that minimizing scanning between flight instruments and the far domain contributes substantially to the observed HUD performance advantage. Experiment 1 provides little evidence for a performance cost attributable to visual clutter. In Experiment 2 the pattern of differences in lateral tracking error between conformal and traditional ILS symbology supports the hypothesis that, to the extent that the symbology forms an object with the far domain, attention may be divided between the superimposed image and its counterpart in the far domain.     

Early cortical processing of vection-inducing visual stimulation as measured by event-related brain potentials (ERP)

Visual motion stimuli can induce the perception of self-motion in stationary observers (known as vection). In the present study, we investigated the sensory processing underlying vection by measuring the human event-related brain potentials (ERPs) elicited by the movement onset of a visual stimulus. We presented participants a visual stimulus consisting of alternating black-and-white vertical bars that moved in horizontal direction, creating the sensation of vection. The stimulus was presented on a screen that was divided into a central and a surrounding peripheral visual area. Both areas moved independently from each other, resulting in four different movement patterns: the peripheral and the central stimulus moved in the same or opposite direction, or one of the two visual areas of the stimulus moved while the other remained stable. In addition, two different stimulus types varying with respect to the bars’ width (narrow vs. wide) were used. Participants were presented with these stimuli in two phases of the experiment: During EEG-recording, only short phases (2.5–3.5 ms) of visual motion were applied. In a separate rating phase, visual motion was presented for 45 s. In this phase, vection onset, intensity, and duration were verbally recorded. Overall, the visual stimulation generated vection with different intensities (i.e., weakest vection with sole central stimulus movement). Stimulus type did not affect vection. In the ERPs, stimulus onset elicited parieto-occipital P2 and N2 components. The amplitudes of the ERP components differed significantly between the four movement patterns (irrespective of stimulus type), however, they did not fully align with the subjective vection ratings. Since the ERPs are associated with early sensory processing preceding vection, we argue that the ERPs mirror the contribution of sensory cortical processes to vection rather than the subjective sensation of vection per se.     

Visual display height

We examined the influence of backrest inclination and vergence demand on the posture and gaze angle that workers adopt to view visual targets placed in different vertical locations. In the study, 12 participants viewed a small video monitor placed in 7 locations around a 0.65-m radius arc (from 65 degrees below to 30 degrees above horizontal eye height). Trunk posture was manipulated by changing the backrest inclination of an adjustable chair. Vergence demand was manipulated by using ophthalmic lenses and prisms to mimic the visual consequences of varying target distance. Changes in vertical target location caused large changes in atlanto-occipital posture and gaze angle. Cervical posture was altered to a lesser extent by changes in vertical target location. Participants compensated for changes in backrest inclination by changing cervical posture, though they did not significantly alter atlanto-occipital posture and gaze angle. The posture adopted to view any target represents a compromise between visual and musculoskeletal demands. These results provide support for the argument that the optimal location of visual targets is at least 15 degrees below horizontal eye level. Actual or potential applications of this work include the layout of computer workstations and the viewing of displays from a seated posture.     

Control-display mapping in brain-computer interfaces

Event-related potential (ERP) based brain-computer interfaces (BCIs) employ differences in brain responses to attended and ignored stimuli. When using a tactile ERP-BCI for navigation, mapping is required between navigation directions on a visual display and unambiguously corresponding tactile stimuli (tactors) from a tactile control device: control-display mapping (CDM). We investigated the effect of congruent (both display and control horizontal or both vertical) and incongruent (vertical display, horizontal control) CDMs on task performance, the ERP and potential BCI performance. Ten participants attended to a target (determined via CDM), in a stream of sequentially vibrating tactors. We show that congruent CDM yields best task performance, enhanced the P300 and results in increased estimated BCI performance. This suggests a reduced availability of attentional resources when operating an ERP-BCI with incongruent CDM. Additionally, we found an enhanced N2 for incongruent CDM, which indicates a conflict between visual display and tactile control orientations. PRACTITIONER SUMMARY: Incongruency in control-display mapping reduces task performance. In this study, brain responses, task and system performance are related to (in)congruent mapping of command options and the corresponding stimuli in a brain-computer interface (BCI). Directional congruency reduces task errors, increases available attentional resources, improves BCI performance and thus facilitates human-computer interaction.     

Cross-modal links among vision, audition, and touch in complex environments

OBJECTIVES: This study sought to determine whether performance effects of cross-modal spatial links that were observed in earlier laboratory studies scale to more complex environments and need to be considered in multimodal interface design. It also revisits the unresolved issue of cross-modal cuing asymmetries. BACKGROUND: Previous laboratory studies employing simple cues, tasks, and/or targets have demonstrated that the efficiency of processing visual, auditory, and tactile stimuli is affected by the modality, lateralization, and timing of surrounding cues. Very few studies have investigated these cross-modal constraints in the context of more complex environments to determine whether they scale and how complexity affects the nature of cross-modal cuing asymmetries. METHOD: Amicroworld simulation of battlefield operations with a complex task set and meaningful visual, auditory, and tactile stimuli was used to investigate cuing effects for all cross-modal pairings. RESULTS: Significant asymmetric performance effects of cross-modal spatial links were observed. Auditory cues shortened response latencies for collocated visual targets but visual cues did not do the same for collocated auditory targets. Responses to contralateral (rather than ipsilateral) targets were faster for tactually cued auditory targets and each visual-tactile cue-target combination, suggesting an inhibition-of-return effect. CONCLUSIONS: The spatial relationships between multimodal cues and targets significantly affect target response times in complex environments. The performance effects of cross-modal links and the observed cross-modal cuing asymmetries need to be examined in more detail and considered in future interface design. APPLICATION: The findings from this study have implications for the design of multimodal and adaptive interfaces and for supporting attention management in complex, data-rich domains.     

Comparison of perceived comfort differences between standard and experimental load carriage systems

The current popularity of backpack-type load carriage systems (LCS) by students has precipitated a prevalence of postural abnormalities and pain. This study compared subjective perceptual comfort in standard and vertically loaded LCSs. Sixteen females ages 18–23 years rated their personal LCSs for perceived shoulder, neck, and lower back comfort and for overall comfort, each day for two weeks using 100 mm visual analogue scales (VAS). Each scale contained polar extremities of ‘very comfortable’ to ‘very uncomfortable’ and a vertical mark placed on the 100 mm line by the participants indicated their perception of comfort. Following two weeks, participants were given LCSs that distributed the weight vertically and were asked to rate the system in the same way for an additional two-week period. Statistical analysis revealed significant differences in shoulder (p=0.015), neck (p=0.005), and lower back (p=0.036) comfort and overall comfort (p=0.001) between the participants' personal LCSs and the experimental LCS. In conclusion, vertical load placement may redistribute the load in a manner that reduces symptoms of selected anatomical discomfort.     

Effects of display modality on critical battlefield e-map search performance

Visual search performance in visual display terminals can be affected by several changeable display parameters, such as the dimensions of screen, target size and background clutter. We found that when there was time pressure for operators to execute the critical battlefield map searching in a control room, efficient visual search became more important. We investigated the visual search performance in a simulated radar interface, which included the warrior symbology. Thirty-six participants were recruited and a three-factor mixed design was used in which the independent variables were three screen dimensions (7, 15 and 21 in.), five icon sizes (visual angle 40, 50, 60, 70 and 80 min of arc) and two map background clutter types (topography displayed [TD] and topography not displayed [TND]). The five dependent variables were completion time, accuracy, fixation duration, fixation count and saccade amplitude. The results showed that the best icon sizes were 80 and 70 min. The 21 in. screen dimension was chosen as the superior screen for search tasks. The TND map background with less clutters produced higher accuracy compared to that of TD background with clutter. The results of this research can be used in control room design to promote operators’ visual search performance.     

Special Issue in Honor of Ben Shneiderman's 60th Birthday: Reflections on Human-Computer Interaction

Abstract

Unlike most Web portals in the world, Chinese Web portals are characterized by a huge amount of information, excessive visual stimuli, and very long Web pages. The objective of this study is to investigate the effect of such rich Web portal designs and floating animations on visual search, emphasizing a comparison between Chinese users and German users. Two experiments were conducted to test 2 proposed hypotheses. Experiment 1 studied the effect of Web portal design (rich and simple) on visual search performance (performance time, errors, and satisfaction) with both Chinese and German participants. Experiment 2 studied the effects of static animations (leaderboards, couplets, and large squares) and floating animations (moving down, moving up/down, and random movement) on visual search performance on Web portals. The dependent variables were the performance time, error, satisfaction, and animation recognition. The results indicated that participants using simple Web portals searched faster, made fewer errors, and were more satisfied than participants using rich Web portals. No significant differences were found between the performance time of Chinese participants and German participants. However, satisfaction of Chinese participants was found to be less influenced by the differences between simple and rich Web portal designs, compared with German participants. No significant differences were found in performance time and animation recognition between static animations and floating animations, which indicated that users are able to detect the pattern of animation movements and were able to avoid floating animations as well as static animations. People searching pages with randomly floating animations were found to use significantly more time compared with those searching pages with no animations. Furthermore, users' satisfaction for pages with randomly floating animations, moving down animations, and moving up/down animations was significantly lower than for pages with no animations. Implications for designers and for future research are discussed.     

Audial engagement: Effects of game sound on learner engagement in digital game-based learning environments

Learner engagement has been considered to be an important factor for successful learning in multimedia learning environments, which include digital game-based learning (DGBL). Nonetheless, there have been few empirical studies and very little consensus on what elements critically affect learners’ engagement in a DGBL environment. Audial and visual stimuli are often the only means to communicate learning materials to the learners because of the virtual environment found in DGBLs. This means that learners’ engagement in DGBL will need to occur via audial or visual stimuli – sans other specialty devices to immerse players in virtual reality (such as Occulus Rifts and Virtusphere). Yet, very few researchers have directly measured the effect of sensory stimuli on learners’ engagement during (game-based) learning. This study examined the effects of non-player characters’ (NPC) voiceovers on play-learners’ engagement in DGBL through a role-playing game (RPG). A randomized control-group post-test only design was used to collect data from 74 participants. Data analysis revealed that the participants who played the game with voiceovers were more engaged than participants who played without voiceovers. The results of this study will guide practitioners to identify more effective ways of adopting, developing, and modifying digital games for educational purposes.     

With flying colours: Pilot performance with colour-coded head-up flight symbology

The manipulation of colour in display symbology design has long been recognised as a method to improve operator experience and performance. Recent developments in colour head-up display (HUD) and helmet-mounted display (HMD) technology underline the necessity to understand the human factors considerations of symbology colour coding against conventional monochrome symbology formats. In this low-fidelity desktop human-in-the-loop experiment, the colour of flight symbology on an overlaid symbology set was coded as a redundant cue to indicate the accuracy of professional and non-professional pilots’ flight profile across a range of simulated flight manoeuvres. The main finding of this study was that colour coding flight symbology supported the manual flying performance of both professional and non-professional pilots. Notably, colour-coding of the bank indicator and airspeed tape minimised performance error during turning and altitude change manoeuvres, respectively. The usability of colour coded symbology was also rated higher than the monochrome symbology. We conclude that colour coded HUD/HMD symbology is preferred by the user and may improve performance during low workload manual flying tasks. A fuller understanding of performance and workload effects will require future studies to employ higher workload flying tasks and examine the utility of colour coding within higher fidelity environments.     

Effects of display factors on Chinese proofreading performance and preferences

This experiment investigated the influence of four display factors, viz. typeface (Ming Liu and Jheng Hei), font size (10-point and 14-point), text direction (horizontal and vertical) and copy placement (left-right and top-bottom) on Chinese proofreading performance and subjective preferences. Proofreading performance was measured in terms of time and accuracy, and preferences were examined in terms of comfort, ease and fatigue. It was found that the horizontal text direction resulted in faster proofreading than the vertical one, but the other three factors were non significant. The faster proofreading using the horizontal text direction was valid for both the left-right and the top-bottom arrangements. There was an interaction between typeface and font size such that, for the smaller character size (10-point), the Ming Liu style produced a faster performance than the Jheng Hei style. However, for the larger character size (14-point), the Jheng Hei style produced a faster performance than the Ming Liu style. Regarding proofreading accuracy, the number of non-detected missing words and related false alarm in left-right copy placement setting was significantly larger than that in top-down setting. Greater accuracy was obtained in proofreading at the cost of a speed-accuracy trade-off – the higher the accuracy in the proofreading task, the longer the proofreading time. Font size and text direction had significant effects on proofreading preferences, but typeface and copy placement did not.

Practitioner Summary: This study examined four display factors on Chinese proofreading performance and subjective preferences. The findings of this experiment provide information and recommendations that should prove useful for the design of proofreading interfaces to improve proofreading time and accuracy, and to satisfy proofreaders' preference.     

小波变换人体目标特征提取方法效果对比

Haar小波和Gabor小波变换是常用的特征提取方法,前者广泛用于目标检测,后者则常用于人脸识别。针对人体目标检测,提出采用Gabor小波变换进行特征提取,并采用三个主要的行人库与Haar小波方法进行对比实验,实验显示：由于二维Gabor小波变换响应能够在多个尺度的多个方向上对目标的局部区域像素值变化进行描述,所以相比只能在水平、垂直和对角线三个方向上描述目标的Haar小波,其优势明显。     

Visual and memory search in complex environments: determinants of eye movements and search performance

Previous research on visual and memory search revealed various top down and bottom up factors influencing performance. However, utilising abstract stimuli (e.g. geometrical shapes or letters) and focussing on individual factors has often limited the applicability of research findings. Two experiments were designed to analyse which attributes of a product facilitate search in an applied environment. Participants scanned displays containing juice packages while their eye movements were recorded. The familiarity, saliency, and position of search targets were systematically varied. Experiment 1 involved a visual search task, whereas Experiment 2 focussed on memory search. The results showed that bottom up (target saliency) and top down (target familiarity) factors strongly interacted. Overt visual attention was influenced by cultural habits, purposes, and current task demands. The results provide a solid database for assessing the impact and interplay of fundamental top down and bottom up determinants of search processes in applied fields of psychology. Practitioner Summary: Our study demonstrates how a product (or a visual item in general) needs to be designed and placed to ensure that it can be found effectively and efficiently within complex environments. Corresponding product design should result in faster and more accurate visual and memory based search processes.     

Tracking and discrete dual task performance with different spatial stimulus–response mappings

The effect of spatial compatibility on dual-task performance for various display–control configurations was studied using a tracking task and a discrete four-choice response task. A total of 36 participants took part in this study, and they were asked to perform the primary tracking task while at the same time to respond to an occasional signal. Different levels of compatibility between the stimuli and responses of the discrete response task were found to lead to different degrees of influence on the tracking task. However, degradation of performance was observed for both tasks, which was probably due to resource competition for the visual and spatial resources required for simultaneous task operation and required for bimanual responses. No right–left prevalence effect for the spatial compatibility task was observed in this study, implying that the use of unimanual two-finger responses may not provide the right conditions for a significant effect in the horizontal right–left dimension.

Practitioner Summary: The effect of spatial compatibility in multiple display–control configurations was examined in a dual-task paradigm. The analyses of keen competition for visual and spatial resources in processing the dual tasks under different degrees of stimulus–response compatibility provide useful ergonomics design implications and recommendations for visual interfaces requiring frequent visual scanning.     

Using visual and auditory cues to locate out-of-view objects in head-mounted augmented reality

When looking for an object in a complex visual scene, Augmented Reality (AR) can assist search with visual cues persistently pointing in the target’s direction. The effectiveness of these visual cues can be reduced if they are placed at a different visual depth plane to the target they are indicating. To overcome this visual-depth problem, we test the effectiveness of adding simultaneous spatialized auditory cues that are fixed at the target’s location. In an experiment we manipulated which cue(s) were available (visual-only vs. visual + auditory), and which disparity plane relative to the target the visual cue was displayed on. Results show that participants were slower at finding targets when the visual cue was placed on a different disparity plane to the target. However, this slowdown in search performance could be substantially reduced with auditory cueing. These results demonstrate the importance of AR cross-modal cueing under conditions of visual uncertainty and show that designers should consider augmenting visual cues with auditory ones.     

Allocating less attention to central vision during vection is correlated with less motion sickness

Visually induced motion sickness (VIMS) is a common discomfort response associated with vection-provoking stimuli. It has been suggested that susceptibility to VIMS depends on the ability to regulate visual performance during vection. To test this, 29 participants, with VIMS susceptibility assessed by Motion Sickness Susceptibility Questionnaire, were recruited to undergo three series of sustained attention to response tests (SARTs) while watching dot pattern stimuli known to provoke roll-vection. In general, SARTs performance was impaired in the central visual field (CVF), but improved in peripheral visual field (PVF), suggesting the reallocation of attention during vection. Moreover, VIMS susceptibility was negatively correlated with the effect sizes, suggesting that participants who were less susceptible to VIMS showed better performance in attention re-allocation. Finally, when trained to re-allocation attention from the CVF to the PVF, participants experienced more stable vection. Findings provide a better understanding of VIMS and shed light on possible preventive measures.

Practitioner Summary: Allocating less visual attention to central visual field during visual motion stimulation is associated with stronger vection and higher resistance to motion sickness. Virtual reality application designers may utilise the location of visual tasks to strengthen and stabilise vection, while reducing the potential of visually induced motion sickness.     

Effects of interface and spatial ability on manipulation of virtual models in a STEM domain

Virtual models are increasingly employed in STEM education to foster learning about spatial phenomena. However, the roles of the computer interface and students’ cognitive abilities in moderating learning and performance with virtual models are not yet well understood. In two experiments students solved spatial organic chemistry problems using a virtual model system. Two aspects of the virtual model interface were manipulated: display dimensionality (stereoscopic vs. monoscopic displays) and the location of the hand-held device used to manipulate the virtual molecules (co-located with the visual display vs. displaced). The experimental task required participants to interpret the spatial structure of organic molecules and to manipulate the models to align them with orientations and configurations depicted by diagrams in Experiment 1 and three-dimensional models in Experiment 2. Co-locating the interaction device with the virtual image led to better performance in both experiments and stereoscopic viewing led to better performance in Experiment 2. The effect of co-location on performance was moderated by spatial ability in Experiment 1, and the effect of providing stereo viewing was moderated by spatial ability in Experiment 2. The results are in line with the ability-as-compensator hypothesis: participants with lower ability uniquely benefited from the treatment, while those with higher ability were not affected by stereo or co-location. The findings suggest that increased fidelity in a virtual model system may be one way of alleviating difficulties of low-spatial participants in learning spatially demanding content in STEM domains.     

Comparison of perceived comfort differences between standard and experimental load carriage systems

The current popularity of backpack-type load carriage systems (LCS) by students has precipitated a prevalence of postural abnormalities and pain. This study compared subjective perceptual comfort in standard and vertically loaded LCSs. Sixteen females ages 18-23 years rated their personal LCSs for perceived shoulder, neck, and lower back comfort and for overall comfort, each day for two weeks using 100 mm visual analogue scales (VAS). Each scale contained polar extremities of 'very comfortable' to 'very uncomfortable' and a vertical mark placed on the 100 mm line by the participants indicated their perception of comfort. Following two weeks, participants were given LCSs that distributed the weight vertically and were asked to rate the system in the same way for an additional two-week period. Statistical analysis revealed significant differences in shoulder (p=0.015), neck (p=0.005), and lower back (p=0.036) comfort and overall comfort (p=0.001) between the participants' personal LCSs and the experimental LCS. In conclusion, vertical load placement may redistribute the load in a manner that reduces symptoms of selected anatomical discomfort.     

Effects of high pixel density on reading comprehension,proofreading performance,mood state,and physical discomfort

Displays with low pixel densities that were common in the 1980s and 1990s were shown to impair visual performance. Display technology, especially pixel density, has tremendously improved in recent years and new technologies allow densities of 264 ppi and beyond. Two experiments were conducted to test whether there are any measurable benefits of high pixel density displays (264 ppi) over moderate pixel density displays (132 ppi). In Experiment 1, participants performed a reading comprehension task on a display with either high or low pixel density. In Experiment 2, participants’ speed and performance in a proofreading task were compared using the same displays with high and low pixel density. There were no differences in reading comprehension and reading time (Experiment 1) as well as proofreading speed and performance (Experiment 2) between a 132 ppi and a 264 ppi display. However, subjective ratings of physical discomfort revealed significantly more complaints about headache and musculoskeletal strain in the 132 ppi condition than in the 264 ppi condition (Experiment 2). Reading comprehension, reading speed, and proofreading performance are unaffected by pixel densities above 132 ppi, but reading from high-resolution screens seems to be less exhausting at least subjectively. Thus, while large performance differences cannot be expected, displays with high pixel densities (264 ppi and above) have some advantage over displays with moderate (132 ppi or lower) pixel densities.     

Spatial stimulus response compatibility for a horizontal visual display with hand and foot controls

In this study, two experiments were conducted to examine spatial stimulus–response compatibility (SRC) effects for a horizontal visual display with hand and foot controls. In experiment 1, the hand and foot controls were in a hands above and feet below vertical plane, such that the display and controls were orthogonal to each other. In experiment 2, the foot pedals were moved forward and placed directly underneath the front row of signals, resulting in an additional horizontal display and control relationship. The results of experiment 1 revealed a strong orthogonal SRC effect, such that the best performance was for the front signals with hand controls and rear signals with foot pedals, which was not altered with the presence of horizontal location mappings in experiment 2. These findings indicate that the front-hand/rear-foot mapping relationship was quite robust with regard to changes in the relative locations of the hand and foot response devices.

Statement of Relevance: The results of this study provide useful ergonomics recommendations for designing control consoles with visual signals presented in a horizontal plane and control devices operated by hands and feet. They are helpful for improving efficiency and overall system performance in person–machine systems.