Perceptual issues in the use of head-mounted visual displays

OBJECTIVE: We provide a review and analysis of much of the published literature on visual perception issues that impact the design and use of head-mounted displays (HMDs). BACKGROUND: Unlike the previous literature on HMDs, this review draws heavily from the basic vision literature in order to help provide insight for future design solutions for HMDs. METHOD: Included in this review are articles and books found cited in other works as well as articles and books obtained from an Internet search. RESULTS: Issues discussed include the effect of brightness and contrast on depth of field, dark focus, dark vergence, and perceptual constancy; the effect of accommodation-vergence synergy on perceptual constancy, eyestrain, and discomfort; the relationship of field of view to the functioning of different visual pathways and the types of visual-motor tasks mediated by them; the relationship of binocular input to visual suppression; and the importance of head movements, head tracking, and display update lag. CONCLUSION: This paper offers a set of recommendations for the design and use of HMDs. APPLICATION: Consideration of the basic vision literature will provide insight for future design solutions for HMDs.     

Subjective and objective evaluation of visual fatigue caused by continuous and discontinuous use of HMDs

During continuous use of displays, a short rest can relax users' eyes and relieve visual fatigue. As one of the most important devices of virtual reality, head‐mounted displays (HMDs) can create an immersive 3D virtual world. When users have a short rest during the using of HMDs, they will experience a transition from virtual world to real world. In order to investigate how this change affects users' eye condition, we designed a 2 × 2 experiment to explore the effects of short rest during continuous using of HMDs and compared the results with those of 2D displays. The Visual Fatigue Scale, critical flicker frequency, visual acuity, pupillary diameter, and accommodation response of 80 participants were measured to assess the subject's performance. The experimental results indicated that a short rest during the using of 2D displays could significantly reduce users' visual fatigue. However, the experimental results of using HMDs showed that short rest during continuous using of HMD induced more severe symptoms of subjectively visual discomfort, but reduced the objectively visual fatigue.     

Head mounted display

This chapter describes head mounted displays (HMDs) from the viewpoint of the human factors. Because it has two separate display systems, HMDs are especially effective in displaying stereoscopic images. To develop better stereoscopic three-dimensional display technologies, it is important to investigate visual functions such as accommodation and convergence. From the results of the experiments, it is now possible to establish the proper settings for HMD devices to reduce the visual load. An example of the industrial application of an HMD is illustrated.     

Binocular vision in a bi-ocular world: new-generation head-mounted displays avoid causing visual deficit

Our previous research highlighted adverse visual effects after wearing a binocular head-mounted display (HMD) for a 10 min stereoscopic visualization task. We have since proposed a theoretical, explanation based on the conflict between the depth. Such cues presented by image disparity and image focal depth. Such conflict, however, is not evident in all HMD configurations, and we replicated our early trials using a new-generation bi-ocular HMD produced by Virtuality Entertainment Ltd. Using similar, conventional optometric procedures with 50 participants, we observed no problems in the use of this display for immersion periods of to 30 min. This study demonstrates that effective HMDs can be produced through careful design and precision engineering. It also suggests a difference between the presentation of binocular and bi-ocular images and the requirements that they place on the visual system. Factors to consider in the future development of binocular displays are discussed.     

The impact of non‐immersive head‐mounted displays (HMDs) on the visual field

Abstract— Binocular head‐mounted displays (HMDs) that could be used non‐immersively produced substantial interruption of the visual field. Monocular HMDs designed to be used non‐immersively created minimal interruption of the visual fields. The scotomata are small enough to allow the HMD to be worn in mobile situations, but inattention associated with use of the HMD may cause safety concerns for some mobile situations. A small opaque display can be positioned to provide a see‐through functionality.     

Sensitivity to visual speed modulation in head-mounted displays depends on fixation

A primary cause of simulator sickness in head-mounted displays (HMDs) is conflict between the visual scene displayed to the user and the visual scene expected by the brain when the user’s head is in motion. It is useful to measure perceptual sensitivity to visual speed modulation in HMDs because conditions that minimize this sensitivity may prove less likely to elicit simulator sickness. In prior research, we measured sensitivity to visual gain modulation during slow, passive, full-body yaw rotations and observed that sensitivity was reduced when subjects fixated a head-fixed target compared with when they fixated a scene-fixed target. In the current study, we investigated whether this pattern of results persists when (1) movements are faster, active head turns, and (2) visual stimuli are presented on an HMD rather than on a monitor. Subjects wore an Oculus Rift CV1 HMD and viewed a 3D scene of white points on a black background. On each trial, subjects moved their head from a central position to face a 15° eccentric target. During the head movement they fixated a point that was either head-fixed or scene-fixed, depending on condition. They then reported if the visual scene motion was too fast or too slow. Visual speed on subsequent trials was modulated according to a staircase procedure to find the speed increment that was just noticeable. Sensitivity to speed modulation during active head movement was reduced during head-fixed fixation, similar to what we observed during passive whole-body rotation. We conclude that fixation of a head-fixed target is an effective way to reduce sensitivity to visual speed modulation in HMDs, and may also be an effective strategy to reduce susceptibility to simulator sickness.     

Target cuing in visual search: the effects of conformality and display location on the allocation of visual attention

Two experiments were performed to examine how frame of reference (world-referenced vs. screen-referenced) and target expectancy can modulate the effects of target cuing in directing attention for see-through helmet-mounted displays (HMDs). In the first experiment, the degree of world referencing was varied by the spatial accuracy of the cue; in the second, the degree of world referencing was varied more radically between a world-referenced HMD and a hand-held display. Participants were asked to detect, identify, and give azimuth information for targets hidden in terrain presented in the far domain (i.e., the world) while performing a monitoring task in the near domain (i.e., the display). The results of both experiments revealed a cost-benefit trade-off for cuing such that the presence of cuing aided the target detection task for expected targets but drew attention away from the presence of unexpected targets in the environment. Analyses support the observation that this effect can be mediated by the display: The world-referenced display reduced the cost of cognitive tunneling relative to the screen-referenced display in Experiment 1; this cost was further reduced in Experiment 2 when participants were using a hand-held display. Potential applications of this research include important design guidelines and specifications for automated target recognition systems as well as any terrain-and-targeting display system in which superimposed symbology is included, specifically in assessing the costs and benefits of attentional cuing and the means by which this information is displayed.     

Suggestions on the applicability of visual instructions with see-through head mounted displays depending on the task

Task instructions have traditionally been communicated orally in many fields. However, recently more and more wearable displays, such as the see-through head mounted displays (HMDs) have been developed, and some studies have provided ideas on applying visual instruction using these new interfaces to particular situations. However, in some cases, where instructions are communicated amongst the workers, the data is not sufficient for field workers to choose the best way of communicating instructions depending on the situation. Thus, this study aims to clarify the cases in which it is effective to apply visual instructions with HMDs, and to provide information that suggests the applicability of such visual instructions instead of or in addition to the traditional auditory instructions in different situations. These suggestions will be a useful reference for workers in safety-critical fields, helping them make better decisions about whether, when, and where to introduce the new method of instructions. It will also address some of the unsolved problems in the field, such as errors, low efficiency, and discomfort in communication.     

Effects of curved computer displays on visual performance,visual fatigue,and emotional image quality

This study investigated the effects of the curvature of curved computer displays on the visual performance, fatigue, and emotional image quality experienced by users during the performance of visual tasks. The characteristics of curved displays were compared and analyzed for different curvatures to investigate whether the curved displays showed improvements in terms of convenience of use when compared with flat displays. The visual performance results for the evaluated displays can be classified into three groups according to their response times, as (2000R = 2500R) > (3000R) > (3800R = 4500R = Flat). The visual fatigue results for average pupil size during a search task was 3.468 mm for flat and 3.355 mm for 2000R. This means that a 2000R curved display was found to be easier to use and thus generated less visual fatigue than a flat display when performing the same visual search task. Further, the results indicated that visual performance improved and visual fatigue decreased significantly as curvature increased. In this study, a curvature limit of 1700R was derived, when considering the image distortion of a 34‐in 21:9 curved display, and there was found to be no difference in the emotional image quality at curvatures greater than 2000R.     

Suggestions on the applicability of visual instructions with see-through head mounted displays depending on the task

Task instructions have traditionally been communicated orally in many fields. However, recently more and more wearable displays, such as the see-through head mounted displays (HMDs) have been developed, and some studies have provided ideas on applying visual instruction using these new interfaces to particular situations. However, in some cases, where instructions are communicated amongst the workers, the data is not sufficient for field workers to choose the best way of communicating instructions depending on the situation. Thus, this study aims to clarify the cases in which it is effective to apply visual instructions with HMDs, and to provide information that suggests the applicability of such visual instructions instead of or in addition to the traditional auditory instructions in different situations. These suggestions will be a useful reference for workers in safety-critical fields, helping them make better decisions about whether, when, and where to introduce the new method of instructions. It will also address some of the unsolved problems in the field, such as errors, low efficiency, and discomfort in communication.     

Effects of displacement magnitude and direction of auditory cues on auditory spatial facilitation of visual search

OBJECTIVE: The objective of this study is to examine the effects of cue error on auditory spatial facilitation (ASF) of visual search. BACKGROUND: ASF is the reduction in time needed to locate and identify a visual target when an auditory cue is presented at the location of the target. Although ASF has been shown to occur when the auditory cue coincides with the target location, it is important to determine whether facilitatory effects are also evident when the cue is displaced. METHOD: Participants performed a visual search task in the presence of an auditory cue that was presented at the center of the screen (uninformative), at the location of the target (accurate), or displaced up to 12 degrees from the target horizontally or vertically. RESULTS: Generally, displaced auditory cues reduced search times as compared with a condition in which the cue was uninformative. When the displacement was always along a single spatial dimension, the cue was as effective as a coincident cue if it was within the local visual area. However, when the dimension along which the cue was displaced varied randomly, the cue did not necessarily reduce search time and hurt performance when the visual search task was difficult. CONCLUSION: Designers of virtual audio displays should be aware that auditory cue accuracy will be affected by the difficulty of the visual task and the operators' knowledge of cue precision and reliability. APPLICATION: Findings from this study can be applied to the design of multimodal interfaces and augmented or virtual environments.     

Using head-mounted displays to examine adaptation and calibration under varying perturbations

Simulator sickness has been a pervasive problem as head-mounted displays (HMDs) grow in popularity. Recent work showed that people can adapt to constant latency in an HMD, but latency that varies has not been examined. In this study, participants performed a shooting task while wearing an HMD during three sessions separated by 48 h under conditions of constant or varying latency. Performance was assessed for both accuracy (targets hit) and speed (time-to-hit targets). It was hypothesized that participants would adapt to constant, but not varying latency as indicated by decreasing simulator sickness over time. Further, it was hypothesized that participants would improve performance over time for both conditions due to practice, but the constant latency group would improve at a faster rate. Results showed reduced sickness with session regardless of latency condition. A similar trend was shown where performance improved with each session, with no effect of the latency condition. Change in sickness and performance were not correlated, suggesting that the changes were independently driven. These findings showed that people reduced sickness and improved performance with repeated exposure, even when experiencing different perturbations. This finding has implications for both HMDs and general understanding of the relationship between latency and adaptation.     

Selecting Menu Items in Mobile Head-Mounted Displays: Effects of Selection Technique and Active Area

ABSTRACT

Head-mounted displays (HMDs) are increasingly available to users after the launch of new-generation consumer devices. Moreover, mobile HMDs such as Samsung Gear VR and Google Daydream View allow users to experience VR through a smartphone, without requiring connection to a PC. Commercial applications for mobile HMDs exploit different techniques to perform menu selection tasks. This paper contrasts the two most used techniques, i.e., dwell-based and touchpad-based selection, which were not experimentally compared before. We consider different versions of a menu pointing and selection task in which participants interacted with a Samsung Gear VR. Results show that participants were slower with the dwell-based technique rather than the touchpad-based technique. However, the dwell-based technique led to fewer errors and was perceived as more usable, more comfortable and less fatiguing than the touchpad-based technique. We also evaluated two different active areas for the selection, discussing the results.     

Exploratory factor analysis and validity of the virtual reality symptom questionnaire and computer use survey

Abstract

The widespread use of virtual reality head-mounted-displays (HMDs) calls for a re-examination of the impact of prolonged exposure to fixed visual displays at close ocular proximity. The purpose of this study is to validate the Virtual Reality Symptoms Questionnaire (VRSQ), created to understand symptoms of prolonged HMDs use, and Computer Use Survey (CUS), created to assess general physical and visual discomfort symptoms. Participants (N?=?100) recorded their general discomfort symptoms using the CUS, performed an interactive task using a HMD for thirty minutes, and then answered the CUS again along with the VRSQ. VRSQ, analysed using an exploratory factor analysis, indicated a clear two-factor solution, and demonstrated very good internal consistency (α?=?0.873). The CUS, also analysed using an exploratory factor analysis, indicated a four-factor solution, and demonstrated good internal consistency (α?=?0.838).

Practitioner Summary: A quantitative-experimental study was conducted to explore the factor structure and validate both the Virtual Reality Symptoms Questionnaire (VRSQ), and the Computer Use Survey (CUS). Findings indicate the VRSQ and CUS are precise and accurate survey instruments for evaluating discomfort after VR-HMD use and the latter for computer use.

Abbreviations: VRSQ: virtual reality symptom questionnaire; CUS: computer use survey; OLED: organic light-emitting diode; MSQ: pensacola motion symptom questionnaire; SSQ: simulator sickness questionnaire; 3?D: three-dimensional computer generated space; VR: virtual reality; VR-HMD: virtual reality head-mounted-display; HMDs: head-mounted-displays; EFA: exploratory factor analysis     

A study on differences in human perception between a real and an AR scene viewed in an OST‐HMD

With the recent growth in the development of augmented reality (AR) technologies, it is becoming important to study human perception of AR scenes. In order to detect whether users will suffer more from visual and operator fatigue when watching virtual objects through optical see‐through head‐mounted displays (OST‐HMDs), compared with watching real objects in the real world, we propose a comparative experiment including a virtual magic cube task and a real magic cube task. The scores of the subjective questionnaires (SQ) and the values of the critical flicker frequency (CFF) were obtained from 18 participants. In our study, we use several electrooculogram (EOG) and heart rate variability (HRV) measures as objective indicators of visual and operator fatigue. Statistical analyses were performed to deal with the subjective and objective indicators in the two tasks. Our results suggest that participants were very likely to suffer more from visual and operator fatigue when watching virtual objects presented by the OST‐HMD. In addition, the present study provides hints that HRV and EOG measures could be used to explore how visual and operator fatigue are induced by AR content. Finally, three novel HRV measures are proposed to be used as potential indicators of operator fatigue.     

Partitioning visual displays aids task-directed visual search

We reduced time to detect target symbols in mock radar screens by adding perceptual boundaries that partitioned displays in accordance with task instructions. Targets appeared among distractor symbols either close to or far from the display center, and participants were instructed to find the target closest to the center. Search time increased with both number of distractors and distance of target from center. However, when close and far regions were delineated by a centrally presented "range ring," the distractor effect was substantially reduced. In addition, eye movement patterns more closely resembled a task-efficient spiral when displays contained a range ring. Results suggest that the addition of perceptual boundaries to visual displays can help to guide search in accordance with task-directed constraints. Actual or potential applications of this research include the incorporation of perceptual boundaries into display designs in order to encourage task-efficient scanpaths (as identified via task analysis and/or empirical testing).     

Light-field head-mounted displays reduce the visual effort: A user study

Head-mounted displays (HMDs) allow the visualization of virtual content and the change of view perspectives in a virtual reality (VR). Besides entertainment purposes, such displays also find application in augmented reality, VR training, or tele-robotic systems. The quality of visual feedback plays a key role for the interaction performance in such setups. In the last years, high-end computers and displays led to the reduction of simulator sickness regarding nausea symptoms, while new visualization technologies are required to further reduce oculomotor and disorientation symptoms. The so-called vergence–accommodation conflict (VAC) in standard stereoscopic displays prevents intense use of 3D displays, so far. The VAC describes the visual mismatch between the projected stereoscopic 3D image and the optical distance to the HMD screen. This conflict can be solved by using displays with correct focal distance. The light-field HMD of this study provides a close-to-continuous depth and high image resolution enabling a highly natural visualization. This paper presents the first user-study on the visual comfort of light-field displays with a close-to-market HMD based on complex interaction tasks. The results provide first evidence that the light-field technology brings clear benefits to the user in terms of physical use comfort, workload, and depth matching performance.     

Driving and side task performance: the effects of display clutter, separation, and modality

In-vehicle technologies (IVTs) create additional tasks for the driver. To the extent that these devices degrade driving performance, there will be safety concerns. This study examines the effects of display clutter from overlay, display separation, and modality on driving and IVT task performance. In a fixed-base simulator, 22 drivers drove different routes and responded to infrequent, unexpected road hazards while engaging in a phone number task presented by different displays. Visual displays were located on a head-up (overlaid on the visual horizon or adjacently, just above the vehicle hood) or head-down display (HDD) located near the midconsole. Alternatively, digits were presented auditorily. In general, there were no differences in performance for the adjacent and overlay displays; however, there were costs associated with the HDD and auditory display for some measures. In particular, responses to hazard events were slowed when drivers used the HDD. Overall, the adjacent display best supported performance on all relevant tasks. Potential applications of this research include the design of IVTs with respect to location and modality.     

Eye movement and pupillary response indices of mental workload during visual search of symbolic displays

Eye movement and pupillary response measures (in addition to search time and accuracy) were collected as indices of visual workload during two experiments designed to evaluate the addition of colour coding to a symbolic tactical display. Displays also varied with regard to symbol density and the type of information participants were required to abstract from the display. These variables were factorially manipulated to examine the effects of colour coding in conditions of varying difficulty. In Experiment 1 (n = 8), search time and the number of eye fixations were affected by all variables and in a similar manner; fixation dwell time and the pupillary response dissociated from the other measures. Compared to monochrome displays, colour coding facilitated search (reduced search time, but not accuracy) during exhaustive search, but had no effect during self-terminating search. Experiment 2 (n = 8) was a replication of Experiment 1 with a pseudo-search control condition added to examine further the pupillary response measures: in particular, to assess the effects of the physical parameters of the displays, and to verify the findings of Experiment 1. Pupillary response measures were sensitive to the information processing demands of the search task, not merely to the physical parameters of the display. Further, the search time, accuracy, and eye movement results from the active search condition generally replicated Experiment 1, but the fixation dwell time data did not. These between-study differences were interpreted as indicating the importance of participant search strategy.     

Depth of focus and visual recognition of imagery presented on simultaneously viewed displays: implications for head-mounted displays

OBJECTIVE: We sought to determine the optimal focal distance for a semitransparent monocular head-mounted display (HMD) integrated with a flight simulator display and to investigate whether observers experienced visual discomfort or impaired target recognition when using an HMD set at the optimal distance. BACKGROUND: When an observer wears a monocular HMD and views a simulator display, focal distances of both displays must be within the observers' depth of focus to prevent blurred imagery. Because focal distance can vary by as much as 0.5 m in U.S. Air Force multifaceted simulator displays, we determined whether a monocular HMD could be integrated with a simulator display without blurred imagery or discomfort. METHOD: Depth of focus and visual recognition were measured with a staircase procedure, and visual discomfort was measured with a questionnaire. RESULTS: Depth of focus was 0.64 diopters in one condition tested, but it was affected by luminance level and display resolution. It was recommended that HMD focal distance equal the optical midpoint of the range of viewing distances encountered in the simulator. Moreover, wearing an HMD produced a decline in recognition performance for targets presented on the simulator display despite both displays being within observers' depth of focus and producing no visual discomfort. CONCLUSION: Monocular HMDs can be integrated with multifaceted simulator displays without blurred imagery or visual discomfort, provided that the correct focal distance is adopted. APPLICATION: For situations involving simultaneously viewed visual displays.