Super multi‐view 3D displays reduce conflict between accommodative and vergence responses

A conflict between accommodation and vergence is one possible cause of visual fatigue and discomfort while viewing conventional three‐dimensional displays. Previous studies have proposed the super multi‐view (SMV) display technique to solve the vergence–accommodation conflict, in which two or more parallax images enter the pupil of the eye with highly directional rays. We simultaneously measured accommodative, vergence, and pupillary responses to SMV three‐dimensional displays to examine whether they can reduce the conflict. For comparison, responses to two‐view stereo images and real objects were also measured. The results show that the range of the accommodative response was increased by the SMV images compared with the two‐view images. The slope of the accommodation–vergence response function for the SMV images was similar to that for the real objects rather than the two‐view images. We also found that enhancement of the accommodative range by the SMV images is noticeable with binocular viewing, indicating that vergence‐induced accommodation plays an important role in viewing SMV displays. These results suggest that SMV displays induced a more natural accommodative response than did conventional, two‐view stereo displays. As a result, SMV displays reduced the vergence–accommodation conflict.     

Visual comfort enhancement study based on visual attention detection for stereoscopic displays

Although numerous potential causes may lead to visual discomfort when viewing content on three‐dimensional (3D) displays, vergence–accommodation conflict is a particular cause of binocular parallax‐based stereoscopic displays, and it is unavoidable. Based on the study of 3D content visual attention, we proposed a novel stereoscopic depth adjustment method to improve the visual comfort and enhance perceived naturalness. The proposed method combined the 3D image saliency and specific viewing condition to establish a novel model for computing the optimum zero‐disparity plane of stereoscopic image. The results of perception experiments, focused on visual comfort and stereoscopic sensation, supported that the proposed method can significantly enhance stereoscopic viewing comfort and even can improve the stereoscopic sensation by insuring the 3D image fusion.     

An integral‐imaging‐based head‐mounted light field display using a tunable lens and aperture array

A head‐mounted light field display based on integral imaging is considered as one of the promising methods that can render correct or nearly correct focus cues and address the well‐known vergence‐accommodation conflict problem in head‐mounted displays. Despite its great potential, it still suffers some of the same limitations of conventional integral imaging‐based displays such as low spatial resolution and crosstalk. In this paper, we present a prototype design using tunable lens and aperture array to render 3D scenes over a large depth range while maintaining high image quality and minimizing crosstalk. Experimental results verify and show that the proposed design could significantly improve the viewing experience.     

Super multi-view and holographic displays using MEMS devices

Holographic displays and super multi-view (SMV) displays have been developed to solve the accommodation–vergence conflict that is responsible for visual fatigue caused by the 3D images that are generated by conventional three-dimensional (3D) displays upon which the eye cannot focus. However, holographic and SMV displays provide 3D images upon which the eye can readily focus so that the accommodation–vergence conflict does not occur. Because these two display techniques require the generation of a very large amount of image data, the high data bandwidth of microelectromechanical (MEMS) devices is effectively utilized. The present article describes the holographic display system that employs a MEMS spatial light modulator (SLM), which increases the screen size and viewing zone angle. Two SMV displays are also described, where one employs MEMS SLMs and the other an array of MEMS projectors. The resolution and the number of viewpoints of the SMV displays have increased. Moreover, the technique using a MEMS SLM to eliminate speckles from holographic reconstructed images is also described.     

Impact of depth of field simulation on visual fatigue: Who are impacted? and how?

While stereoscopic content can be compelling, it is not always comfortable for users to interact with on a regular basis. This is because the stereoscopic content on displays viewed at a short distance has been associated with different symptoms such as eye-strain, visual discomfort, and even nausea. Many of these symptoms have been attributed to cue conflict, for example between vergence and accommodation. To resolve those conflicts, volumetric and other displays have been proposed to improve the user's experience. However, these displays are expensive, unduly restrict viewing position, or provide poor image quality. As a result, commercial solutions are not readily available. We hypothesized that some of the discomfort and fatigue symptoms exhibited from viewing in stereoscopic displays may result from a mismatch between stereopsis and blur, rather than between sensed accommodation and vergence. To find factors that may support or disprove this claim, we built a real-time gaze-contingent system that simulates depth of field (DOF) that is associated with accommodation at the virtual depth of the point of regard (POR). Subsequently, a series of experiments evaluated the impact of DOF on people of different age groups (younger versus older adults). The difference between short duration discomfort and fatigue due to prolonged viewing was also examined. Results indicated that age may be a determining factor for a user's experience of DOF. There was also a major difference in a user's perception of viewing comfort during short-term exposure and prolonged viewing. Primarily, people did not find that the presence of DOF enhanced short-term viewing comfort, while DOF alleviated some symptoms of visual fatigue but not all.     

Measurement of the lens accommodation in viewing stereoscopic displays

In observing the stereoscopic display at the viewing distance of 1 m, the amount of the perceived depth was determined by the positions of the crossing point that the viewing direction of two eyes intersect. The positions of the crossing points of stereoscopic stimuli were controlled, and the accommodation was measured by the autorefractometer for the seven participants. Accommodation was also measured when viewing the real film chart which was placed at the same position as these crossing points. The accommodation change when viewing the stereoscopic display was measured to be noticeable only when the crossing point was quite near the participant, but this change was still much smaller compared with the accommodation change when viewing the real film chart. This change in accommodation implies the possible occurrence of fatigue related to the accommodation–convergence conflict, while the constant accommodation within the range of DOF implies no conflict between the accommodation and convergence. This measurement scheme may be used to define the range of DOF where the accommodation remains little changed, and thus define the depth of the 3D object at which no accommodation–convergence conflict occurs, for a given stereoscopic display.     

A multi‐plane optical see‐through head mounted display design for augmented reality applications

Augmented reality (AR) display technology greatly enhances users' perception of and interaction with the real world by superimposing a computer‐generated virtual scene on the real physical world. The main problem of the state‐of‐the‐art 3D AR head‐mounted displays (HMDs) is the accommodation‐vergence conflict because the 2D images displayed by flat panel devices are at a fixed distance from the eyes. In this paper, we present a design for an optical see‐through HMD utilizing multi‐plane display technology for AR applications. This approach manages to provide correct depth information and solves the accommodation‐vergence conflict problem. In our system, a projector projects slices of a 3D scene onto a stack of polymer‐stabilized liquid crystal scattering shutters in time sequence to reconstruct the 3D scene. The polymer‐stabilized liquid crystal shutters have sub‐millisecond switching time that enables sufficient number of shutters to achieve high depth resolution. A proof‐of‐concept two‐plane optical see‐through HMD prototype is demonstrated. Our design can be made lightweight, compact, with high resolution, and large depth range from near the eye to infinity and thus holds great potential for fatigue‐free AR HMDs.     

Stereoscopic 3‐D display with optical correction for the reduction of the discrepancy between accommodation and convergence

Abstract— This paper describes a method for reducing the discrepancy between accommodation and convergence when viewing stereoscopic 3‐D images. The method uses a newly developed stereoscopic 3‐D display system with a telecentric optical system and a mobile LCD. The examination of a mono‐focal lens showed that a correction lens having the appropriate refractive power and conditions for presenting stereoscopic 3‐D images clearly reduces the discrepancy between accommodation and convergence. The authors also developed a stereoscopic 3‐D display that uses dynamic optical correction to reduce the discrepancy between accommodation and convergence. The display equalizes the theoretical points of accommodation and convergence. The purpose of the development was to expand the regeneration range of a stereoscopic 3‐D image having the appropriate accommodation. An evaluation of the developed display showed that it resolves the discrepancy between convergence and accommodation.     

Image formation modeling and analysis of near‐eye light field displays

Near‐eye light field displays based on integral imaging through a microlens array provide attractive features like ultra‐compact volume and freedom of the vergence‐accommodation conflict to head‐mounted displays with virtual or augmented reality functions. To enable optimal design and analysis of such systems, it is desirable to have a physical model that incorporates all factors that affect the image formation, including diffraction, aberration, defocusing, and pixel size. Therefore, in this study, using the fundamental Huygens‐Fresnel principle and the Arizona eye model with adjustable accommodation, we develop an image formation model that can numerically calculate the retinal light field image with near‐perfect accuracy, and experimentally verify it with a prototype system. Next, based on this model, the visual resolution is analyzed for different field of views (FOVs). As a result, a rapid resolution decay with respect to FOV caused by off‐axis aberration is demonstrated. Finally, resolution variations as a function of image depth are analyzed based on systems with different central depth planes. Significantly, the resolution decay is revealed to plateau when the image depth is large enough, which is different from real‐image type light field displays.     

Eyepiece focus and vergence/accommodation conflict: Implications for night‐vision devices

Abstract— Aviation increasingly conducts night operations using night‐vision devices. However, design of some aspects may limit performance. As pointed out by Kotulak, vergence/accommodation mismatch in NVDs, usually due to eyepiece focus misadjustments, is sometimes a source of visual acuity (VA) decrement. The increased separation between sensors, existing in some binocular helmetmounted displays, was also identified to be potentially responsible for decreased VA at short distances. Based upon knowledge pertaining to vergence and accommodation, a study was performed to better understand the problem of dissociation accommodation/convergence. Different conditions of interocular separation and viewing distances were used. Twelve subjects participated and were asked to resolve Landolt C charts using night‐vision goggles. The results show that, with the eyepiece focus fixed at 10 m, the decrease in VA is roughly proportional to the interocular separation, when looking at short distances. A fixed eyepiece focus at 4 m considerably reduces the conflict and results in improved VA. An additional experiment was conducted to investigate the effect of fixing the objective lenses focus at infinity. With this setting, the decrease in VA at a short distance was such that effects of the mismatch accommodation/convergence are no longer apparent regardless of the interocular separation.     

Relationship between phoria and visual fatigue in autostereoscopic 3D displays

Many people complain about visual fatigue arising from viewing three‐dimensional (3D) displays. This paper investigates relationship between visual fatigue and viewers' phoria for viewing autostereoscopic 3D displays. Visual fatigue is evaluated through subjective symptoms with a questionnaire and optometric indicators comprising fusion range as well as accommodation convergence/accommodation (AC/A) ratio to measure the variation in visual functions. A screening test is adopted to divide the subjects into two groups based on whether they suffer from phoria. Then a 2 × 2 × 2 mixed design experiment is conducted with display type, viewing stage, and visual state as factors to examine visual fatigue during viewing session. The results show that phoria subjects obtain more severe visual fatigue than normal on subjective evaluation. The normal subjects reveal a more marked difference with phoria in fusion range and AC/A ratio after viewing 3D video clip. Fusion range can significantly distinguish between the two‐dimensional (2D) and 3D condition as well as between the pre‐ and post‐viewing stages. The sensitivity and specificity of fusion range is higher than AC/A ratio with respect to viewing of 3D contents, so it is more appropriate as an optometric indicator of visual fatigue for autostereoscopic 3D displays.     

Eye movements and visual discomfort when viewing stereoscopic 3D content

The visual brain fuses the left and right images projected onto the two eyes from a stereoscopic 3D (S3D) display, perceives parallax, and rebuilds a sense of depth. In this process, the eyes adjust vergence and accommodation to adapt to the depths and parallax of the points they gazed at. Conflicts between accommodation and vergence when viewing S3D content potentially lead to visual discomfort. A variety of approaches have been taken towards understanding the perceptual bases of discomfort felt when viewing S3D, including extreme disparities or disparity gradients, negative disparities, dichoptic presentations, and so on. However less effort has been applied towards understanding the role of eye movements as they relate to visual discomfort when viewing S3D. To study eye movements in the context of S3D viewing discomfort, a Shifted-S3D-Image-Database (SSID) is constructed using 11 original nature scene S3D images and their 6 shifted versions. We conducted eye-tracking experiments on humans viewing S3D images in SSID while simultaneously collecting their judgments of experienced visual discomfort. From the collected eye-tracking data, regions of interest (ROIs) were extracted by kernel density estimation using the fixation data, and an empirical formula fitted between the disparities of salient objects marked by the ROIs and the mean opinion scores (MOS). Finally, eye-tracking data was used to analyze the eye movement characteristics related to S3D image quality. Fifteen eye movement features were extracted, and a visual discomfort predication model learned using a support vector regressor (SVR). By analyzing the correlations between features and MOS, we conclude that angular disparity features have a strong correlation with human judgments of discomfort.     

Stereoscopic 3D displays and human performance: A comprehensive review

To answer the question: “what is 3D good for?” we reviewed the body of literature concerning the performance implications of stereoscopic 3D (S3D) displays versus non-stereo (2D or monoscopic) displays. We summarized results of over 160 publications describing over 180 experiments spanning 51 years of research in various fields including human factors psychology/engineering, human–computer interaction, vision science, visualization, and medicine. Publications were included if they described at least one task with a performance-based experimental evaluation of an S3D display versus a non-stereo display under comparable viewing conditions. We classified each study according to the experimental task(s) of primary interest: (a) judgments of positions and/or distances; (b) finding, identifying, or classifying objects; (c) spatial manipulations of real or virtual objects; (d) navigation; (e) spatial understanding, memory, or recall and (f) learning, training, or planning. We found that S3D display viewing improved performance over traditional non-stereo (2D) displays in 60% of the reported experiments. In 15% of the experiments, S3D either showed a marginal benefit or the results were mixed or unclear. In 25% of experiments, S3D displays offered no benefit over non-stereo 2D viewing (and in some rare cases, harmed performance). From this review, stereoscopic 3D displays were found to be most useful for tasks involving the manipulation of objects and for finding/identifying/classifying objects or imagery. We examine instances where S3D did not support superior task performance. We discuss the implications of our findings with regard to various fields of research concerning stereoscopic displays within the context of the investigated tasks.     

Characterizing image quality of autostereoscopic displays by using the maximum luminance at each viewing position

A metric of the 3D image quality of autostereoscopic displays based on optical measurements is proposed. This metric uses each view's luminance contrast, which is defined as the ratio of maximum luminance at each viewing position to total luminance at that position. Conventional metrics of the autostereoscopic display based on crosstalk, which uses “wanted” and “unwanted” lights. However, in case of the multiple‐views‐type autostereoscopic displays, it is difficult to distinguish exactly which lights are wanted lights and which are unwanted lights. This paper assumes that the wanted light has a maximum luminance at the good stereoscopic viewing position, and the unwanted light also has a maximum luminance at the worst pseudo‐stereoscopic viewing position. By using the maximum luminance that is indexed by view number of the autostereoscopic display, the proposed method enables characterizing stereoscopic viewing conditions without using wanted/unwanted light. A 3D image quality metric called “stereo luminance contrast,” the average of both eyes' contrast, is proposed. The effectiveness of the proposed metric is confirmed by the results of optical measurement analyses of different types of autostereoscopic displays, such as the two‐view, scan‐backlight, multi‐view, and integral.     

Objective evaluation of qualified viewing spaces for near‐to‐eye and autostereoscopic two‐view displays

Abstract— Virtual‐image (near‐to‐eye) and two‐view autostereoscopic (3‐D) displays share similar optical properties in the comfortable user position for viewing. In this paper, the definitions and criteria of qualified viewing space (QVS) and qualified stereoscopic viewing space (QSVS) are discussed. Due to the complex nature of these viewing spaces, the related presumptions and the required optical characteristics and their measurements are specified. The effects of different display and observer parameters, such as interpupillary distance, to the resulting viewing spaces are discussed. Finally, real measurement data of two autostereoscopic display devices are presented.     

Multiplane displays based on liquid crystals for AR applications

Multiplane displays are capable of displaying 3D scenes with correct focus cues by creating multilayer 2D images in the display volume. Hence, such a 3D display technique could effectively address the accommodation‐vergence conflict (AVC) problem, which is prevalent in augmented reality (AR) displays. In this paper, we review the recent progress on multiplane AR displays based on liquid crystals (LCs) for AR applications. The working principle of multiplane AR displays is illustrated, the electro‐optical properties of the tunable LC devices are investigated and display prototypes are demonstrated. Finally, we discuss the prospects and challenges of multiplane AR displays based on LCs.     

Simultaneous measurement of lens accommodation and convergence in natural and artificial 3D vision

Recent advances in 3D technology have been accompanied by increasing complaints of visual fatigue. The usual explanation for such fatigue is that accommodation and convergence are mismatched during stereoscopic vision. The aim of this study was to measure fixation distances between lens accommodation and convergence in young subjects while they viewed real objects and 3D video clips. Measurements were made using an original instrument. The 3D video clips were presented to subjects using a liquid crystal shutter glass system. The results showed that when viewing real objects, the diopter values of subjects' accommodation and convergence were similar and changed periodically. This measurement method was thus considered to be appropriate for the measurement of stereoscopic vision. We also investigated lens accommodation and convergence when subjects viewed 3D video clips. Both accommodation and convergence were found to move along with the virtual position of 3D video clips. Therefore, there was little discrepancy between accommodation and convergence during the viewing of 3D images.     

Fixation disparity: binocular vergence accuracy for a visual display at different positions relative to the eyes

Some observers do not fixate accurately at the point of regard: Their vergence angle (between the visual axes of the two eyes) may correspond to points slightly nearer or farther away. This vergence error, or fixation disparity, was measured with nonius (vernier) lines at six positions of a visual display relative to the eyes. At viewing distances of 40, 60, and 100 cm, the display was located either at eye level or at a downward inclination of gaze direction of -25 degrees relative to horizontal. Viewing conditions resembled typical office work. Lowering the screen induced a near shift in mean vergence response of 0.6 min arc, irrespective of viewing distance; the closer the screen, the more distant was the vergence response relative to the target (by 2.5 min arc on average). The slope of this proximity-fixation-disparity curve is an individual parameter of the vergence system. Actual or potential applications of this research include recommendations for the comfortable viewing distance of visual displays.     

Developing visual systems and exposure to virtual reality and stereo displays: some concerns and speculations about the demands on accommodation and vergence

Little is known about the developmental plasticity of the vergence and accommodative systems, an important issue since abnormalities can lead to visual problems, e.g. strabismus. One way of artificially altering the links between accommodation and vergence is to vary the stimulus to vergence while fixing the accommodative stimulus, as is found in virtual reality displays. While it would be of interest to study developmental plasticity in this situation, since many children are exposed to games machines which use this arrangement, no studies to date have tackled this issue. There is, however, some indication that long-term VR viewing in adults can lead to visual problems. It seems important to determine the safety of these systems for the developing human visual system before they come into common use. In this paper, adaptation of the accommodation and vergence systems and the effect of VR viewing in adults is discussed. The sparse literature on adaptation in children is then reviewed, and suggestions made for approaches that would enhance our knowledge of plasticity of accommodation and vergence in children.     

Effects of 3D perspective on head gaze estimation with a multiview autostereoscopic display

Head gaze, or the orientation of the head, is a very important attentional cue in face to face conversation. Some subtleties of the gaze can be lost in common teleconferencing systems, because a single perspective warps spatial characteristics. A recent random hole display is a potentially interesting display for group conversation, as it allows multiple stereo viewers in arbitrary locations, without the restriction of conventional autostereoscopic displays on viewing positions. We represented a remote person as an avatar on a random hole display. We evaluated this system by measuring the ability of multiple observers with different horizontal and vertical viewing angles to accurately and simultaneously judge which targets the avatar is gazing at. We compared three perspective conditions: a conventional 2D view, a monoscopic perspective-correct view, and a stereoscopic perspective-correct views. In the latter two conditions, the random hole display shows three and six views simultaneously. Although the random hole display does not provide high quality view, because it has to distribute display pixels among multiple viewers, the different views are easily distinguished. Results suggest the combined presence of perspective-correct and stereoscopic cues significantly improved the effectiveness with which observers were able to assess the avatar׳s head gaze direction. This motivates the need for stereo in future multiview displays.