Super multi‐view 3D displays reduce conflict between accommodative and vergence responses

A conflict between accommodation and vergence is one possible cause of visual fatigue and discomfort while viewing conventional three‐dimensional displays. Previous studies have proposed the super multi‐view (SMV) display technique to solve the vergence–accommodation conflict, in which two or more parallax images enter the pupil of the eye with highly directional rays. We simultaneously measured accommodative, vergence, and pupillary responses to SMV three‐dimensional displays to examine whether they can reduce the conflict. For comparison, responses to two‐view stereo images and real objects were also measured. The results show that the range of the accommodative response was increased by the SMV images compared with the two‐view images. The slope of the accommodation–vergence response function for the SMV images was similar to that for the real objects rather than the two‐view images. We also found that enhancement of the accommodative range by the SMV images is noticeable with binocular viewing, indicating that vergence‐induced accommodation plays an important role in viewing SMV displays. These results suggest that SMV displays induced a more natural accommodative response than did conventional, two‐view stereo displays. As a result, SMV displays reduced the vergence–accommodation conflict.     

True 3‐D scanned voxel displays using single or multiple light sources

Abstract— Conventional stereoscopic displays require viewers to unnaturally keep eye accommodation fixed at one focal distance while they dynamically change vergence to view objects at different distances. This forced decoupling of reflexively linked processes fatigues eyes, causes discomfort, compromises image quality, and may lead to pathologies in developing visual systems. Volumetric displays can overcome this conflict, but only for small objects placed within a limited range of viewing distances and accommodation levels, and cannot render occlusion cues correctly. Our multi‐planar True 3‐D displays generate accommodation cues that match vergence and stereoscopic retinal disparity demands and can display images and objects at viewing distances throughout the full range of human accommodation (from 6.25 cm to infinity), better mimicking natural vision and minimizing eye fatigue.     

Multiplane displays based on liquid crystals for AR applications

Multiplane displays are capable of displaying 3D scenes with correct focus cues by creating multilayer 2D images in the display volume. Hence, such a 3D display technique could effectively address the accommodation‐vergence conflict (AVC) problem, which is prevalent in augmented reality (AR) displays. In this paper, we review the recent progress on multiplane AR displays based on liquid crystals (LCs) for AR applications. The working principle of multiplane AR displays is illustrated, the electro‐optical properties of the tunable LC devices are investigated and display prototypes are demonstrated. Finally, we discuss the prospects and challenges of multiplane AR displays based on LCs.     

A multi‐plane optical see‐through head mounted display design for augmented reality applications

Augmented reality (AR) display technology greatly enhances users' perception of and interaction with the real world by superimposing a computer‐generated virtual scene on the real physical world. The main problem of the state‐of‐the‐art 3D AR head‐mounted displays (HMDs) is the accommodation‐vergence conflict because the 2D images displayed by flat panel devices are at a fixed distance from the eyes. In this paper, we present a design for an optical see‐through HMD utilizing multi‐plane display technology for AR applications. This approach manages to provide correct depth information and solves the accommodation‐vergence conflict problem. In our system, a projector projects slices of a 3D scene onto a stack of polymer‐stabilized liquid crystal scattering shutters in time sequence to reconstruct the 3D scene. The polymer‐stabilized liquid crystal shutters have sub‐millisecond switching time that enables sufficient number of shutters to achieve high depth resolution. A proof‐of‐concept two‐plane optical see‐through HMD prototype is demonstrated. Our design can be made lightweight, compact, with high resolution, and large depth range from near the eye to infinity and thus holds great potential for fatigue‐free AR HMDs.     

Projection‐based head‐tracking 3‐D displays

Abstract— This paper describes the construction and operation of four 3‐D displays in which each display produces two images for each eye and thus fits into the category of projection‐based binocular stereoscopic displays. The four 3‐D displays described are pico‐projector‐based, liquid‐ crystal—on—silicon (LCOS) conventional projector‐based, 120‐Hz digital‐light‐processor (DLP) projector‐ based, and the HELIUM3D system. In the first three displays, images are produced on a direct‐view LCD whose conventional backlight is replaced with a projection illumination source that is controlled by a multi‐user head tracker; novel steering optics direct the projector output to regions referred to as exit pupils located at the viewers' eyes. In the HELIUM3D display, the image information is supplied by a horizontally scanned, fast, light valve whose output is controlled by a spatial light modulator (SLM) to direct images to the appropriate viewers' eyes. The current statu s and the multimodal potential of the HELIUM3D display are described.     

An integral‐imaging‐based head‐mounted light field display using a tunable lens and aperture array

A head‐mounted light field display based on integral imaging is considered as one of the promising methods that can render correct or nearly correct focus cues and address the well‐known vergence‐accommodation conflict problem in head‐mounted displays. Despite its great potential, it still suffers some of the same limitations of conventional integral imaging‐based displays such as low spatial resolution and crosstalk. In this paper, we present a prototype design using tunable lens and aperture array to render 3D scenes over a large depth range while maintaining high image quality and minimizing crosstalk. Experimental results verify and show that the proposed design could significantly improve the viewing experience.     

Visual comfort enhancement study based on visual attention detection for stereoscopic displays

Although numerous potential causes may lead to visual discomfort when viewing content on three‐dimensional (3D) displays, vergence–accommodation conflict is a particular cause of binocular parallax‐based stereoscopic displays, and it is unavoidable. Based on the study of 3D content visual attention, we proposed a novel stereoscopic depth adjustment method to improve the visual comfort and enhance perceived naturalness. The proposed method combined the 3D image saliency and specific viewing condition to establish a novel model for computing the optimum zero‐disparity plane of stereoscopic image. The results of perception experiments, focused on visual comfort and stereoscopic sensation, supported that the proposed method can significantly enhance stereoscopic viewing comfort and even can improve the stereoscopic sensation by insuring the 3D image fusion.     

Dual-view holographic AR display based on Bragg mismatched reconstruction of the holographic optical element

A method for dual-view holographic display based on Bragg mismatched reconstruction of holographic optical element (HOE) is proposed. Under the Bragg mismatched condition, the reconstructed images are guided into two separated viewing zones to realize dual-view holographic display. Meanwhile, the viewing angle of each perspective is increased to 11.2°, which is almost 2.5 times as large as the traditional holographic display system. The design process of HOE is simple only by interference of plane reference wave and converging spherical signal wave, which has high practicability. Furthermore, the HOE can mix the virtual 3D image with real-world scenes, which could implement augmented reality (AR) display. Experiments validate that the proposed system can achieve dual-view holographic AR three-dimensional (3D) display with accommodation effect.     

Analysis of longitudinal viewing freedom of reduced‐view super multi‐view display and increased longitudinal viewing freedom using eye‐tracking technique

Abstract— The viewing freedom of the reduced‐view super multi‐view (SMV) display was analyzed. It was found that there are separate multiple viewing ranges in the depth direction; thus, a technique that selects an appropriate viewing range to increase the longitudinal viewing freedom has been developed. Pixels of a flat‐panel display viewed by the viewer's eyes through a lenticular lens were determined from three‐dimensional (3‐D) positions of the viewer's eyes, which were obtained using an eye‐tracking system that employed a stereo camera. Parallax images corresponding to the 3‐D positions of the viewer's eyes were generated, which were displayed by the determined pixels. The experimental results show that the proposed technique successfully increased the longitudinal viewing freedom. It is also shown that a video camera was able to focus on the produced SMV images.     

Measurement of the lens accommodation in viewing stereoscopic displays

In observing the stereoscopic display at the viewing distance of 1 m, the amount of the perceived depth was determined by the positions of the crossing point that the viewing direction of two eyes intersect. The positions of the crossing points of stereoscopic stimuli were controlled, and the accommodation was measured by the autorefractometer for the seven participants. Accommodation was also measured when viewing the real film chart which was placed at the same position as these crossing points. The accommodation change when viewing the stereoscopic display was measured to be noticeable only when the crossing point was quite near the participant, but this change was still much smaller compared with the accommodation change when viewing the real film chart. This change in accommodation implies the possible occurrence of fatigue related to the accommodation–convergence conflict, while the constant accommodation within the range of DOF implies no conflict between the accommodation and convergence. This measurement scheme may be used to define the range of DOF where the accommodation remains little changed, and thus define the depth of the 3D object at which no accommodation–convergence conflict occurs, for a given stereoscopic display.     

MUTED: Multi‐user 3‐D display

Abstract— Research described in this paper encompasses the design and building of glasses‐free (autostereoscopic) displays that utilize a direct‐view liquid‐crystal display whose backlight is provided by a projector and novel steering optics. This is controlled by the output of a multi‐user head‐position tracker. As the displays employ spatial multiplexing on a liquid‐crystal‐display screen, they are inherently 2‐D/3‐D switchable with 2‐D being achieved by simply displaying the same image in the left and right channels. Two prototypes are described in this paper; one incorporating a holographic projector and the other a conventional LCOS projector. The LCOS projector version addresses the limitations of brightness, cross‐talk, banding in the images, and laser stability that occur in the holographic projector version. The future development is considered and a comparison between the prototypes and with other 3‐D displays is given.     

Image formation modeling and analysis of near‐eye light field displays

Near‐eye light field displays based on integral imaging through a microlens array provide attractive features like ultra‐compact volume and freedom of the vergence‐accommodation conflict to head‐mounted displays with virtual or augmented reality functions. To enable optimal design and analysis of such systems, it is desirable to have a physical model that incorporates all factors that affect the image formation, including diffraction, aberration, defocusing, and pixel size. Therefore, in this study, using the fundamental Huygens‐Fresnel principle and the Arizona eye model with adjustable accommodation, we develop an image formation model that can numerically calculate the retinal light field image with near‐perfect accuracy, and experimentally verify it with a prototype system. Next, based on this model, the visual resolution is analyzed for different field of views (FOVs). As a result, a rapid resolution decay with respect to FOV caused by off‐axis aberration is demonstrated. Finally, resolution variations as a function of image depth are analyzed based on systems with different central depth planes. Significantly, the resolution decay is revealed to plateau when the image depth is large enough, which is different from real‐image type light field displays.     

Effects of contrast and ambient illumination on visual discomfort of autostereoscopic display

The conflict between vergence and accommodation is the main perceptual factors contributing to visual discomfort when viewing autostereoscopic display. The key factors relevant to the accommodation and vergence are the ambient illumination and the contrast. The current study was a 2 × 3 × 3 mixed design comparing VFSI and VIMS symptoms between 2D and 3D video clips with three contrast levels under three ambient illumination levels on the autostereoscopic display. Twenty participants were required to evaluate the degree of discomfort by filing out questionnaires after watching those video clips. According to the result analysis, the 3D viewing participants reported more severe symptoms compared to 2D. The moderate contrast conditions were found to be the optimum for viewing comfort. The difference between VFSI and VIMS symptoms become larger with increasing contrast. The results also suggested the optimum 3D illumination condition should not be too high to effectively relieve visual discomfort. VIMS symptoms were more sensitive to the changes in ambient illumination than VFSI. Moreover, significant interaction between contrast and ambient illumination was found. Participants felt the most comfortable in the combination effect of moderate level of contrast and the high level of ambient illumination.     

Eye movements and visual discomfort when viewing stereoscopic 3D content

The visual brain fuses the left and right images projected onto the two eyes from a stereoscopic 3D (S3D) display, perceives parallax, and rebuilds a sense of depth. In this process, the eyes adjust vergence and accommodation to adapt to the depths and parallax of the points they gazed at. Conflicts between accommodation and vergence when viewing S3D content potentially lead to visual discomfort. A variety of approaches have been taken towards understanding the perceptual bases of discomfort felt when viewing S3D, including extreme disparities or disparity gradients, negative disparities, dichoptic presentations, and so on. However less effort has been applied towards understanding the role of eye movements as they relate to visual discomfort when viewing S3D. To study eye movements in the context of S3D viewing discomfort, a Shifted-S3D-Image-Database (SSID) is constructed using 11 original nature scene S3D images and their 6 shifted versions. We conducted eye-tracking experiments on humans viewing S3D images in SSID while simultaneously collecting their judgments of experienced visual discomfort. From the collected eye-tracking data, regions of interest (ROIs) were extracted by kernel density estimation using the fixation data, and an empirical formula fitted between the disparities of salient objects marked by the ROIs and the mean opinion scores (MOS). Finally, eye-tracking data was used to analyze the eye movement characteristics related to S3D image quality. Fifteen eye movement features were extracted, and a visual discomfort predication model learned using a support vector regressor (SVR). By analyzing the correlations between features and MOS, we conclude that angular disparity features have a strong correlation with human judgments of discomfort.     

Eyepiece focus and vergence/accommodation conflict: Implications for night‐vision devices

Abstract— Aviation increasingly conducts night operations using night‐vision devices. However, design of some aspects may limit performance. As pointed out by Kotulak, vergence/accommodation mismatch in NVDs, usually due to eyepiece focus misadjustments, is sometimes a source of visual acuity (VA) decrement. The increased separation between sensors, existing in some binocular helmetmounted displays, was also identified to be potentially responsible for decreased VA at short distances. Based upon knowledge pertaining to vergence and accommodation, a study was performed to better understand the problem of dissociation accommodation/convergence. Different conditions of interocular separation and viewing distances were used. Twelve subjects participated and were asked to resolve Landolt C charts using night‐vision goggles. The results show that, with the eyepiece focus fixed at 10 m, the decrease in VA is roughly proportional to the interocular separation, when looking at short distances. A fixed eyepiece focus at 4 m considerably reduces the conflict and results in improved VA. An additional experiment was conducted to investigate the effect of fixing the objective lenses focus at infinity. With this setting, the decrease in VA at a short distance was such that effects of the mismatch accommodation/convergence are no longer apparent regardless of the interocular separation.     

Color holographic display system based on utilization of effective viewing area

In this paper, a color holographic display system based on the utilization of effective viewing area (EVA) is proposed. Color reconstruction is achieved by using a single spatial light modulator (SLM). The EVA of three color reconstructed images is analyzed based on the holographic diffraction theory and the principle of geometrical optics. Then, the effective holograms of three colors are calculated based on the EVA of the system. When three color lasers are used to collimate the SLM, three color reconstructed images can be coincided in the same position with a fast speed of calculation. Moreover, the chromatic aberration can be eliminated and the experimental results verify the feasibility of the proposed system.     

MEMS-based tactile displays

This paper discusses state-of-the-art tactile displays fabricated by a micro-electronic-mechanical-system (MEMS). A tactile display conveys tactile sensations to users by using actuators. Traditional tactile displays consist of large size actuators, such as a motor or an ultrasound vibrator, to convey tactile feedback by vibration. In addition, the tactile sensation of traditional displays has poor resolution. Microelectromechanical system (MEMS) technology, which is a miniature fabrication process, enables etching, sputtering and assembling of miniature structures. Recently, the technology was applied to tactile displays. For example, shape memory alloy (SMA) actuators are widely used in tactile displays to convey roughness or vibration. The actuators are fabricated by a sputtering process and then thinned. The displays convey various tactile sensations, including feedback and tactile sensations of objects such as paper or wood. This paper is a review of tactile displays fabricated by MEMS technology. We also describe the fabrication processes and stimulation methods to present the potential and applications of the displays.     

Cross‐talk acceptability in natural still images for different (auto)stereoscopic display technologies

Abstract— One of the most annoying distortions in (auto)stereoscopic displays is probably cross‐talk, visible as double edges, which is mainly caused by an imperfect separation of left‐ and right‐eye images. For different types of three‐dimensional (3‐D) displays, cross‐talk is caused by different origins, which could result in different levels of perceived image distortion. To evaluate the influence of (auto)stereoscopic display technology on cross‐talk perception, optical measurements and subjective assessments were performed with three different types of 3‐D displays. It is shown with natural still images that the 3‐D display technology with the lowest luminance and contrast level tolerates the highest level of cross‐talk, while still maintaining an acceptable image‐quality level.     

Fast optimization of computer-generated holograms using conjugate gradient

Holographic display, as an ideal three-dimensional (3D) display method, can provide a realistic stereo-perception sense, and is expected to be used in the next generation of augmented reality (AR) prototypes without complex optical devices. While holographic 3D displays have been proposed in the past decades, high-quality holographic reconstructed images have always relied on well-designed generation algorithms, and it is still a challenge to obtain an accurate phase profile fast. In this work, we utilize a first-order optimization algorithm to optimize the phase profile using the conjugate gradient direction instead of the steepest gradient direction to update the phase parameters. We validate this proposed method by comparing the current widely used stochastic gradient method and the wirtinger holography method. The simulation and experimental results illustrate the superiority of the proposed method, which can achieve faster optimization with the same quality metrics.     

Light-field head-mounted displays reduce the visual effort: A user study

Head-mounted displays (HMDs) allow the visualization of virtual content and the change of view perspectives in a virtual reality (VR). Besides entertainment purposes, such displays also find application in augmented reality, VR training, or tele-robotic systems. The quality of visual feedback plays a key role for the interaction performance in such setups. In the last years, high-end computers and displays led to the reduction of simulator sickness regarding nausea symptoms, while new visualization technologies are required to further reduce oculomotor and disorientation symptoms. The so-called vergence–accommodation conflict (VAC) in standard stereoscopic displays prevents intense use of 3D displays, so far. The VAC describes the visual mismatch between the projected stereoscopic 3D image and the optical distance to the HMD screen. This conflict can be solved by using displays with correct focal distance. The light-field HMD of this study provides a close-to-continuous depth and high image resolution enabling a highly natural visualization. This paper presents the first user-study on the visual comfort of light-field displays with a close-to-market HMD based on complex interaction tasks. The results provide first evidence that the light-field technology brings clear benefits to the user in terms of physical use comfort, workload, and depth matching performance.