Effect of 3D depth on the resolution and the observed image for auto‐stereoscopic multi‐view 3D display

In auto‐stereoscopic multi‐views, blurring occurs due to the incomplete separation of views for non‐zero depths. How this blur affects a 3D image was investigated using the commercial multi‐view 3D. The 3D input signal consisted of the square pattern and the gratings of various width and gray level values of G1 and G2. The various combinations of G1 and G2 were used to investigate the dependence of blur on gray G1 and G2 values. The 3D depth caused blurring, which caused a decrease in contrast modulation. Hence, the 3D resolution determined from contrast modulation was affected by the depth and became worse with increasing depth. Therefore, 3D resolution may be used to define the depth range within which the image degradation due to blurring is acceptable. Blur edge width values at the boundaries of gray G1 and G2 were measured and found to be similar irrespective of G1 and G2 values at the same depth. This was because blur was caused by the incomplete separation of views that are independent of G1 and G2. Hence, the blurriness of the observed 3D image is determined only by the depth. The 3D resolution and blur edge width might be useful to characterize the performance of auto‐stereoscopic multi‐view 3D.     

Influence of 3‐D cross‐talk on qualified viewing spaces in two‐ and multi‐view autostereoscopic displays

Abstract— To estimate the qualified viewing spaces for two‐ and multi‐view autostereoscopic displays, the relationship between image quality (image comfort, annoying ghost image, depth perception) and various pairings between 3‐D cross‐talk in the left and right views are studied subjectively using a two‐view autostereoscopic display and test charts for the left and right views with ghost images due to artificial 3‐D cross‐talk. The artificial 3‐D cross‐talk was tuned to simulate the view in the intermediate zone of the viewing spaces. It was shown that the stereoscopic images on a two‐view autostereoscopic display cause discomfort when they are observed by the eye in the intermediate zone between the viewing spaces. This is because the ghost image due to large 3‐D cross‐talk in the intermediate zone elicits different depth perception from the depth induced by the original images for the left and right views, so the observer's depth perception is confused. Image comfort is also shown to be better for multi‐views, especially the width of the viewing space, which is narrower than the interpupillary distance, where the parallax of the cross‐talking image is small.     

Effect of number of views to the viewing experience with autostereoscopic 3‐D displays

Abstract— The observers' 3‐D viewing experience when the way the content is created and shown on an autostereoscopic 3‐D display alternate is evaluated. The observer's depth impression, and the perceived contour accuracy and image naturalness or peskiness of the content shown on a 3‐D display, has been investigated. In addition, the consequences of the way the content is created to the results from the optical characterization for the same display have been studied. The alternation of the content was realized in two different ways. Firstly, the number of views for creating the image was varied. Two, five, and 14 views were used; the main focus being on testing the same display and treating it as an ordinary two‐view and a 14‐view display with inter‐sub‐view crosstalk. Also, the intermediate condition where five views with non‐uniform view‐specific crosstalk were used has been investigated. Secondly, the way the content is created was varied by using images with computer‐generated content and photos. The effect of these parameters on viewing experience as such and especially the effect of 3‐D crosstalk on the viewing experience were studied.     

Viewpoint image generation for head tracking 3D display using multi‐camera and approximate depth information

A simple and high image quality method for viewpoint image synthesis from multi‐camera images for a stereoscopic 3D display using head tracking is proposed. In this method, slices of images for depth layers are made using approximate depth information, the slices are linearly blended corresponding to the distance between the viewpoint and cameras at each layer, and the layers are overlaid from the perspective of viewpoint. Because the linear blending automatically compensates for depth error because of the visual effects of depth‐fused 3D (DFD), the resulting image is natural to observer's perception. Smooth motion parallax of wide depth range objects induced by viewpoint movement for left‐and‐right and front‐and‐back directions is achieved using multi‐camera images and approximate depth information. Because the calculation algorithm is very simple, it is suitable for real time 3D display applications.     

Subjective evaluation of visual fatigue due to misalignment of motion and still images in a stereoscopic display

Abstract— In this paper, the relative influences of misalignment such as cross‐talk, vertical shift, and motion blur on visual fatigue by using a binocular stereoscopic display has been verified. Experiments were conducted for two cases: a still image and a motion image. They were evaluated by using the simulator sickness questionnaire (SSQ). By changing the disparity angle, cross‐talk, and vertical shift in the still‐image experiment, it was found that the SSQ score of each parameter increased as the amount of each factor increased. With a two‐sample t‐test between the presence and absence of each factor, a significant difference was found for the case of a more than 36‐arcmin disparity, more than 20% cross‐talk, and more than 10% vertical shift. In the motion‐image experiment, in which motion speed and the misalignment were varied, it was found that movement of the disparity angle caused much more visual fatigue in comparison with the misalignment factors, which were cross‐talk and vertical shift. In contrast, motion images in addition to the cross‐talk and/or vertical shift had a slight but unnoticeable relationship to an increase in visual fatigue. Therefore, it was concluded that vertical shift dominated the evaluation for still images, and the movement itself dominated the evaluation for motion images. The results suggest that it is necessary to evaluate visual fatigue according to the representing case, still or motion images, of a 3‐D stereoscopic display.     

Depth‐map generation for multi‐view autostereoscopic 3‐D displays based on the SIFT algorithm constrained by boundary

Abstract— A depth‐map estimation method, which converts two‐dimensional images into three‐dimensional (3‐D) images for multi‐view autostereoscopic 3‐D displays, is presented. The proposed method utilizes the Scale Invariant Feature Transform (SIFT) matching algorithm to create the sparse depth map. The image boundaries are labeled by using the Sobel operator. A dense depth map is obtained by using the Zero‐Mean Normalized Cross‐Correlation (ZNCC) propagation matching method, which is constrained by the labeled boundaries. Finally, by using depth rendering, the parallax images are generated and synthesized into a stereoscopic image for multi‐view autostereoscopic 3‐D displays. Experimental results show that this scheme achieves good performances on both parallax image generation and multi‐view autostereoscopic 3‐D displays.     

Invited Paper: Depth reproducibility of multiview depth‐fused 3‐D display

Abstract— A multi‐view depth‐fused 3‐D (DFD) display that provides smooth motion parallax for wide viewing angles is proposed. A conventional DFD display consists of a stack of two transparent emitting screens. It can produce motion parallax for small changes in observation angle, but its viewing zone is rather narrow due to the split images it provides in inclined views. On the other hand, even though multi‐view 3‐D displays have a wide viewing angle, motion parallax in them is discrete, depending on the number of views they show. By applying a stacked structure to multi‐view 3‐D displays, a wide‐viewing‐angle 3‐D display with smooth motion parallax was fabricated. Experimental results confirmed the viewing‐zone connection of DFD displays while the calculated results show the feasibility of stacked multi‐view displays.     

High‐quality direct‐view display combining multiple integral 3D images

This paper proposes a method for combining multiple integral three‐dimensional (3D) images using direct‐view displays to obtain high‐quality results. A multi‐image combining optical system (MICOS) is used to enlarge and combine multiple integral 3D images without gaps. An optical design with a simple lens configuration that does not require a diffuser plate prevents the deterioration in resolution resulting from lens arrangement errors and the diffuser plate. An experiment was performed to compare a previously developed method with the proposed method, and the latter showed a significant improvement in image quality. A method for expanding the effective viewing angle of the proposed optical design was also developed, and its effectiveness was confirmed experimentally. A prototype device of the proposed optical design was constructed using a high‐density organic light‐emitting diode (OLED) panel with 8K resolution and 1058 ppi pixel density to achieve 311 (H) × 175 (V) elemental images, a viewing angle of 20.6° in both the horizontal and vertical directions, and a display size of 9.1 in. In addition, the proposed optical design enabled making device considerably thinner, ie, with a thickness of only 47 mm.     

Two‐way multi‐view 2‐D/3‐D display combining LC lens and HVxDP panel using novel pixel arrangement

We have developed a two‐way multi‐view 2‐D/3‐D display combining a liquid crystal lens and horizontally and vertically x times‐density pixels (HVxDP) arrangement. The two‐way multi‐view display features the same display resolution in 2‐D and 3‐D modes and a quite small color moiré for landscape and portrait, respectively, when using the HVxDP arrangement. In this paper, we realized suitable 3‐D properties for achieving a good balance between 3‐D moiré and 3‐D crosstalk for landscape and portrait by a two‐way liquid crystal lens with two kinds of focal lengths for the edge part and the center part of the lens.     

3‐D crosstalk and luminance uniformity from angular luminance profiles of multiview autostereoscopic 3‐D displays

Abstract— Autostereoscopic 3‐D display technologies enable a more immersive media experience by adding real depth to the visual content. However, the method used for the creation of a sensation of depth or stereo illusion contains several display design and content‐related issues that need to be carefully considered to maintain sufficient image quality. Conventionally, methods used for 3‐D image‐quality evaluations have been based on subjective testing. Optical measurements, in addition to subjective testing, can be used as an efficient tool for 3‐D display characterization. Objective characterization methods for autostereoscopic displays have been developed. How parameters affecting stereo image quality can be defined and measured, and how their effect on the stereo image quality can be evaluated have been investigated. Developed characterization methods are based on empirically gathered data. In this paper, previously presented methodology for two‐view displays is extended to cover autostereoscopic multiview displays. A distinction between displays where the change in content occurs in clear steps when the user moves in front of the display, and displays where the apparent movement of the objects is more continuous as a function of the head movement is made. Definitions for 3‐D luminance and luminance uniformity, which are equally important, as well as 3‐D crosstalk, which is the dominant factor in the evaluations of the perceived 3‐D image quality, is focused upon.     

Motion‐blur characterization on liquid‐crystal displays

Abstract— In this paper, several methods to characterize motion blur on liquid‐crystal displays are reviewed. Based on the assumptions of smooth‐pursuit eye tracking and one‐frame temporal luminance integration, a simple algorithm has been proposed to calculate the normalized blurred edge width (N‐BEW) and motion‐picture response time (MPRT) with a one‐frame‐time moving‐window function to LC temporal step response curves. A custom measurement system with a fast‐eye‐sensitivity‐compensated photodiode has been developed to characterize motion blur based on LC response curves (LCRCs). MPRT values obtained by using the algorithm mentioned above and those from the smooth‐pursuit‐camera methods agree. Perception experiments were conducted to validate the correspondence between the simulated results and actual perceived images by the human eyes. In addition, the insufficiency of MPRT to evaluate motion blur on impulse‐type light‐generation LCDs, by analyzing the measurement results of a scanning backlight LCD, is discussed.     

Distortion conspicuity on stereoscopically viewed 3D images may correlate to scene content and distortion type

We describe two studies that were aimed towards increasing our understanding of how the visibility of distortions on stereoscopically viewed 3D images is affected by scene content and distortion types. By assuming that subjects' performance would be highly correlated with the visibility of local distorted patches, we analyzed subjects' performance in locating distortion patches when viewing stereoscopic 3D images. Subjects' performances are measured by whether they successfully locate a local distorted patch, the times they spent to finish the task, and subjective quality ratings given by subjects. The visual data used in this work are co‐registered stereo images with co‐registered “ground truth” range (depth) data. Varied statistical analysis methods were used to discuss the significance of our observations. Three observations are drawn from our analyses. First, blur, JPEG, and JP2K distortions in stereo 3D images may be suppressed if one of the left or right views is undistorted. Second, contrast masking ^a does not occur, or is reduced, while viewing white noise distorted stereo 3D images. Third, there is no depth/disparity masking effect when viewing stereo 3D images, but there may be (conversely) depth‐related facilitation ^b effects for blur, JPEG, and JP2K distorted stereo 3D images.     

Depth image‐based rendering for multiview generation

Abstract— Techniques for 3‐D display have evolved from stereoscopic 3‐D systems to multiview 3‐D systems, which provide images corresponding to different viewpoints. Currently, new technology is required for application in multiview display systems that use input‐source formats such as 2‐D images to generate virtual‐view images of multiple viewpoints. Due to the changes in viewpoints, occlusion regions of the original image become disoccluded, resulting in problems related to the restoration of output image information that is not contained in the input image. In this paper, a method for generating multiview images through a two‐step process is proposed: (1) depth‐map refinement and (2) disoccluded‐area estimation and restoration. The first step, depth‐map processing, removes depth‐map noise, compensates for mismatches between RGB and depth, and preserves the boundaries and object shapes. The second step, disoccluded‐area estimation and restoration, predicts the disoccluded area by using disparity and restores information about the area by using information about neighboring frames that are most similar to the occlusion area. Finally, multiview rendering generates virtual‐view images by using a directional rendering algorithm with boundary blending.     

Perceptual attributes of crosstalk in 3D images

Nowadays, crosstalk is probably one of the most annoying distortions in 3D displays. So far, display designers still have a relative lack of knowledge about the relevant subjective attributes of crosstalk and how they are combined in an overall 3D viewing experience model. The aim of the current experiment is to investigate three perceptually important attributes influencing the overall viewing experience: perceived image distortion, perceived depth, and visual strain. The stimulus material used in this experiment consisted of two natural scenes varying in depth (0, 4, and 12 cm camera base distance) and crosstalk level (0, 5, 10, and 15%). Subjects rated the attributes according to the ITU BT.500–10 in a controlled experiment. Results show that image distortion ratings show a clear increase with increasing crosstalk and increasing camera base distance. Especially higher crosstalk levels are more visible at larger camera base distances. Ratings of visual strain and perceived depth only increase with increasing camera base distance and remain constant with increasing crosstalk (at least until 15% crosstalk).     

Projection‐based head‐tracking 3‐D displays

Abstract— This paper describes the construction and operation of four 3‐D displays in which each display produces two images for each eye and thus fits into the category of projection‐based binocular stereoscopic displays. The four 3‐D displays described are pico‐projector‐based, liquid‐ crystal—on—silicon (LCOS) conventional projector‐based, 120‐Hz digital‐light‐processor (DLP) projector‐ based, and the HELIUM3D system. In the first three displays, images are produced on a direct‐view LCD whose conventional backlight is replaced with a projection illumination source that is controlled by a multi‐user head tracker; novel steering optics direct the projector output to regions referred to as exit pupils located at the viewers' eyes. In the HELIUM3D display, the image information is supplied by a horizontally scanned, fast, light valve whose output is controlled by a spatial light modulator (SLM) to direct images to the appropriate viewers' eyes. The current statu s and the multimodal potential of the HELIUM3D display are described.     

True 3‐D scanned voxel displays using single or multiple light sources

Abstract— Conventional stereoscopic displays require viewers to unnaturally keep eye accommodation fixed at one focal distance while they dynamically change vergence to view objects at different distances. This forced decoupling of reflexively linked processes fatigues eyes, causes discomfort, compromises image quality, and may lead to pathologies in developing visual systems. Volumetric displays can overcome this conflict, but only for small objects placed within a limited range of viewing distances and accommodation levels, and cannot render occlusion cues correctly. Our multi‐planar True 3‐D displays generate accommodation cues that match vergence and stereoscopic retinal disparity demands and can display images and objects at viewing distances throughout the full range of human accommodation (from 6.25 cm to infinity), better mimicking natural vision and minimizing eye fatigue.     

High‐resolution autostereoscopic 3‐D projection display with a space‐dividing iris‐plane shutter

Abstract— A high‐resolution autostereoscopic 3‐D projection display with a polarization‐control space dividing the iris‐plane liquid‐crystal shutter is proposed. The polarization‐control iris‐plane shutter can control the direction of stereo images without reducing the image quality of the microdis‐play. This autostereoscopic 3‐D projection display is 2‐D/3‐D switchable and has a high resolution and high luminance. In addition, it has no cross‐talk between the left and right viewing zones, a simple structure, and the capability to show multi‐view images.     

Head tracked retroreflecting 3D display

In this paper, we describe a single‐user glasses‐free (autostereoscopic) 3D display where images from a pair of picoprojectors are projected on to a retroreflecting screen. Real images of the projector lenses formed at the viewer's eyes produce exit pupils that follow the eye positions by the projectors moving laterally under the control of a head tracker. This provides the viewer with a comfortable degree of head movement. The retroreflecting screen, display hardware, infrared head tracker, and means of stabilizing the image position on the screen are explained. The performance of the display in terms of crosstalk, resolution, image distortion, and other parameters is described. Finally, applications of this display type are suggested.     

Measuring 3D crosstalk uniformity and its influence on contrast ratio

An array detector was employed to characterize the crosstalk and contrast ratio uniformity of three‐dimensional (3D) displays. The measurement method is described and demonstrated on a stereoscopic display with passive glasses. The resulting high resolution spatial uniformity maps enable a comprehensive statistical characterization of the display and provide a useful visual assessment tool. The statistical uniformity data are used to evaluate the crosstalk dependence on viewing conditions (such as viewing distance), and show the degradation in display performance that could not be captured with a discrete spot measurement. The measurement method was also employed to examine the influence of crosstalk on contrast ratio. It is shown that for some 3D displays, the crosstalk uniformity can dominate the perceived contrast. A metric is proposed that defines the maximal crosstalk contribution to the perceived contrast for 3D images with small features. The summary parameters extracted from the uniformity maps can be utilized to define the boundary viewing conditions needed to maintain minimum image quality.     

Guiding principles for high‐quality moving picture in LCD TVs

Abstract— The moving‐picture quality of several LCD modules was evaluated by using the quantitative parameter, normalized blurred edge width (N‐BEW), or the N‐BEW value normalized by time (N‐BET), measured and calculated by the developed time‐sequence‐image integration system which has taken LCD‐response characteristics and human‐vision characteristics into consideration. The quality of several LCD TVs is also discussed by using subjective evaluation and the unified quantitative parameter moving‐picture response time (MPRT), which is based on N‐BEW. According to the experimental and calculated results, it is clear that the value of N‐BET can express moving‐picture quality, which depends on the liquid‐crystal response time and the hold‐type character of LCDs. Also, it is confirmed that the value of MPRT can express the moving‐picture quality by comparison with subjective evaluation. The target values of MPRT and N‐BET for the motion‐blur‐less picture are deduced by extrapolating the subjective evaluation results. Then, guidelines to improve the moving picture quality are demonstrated.