Human factors of 3‐D displays

Abstract— This paper provides a selected review of a number of important perceptual and human‐factors issues that arise when 3‐D displays are designed and used. This review and analysis draws heavily from the basic vision literature in order to help provide insight for future design solutions for 3‐D displays. Issues discussed include (1) the basics of human stereopsis, (2) interocular crosstalk, (3) oculomotor responding and distance scaling of binocular disparity information, (4) accommodative‐vergence mismatch, and (5) stereoanomly. The paper concludes with a presentation of several recommendations for the design of 3‐D displays.     

Stereoscopic depth perception survives significant interocular luminance differences

Abstract— The effects of interocular luminance differences on stereoscopic depth perception has been investigated. The stimuli were stereoscopic square‐shaped targets created from disparity embedded in a dynamic random‐dot stereogram, which eliminated other cues to depth perception. The results revealed that stereoscopic depth perception survived significant interocular differences in luminance levels, even up to differences of 60%, provided that display luminance was approximately 0.63 cd/m² or higher. In terms of design criteria, developers of stereo displays can use a fairly large range of interocular luminance levels and still induce good stereo depth perception.     

Eyepiece focus and vergence/accommodation conflict: Implications for night‐vision devices

Abstract— Aviation increasingly conducts night operations using night‐vision devices. However, design of some aspects may limit performance. As pointed out by Kotulak, vergence/accommodation mismatch in NVDs, usually due to eyepiece focus misadjustments, is sometimes a source of visual acuity (VA) decrement. The increased separation between sensors, existing in some binocular helmetmounted displays, was also identified to be potentially responsible for decreased VA at short distances. Based upon knowledge pertaining to vergence and accommodation, a study was performed to better understand the problem of dissociation accommodation/convergence. Different conditions of interocular separation and viewing distances were used. Twelve subjects participated and were asked to resolve Landolt C charts using night‐vision goggles. The results show that, with the eyepiece focus fixed at 10 m, the decrease in VA is roughly proportional to the interocular separation, when looking at short distances. A fixed eyepiece focus at 4 m considerably reduces the conflict and results in improved VA. An additional experiment was conducted to investigate the effect of fixing the objective lenses focus at infinity. With this setting, the decrease in VA at a short distance was such that effects of the mismatch accommodation/convergence are no longer apparent regardless of the interocular separation.     

A method for reproducing apparent continuous depth in a stereoscopic display using “Depth‐Fused 3D” technology

Abstract— A new method that can present fine depth increments in a stereoscopic display is proposed. In typical stereoscopic displays, depth can be presented by binocular disparity, but binocular disparity of less than one pixel cannot be displayed because, in general, electronic displays have discrete pixels. We combined binocular disparity and modulation of the edge luminance in DFD (depth‐fused 3D) displays. In an experiment, the perceived depth could be continuously changed by modulating the edge luminance only. Therefore, continuous depth can be produced by combining binocular disparity and modulation of edge luminance distribution.     

Super multi‐view 3D displays reduce conflict between accommodative and vergence responses

A conflict between accommodation and vergence is one possible cause of visual fatigue and discomfort while viewing conventional three‐dimensional displays. Previous studies have proposed the super multi‐view (SMV) display technique to solve the vergence–accommodation conflict, in which two or more parallax images enter the pupil of the eye with highly directional rays. We simultaneously measured accommodative, vergence, and pupillary responses to SMV three‐dimensional displays to examine whether they can reduce the conflict. For comparison, responses to two‐view stereo images and real objects were also measured. The results show that the range of the accommodative response was increased by the SMV images compared with the two‐view images. The slope of the accommodation–vergence response function for the SMV images was similar to that for the real objects rather than the two‐view images. We also found that enhancement of the accommodative range by the SMV images is noticeable with binocular viewing, indicating that vergence‐induced accommodation plays an important role in viewing SMV displays. These results suggest that SMV displays induced a more natural accommodative response than did conventional, two‐view stereo displays. As a result, SMV displays reduced the vergence–accommodation conflict.     

A spatial variant approach for vergence control in complex scenes

The flexibility in primate vision that utilizes active binocular vision is unparalleled even with modern fixed-stereo-based vision systems. However to follow the path of active binocular vision, the difficulty of attaining fixative capabilities is of primary concern. This paper presents a binocular vergence model that utilizes the retino-cortical log polar mapping in the primate vision system. Individual images of the binocular pair were converted to multi-resolution pyramids bearing a coarse-to-fine architecture (low resolution to high resolution) and disparity estimation on these pyramidal resolutions was performed using normalized cross correlation on the log polar images. The model was deployed on an actual binocular vergence system with independent pan-tilt controls for each camera and the system was able to robustly verge on objects even in cluttered environment with real time performance. This paper even presents the experimental results of the system functioning in unbalanced contrast exposures between the two cameras. The results proved favorable for real world robotic vision applications where noise is prevalent. The proposed vergence control model was also compared with a standard window based Cartesian stereo matching method and showed superior performance.     

基于改进Census变换的多特性立体匹配算法

针对当前立体匹配算法存在的匹配准确率低，难以达到实用的高精度水平的问题，提出了一种基于改良的Census变换与色彩信息和梯度测度相结合的多特性立体匹配算法，实现高精度的双目立体匹配。算法首先在初始代价匹配阶段，将改进的Census变换、色彩和梯度测度赋权求和得出可靠的初始匹配代价；在聚合阶段，采取高效快捷的最小生成树聚合，获得匹配代价矩阵；最后根据胜者为王法则得到初始视差图，并引入左右一致性检测等策略优化视差图，获得高精度的视差图，实验阶段对源自Middlebury上的标准测试图进行测试验证，实验结果表明，经本文算法处理得到的15组测试数据集的视差图在非遮挡区域的平均误匹配率为6.81%，算法实时响应性优良。     

Limits of fusion and depth judgment in stereoscopic color displays

The effective use of stereoscopic display systems is dependent, in part, on reliable data describing binocular fusion limits and the accuracy of depth discrimination for such visual display devices. These issues were addressed in three experiments, as were the effects of interocular cross talk. Results showed that limits of fusion were approximately 27.0 min arc for crossed disparity and 24.0 min arc for uncrossed disparity. Subjects were extremely accurate in distinguishing the relative distance among four groups of stimuli, were able to identify a pair of stimuli colocated at the same depth plane within each group, and were fairly accurate in scaling stimuli along the depth dimension. The mean error in using disparity as a depth cue was approximately 2.2 min arc. Interocular cross talk had little effect on fusion limits for 200-ms stimulus presentations but significantly affected fusion for longer (2 s) presentations that enabled vergence responses to be executed. Depth discrimination performance was essentially unaffected by interocular cross talk; however, cross talk significantly influenced subjective ratings of image quality and visual comfort.     

基于自适应匹配窗及多特征融合的立体匹配*

针对局部立体匹配方法中存在的匹配窗口大小选择困难、边缘处视差模糊及弱纹理区域、斜面或曲面匹配精度较低等问题,提出基于CIELAB空间下色度分割的自适应窗选取及多特征融合的局部立体匹配算法.首先,在CIELAB空间上对立体图像对进行色度分割,依据同质区域的分布获取初始匹配支持域,同时估计遮挡区域,更新匹配支持域.然后,基于更新后的匹配支持域,采用自适应权值的线性加权多特征融合匹配方法得到初始视差图.最后,利用左右视差一致性检测方法进行误匹配检验,利用基于分割的均值滤波器进行视差优化及细化,得到稠密匹配视差结果.实验表明文中算法有效,匹配精度较高,尤其在弱纹理区域及斜面等情况下匹配效果较好.     

A common approach to characterizing autostereoscopic and polarization‐based 3‐D displays

Abstract— Autostereoscopic and polarization‐based stereoscopic 3‐D displays recreate 3‐D images by providing different images in the two eyes of an observer. This aim is achieved differently for these two families of 3‐D displays. It is shown that viewing‐angle measurements can be applied to characterize both types of displays. Viewing‐angle luminance measurements are made at different locations on the display surface for each view emitted by the display. For autostereoscopic displays, a Fourier‐optics instrument with an ultra‐high‐angular‐resolution VCMaster3D is used. For polarization‐based displays, a standard Fourier‐optics instrument with additional glass filters is used. Then, what will be seen by an observer in front of the display is computed. Monocular and binocular quality criteria (left‐ and right‐eye contrast, 3‐D contrast) was used to quantify the ability to perceive depth for any observer position. Qualified monocular and binocular viewing spaces (QMVS and QBVS) are deduced. Precise 3‐D characteristics are derived such as maximum 3‐D contrast, optical viewing freedom in each direction, color shifts, and standard contrast. A quantitative comparison between displays of all types becomes possible.     

Visual comfort enhancement study based on visual attention detection for stereoscopic displays

Although numerous potential causes may lead to visual discomfort when viewing content on three‐dimensional (3D) displays, vergence–accommodation conflict is a particular cause of binocular parallax‐based stereoscopic displays, and it is unavoidable. Based on the study of 3D content visual attention, we proposed a novel stereoscopic depth adjustment method to improve the visual comfort and enhance perceived naturalness. The proposed method combined the 3D image saliency and specific viewing condition to establish a novel model for computing the optimum zero‐disparity plane of stereoscopic image. The results of perception experiments, focused on visual comfort and stereoscopic sensation, supported that the proposed method can significantly enhance stereoscopic viewing comfort and even can improve the stereoscopic sensation by insuring the 3D image fusion.     

On the Importance of Being Asymmetric in Stereopsis—Or Why We Should Use Skewed Parallel Cameras

This paper presents a new clever camera sensor, where relative pose determination is not needed, and the sensor is simultaneously capable of using vergence micromovements. Sweeping depth using vergence micromovements promises subpixel depth precision, measuring zero disparity at each time instant. We show that curves preserving zero disparity are exactly conics, nondegenerate or degenerate. Oddly enough, only circles (Vieth-Müller circles) are routinely considered, either theoretically or in practical work, in vergence stereo. Horopters in human vision, cf. Ogle (1932), closely resemble conics.We introduce translational vergence by suggesting the use of a pair of shift-optics CCD cameras. The nonrigidity causes zero disparity curves to become planes, for each fixation. (They are degenerate conics.) We have parallel optical axes, but slanting left and right primary lines of sight. During vergence movements, the primary lines of sight move over time. This has farreaching consequences: Binocular head-eye systems all involve relative camera rotation, to fixate. But, camera rotation is unnecessary. Hence, for relative depth maps, there is no need for measuring camera rotation (relative camera pose) from mechanical sources. Nor are algorithms needed for calculating epipolar lines. The suggested technique removes the need for camera rotations about the optical centers in a binocular head-eye system.     

Measuring visual fatigue and visual discomfort associated with 3‐D displays

Abstract— This research identifies methods for signs of visual complaints associated with stereoscopic displays. Two potential causes for contradictory results concerning these signs in previous research are hypothesized: (1) not all clinical tests are equally appropriate and (2) there is a natural variation in susceptibility amongst people with normal vision. An optometric screening differentiated participants with moderate binocular status (MBS) from those with good binocular status (GBS). In a 2 × 2 within‐subjects design (2‐D vs. 3‐D and MBS vs. GBS), a questionnaire and eight optometric tests (i.e., binocular acuity, aligning prism, fixation disparity, heterophoria, convergent and divergent fusional reserves, vergence facility, and accommodation response) were each administered before and immediately after a reading task. Results revealed that only participants with MBS in 3‐D conditions showed clinically meaningful changes in fusion range, experienced more visual discomfort, and performed worse on the reading task. Acombination of fusion range measurements and self‐report is appropriate for evaluating visual complaints of stereoscopic stills, and people with a MBS are more susceptible to visual complaints associated with stereoscopic displays. A simple measurement tool, i.e., the ratio of reading performance between 2‐D and 3‐D, to categorize people based on their binocular status is proposed.     

Direct recovery of three-dimensional scene geometry from binocularstereo disparity

An analysis of disparity is presented. It makes explicit the geometric relations between a stereo disparity field and a differentially project scene. These results show how it is possible to recover three-dimensional surface geometry through first-order (i.e., distance and orientation of a surface relative to an observer) and binocular viewing parameters in a direct fashion from stereo disparity. As applications of the analysis, algorithms have been developed for recovering three-dimensional surface orientation and discontinuities from stereo disparity. The results of applying these algorithms to natural image binocular stereo disparity information are presented     

基于PatchMatch的半全局高效双目立体匹配算法

近年来双目立体匹配技术发展迅速,高精度、高分辨率、大视差的应用需求无疑对该技术的计算效率提出了更高的要求.由于传统立体匹配算法固有的计算复杂度正比于视差范围,已经难以满足高分辨率、大视差的应用场景.因此,从计算复杂度、匹配精度、匹配原理等多方面综合考虑,提出了一种基于PatchMatch的半全局双目立体匹配算法,在路径...     

Characterizing image quality of autostereoscopic displays by using the maximum luminance at each viewing position

A metric of the 3D image quality of autostereoscopic displays based on optical measurements is proposed. This metric uses each view's luminance contrast, which is defined as the ratio of maximum luminance at each viewing position to total luminance at that position. Conventional metrics of the autostereoscopic display based on crosstalk, which uses “wanted” and “unwanted” lights. However, in case of the multiple‐views‐type autostereoscopic displays, it is difficult to distinguish exactly which lights are wanted lights and which are unwanted lights. This paper assumes that the wanted light has a maximum luminance at the good stereoscopic viewing position, and the unwanted light also has a maximum luminance at the worst pseudo‐stereoscopic viewing position. By using the maximum luminance that is indexed by view number of the autostereoscopic display, the proposed method enables characterizing stereoscopic viewing conditions without using wanted/unwanted light. A 3D image quality metric called “stereo luminance contrast,” the average of both eyes' contrast, is proposed. The effectiveness of the proposed metric is confirmed by the results of optical measurement analyses of different types of autostereoscopic displays, such as the two‐view, scan‐backlight, multi‐view, and integral.     

Vergence dynamics predict fixation disparity.

The neural origin of the steady-state vergence eye movement error, called binocular fixation disparity, is not well understood. Further, there has been no study that quantitatively relates the dynamics of the vergence system to its steady-state behavior, a critical test for the understanding of any oculomotor system. We investigate whether fixation disparity can be related to the dynamics of opponent convergence and divergence neural pathways. Using binocular eye movement recordings, we first show that opponent vergence pathways exhibit asymmetric angle-dependent gains. We then present a neural model that combines physiological properties of disparity-tuned cells and vergence premotor cells with the asymmetric gain properties of the opponent pathways. Quantitative comparison of the model predictions with our experimental data suggests that fixation disparity can arise when asymmetric opponent vergence pathways are driven by a distributed disparity code.     

Purposeful gazing and vergence control for active vision

The purposeful-gazing capability of active vision offers advantages to many manufacturing tasks. This paper discusses the problems associated with purposeful gazing and fixation of attention for active vision. In binocular active vision, gazing at a selected target refers to directing the visual axes to capture the target in the (appropriate part of) the visual field by both sensors (cameras), and holding gaze refers to directing the visual axes of the sensors so as to maintain the target or point of interest in the visual field of both sensors. This paper proposes solutions to the important problems involved in gaze stabilization by developing techniques for vergence error extraction, and vergence servo control. Vergence is the tilt movement process of two visual sensors (in a binocular system) in opposite directions to fixate at a selected point. Binocular gazing is realized by decreasing the disparity which represents the vergence error. In order to obtain the disparity for extraction of vergence error, a phase-based approach that robustly and efficiently estimates vergence disparity is developed. To control vergence, we present a dual sampling-rate approach for vision-sensor-based dynamic servo control.     

基于单目和双目视觉信息的全参考立体图像质量评价模型

在立体图像质量评价领域,有效地模拟人类视觉系统对图像质量进行评价具有重要意义,考虑到人眼的视觉感知特性,基于单目和双目视觉信息构建一种立体图像质量评价模型MB-FR-SIQA。采用基于结构相似性的立体视差算法得到参考和失真立体图像的视差矩阵,结合Gabor能量响应图、显著性图和视差矩阵生成中间视图,并优化左右眼加权系数计算方法,以提高生成中间视图的准确性。分别利用单目图像和中间视图提取单目和双目视觉信息,计算单目质量分数和双目质量分数,并融合得到立体图像的质量分数,达到评价立体图像质量的目的。实验结果表明,MB-FR-SIQA模型在LIVE-I数据库上具有较高的预测精度,其斯皮尔曼等级相关系数、皮尔森线性相关系数、均方根误差分别为0.945、0.951、5.318,且预测的质量分数符合人类主观评估。     

Preferred vertical gaze direction and observation distance

Hill and Kroemer (1986) and Kroemer and Hill (1986) found that the preferred vertical direction of gaze is lower with a nearer binocular stimulus than with a more distant one. A model is proposed that relates this phenomenon to characteristics of the resting state of the oculomotor system. Three predictions of the model were tested, based on measurements of the preferred vertical gaze direction and dark vergence in the same subject sample. On average the effect of observation distance on the preferred vertical gaze direction served to reduce the discrepancy between the resting state of vergence, operationally defined as dark vergence, and actual convergence during inspection of the binocular stimulus. Second, dark vergence data from individual subjects could be successfully used to predict whether they raised or lowered their eyes on inspection of a binocular stimulus as compared with the preferred vertical gaze direction while viewing a monocular stimulus. Finally, predictions of the size of the change of the preferred vertical gaze direction on introduction of a binocular stimulus produced only small and non-significant correlations.