基于视差重映射的立体图像视觉舒适度提升

目的 针对人眼观看立体图像内容可能存在的视觉不舒适性,基于视差对立体图像视觉舒适度的影响,提出了一种结合全局线性和局部非线性视差重映射的立体图像视觉舒适度提升方法。方法 首先,考虑双目融合限制和视觉注意机制,分别结合空间频率和立体显著性因素提取立体图像的全局和局部视差统计特征,并利用支持向量回归构建客观的视觉舒适度预测模型作为控制视差重映射程度的约束;然后,通过构建的预测模型对输入的立体图像的视觉舒适性进行分析,就欠舒适的立体图像设计了一个两阶段的视差重映射策略,分别是视差范围的全局线性重映射和针对提取的潜在欠舒适区域内视差的局部非线性重映射;最后,根据重映射后的视差图绘制得到舒适度提升后的立体图像。结果 在IVY Lab立体图像舒适度测试库上的实验结果表明,相较于相关有代表性的视觉舒适度提升方法对于欠舒适立体图像的处理结果,所提出方法在保持整体场景立体感的同时,能更有效地提升立体图像的视觉舒适度。结论 所提出方法能够根据由不同的立体图像特征构建的视觉舒适度预测模型来自动实施全局线性和局部非线性视差重映射过程,达到既改善立体图像视觉舒适度、又尽量减少视差改变所导致的立体感削弱的目的,从而提升立体图像的整体3维体验。     

Visual comfort enhancement study based on visual attention detection for stereoscopic displays

Although numerous potential causes may lead to visual discomfort when viewing content on three‐dimensional (3D) displays, vergence–accommodation conflict is a particular cause of binocular parallax‐based stereoscopic displays, and it is unavoidable. Based on the study of 3D content visual attention, we proposed a novel stereoscopic depth adjustment method to improve the visual comfort and enhance perceived naturalness. The proposed method combined the 3D image saliency and specific viewing condition to establish a novel model for computing the optimum zero‐disparity plane of stereoscopic image. The results of perception experiments, focused on visual comfort and stereoscopic sensation, supported that the proposed method can significantly enhance stereoscopic viewing comfort and even can improve the stereoscopic sensation by insuring the 3D image fusion.     

Visual comfort enhancement in stereoscopic 3D images using saliency-adaptive nonlinear disparity mapping

Perceptually salient regions have a significant effect on visual comfort in stereoscopic 3D (S3D) images. The conventional method of obtaining saliency maps is linear combination, which often weakens the saliency influence and distorts the original disparity range significantly. In this paper, we propose visual comfort enhancement in S3D images using saliency-adaptive nonlinear disparity mapping. First, we obtain saliency-adaptive disparity maps with visual sensitivity to maintain the disparity-based saliency influence. Then, we perform nonlinear disparity mapping based on a sigmoid function to minimize disparity distortions. Finally, we generate visually comfortable S3D images based on depth-image-based-rendering (DIBR). Experimental results demonstrate that the proposed method successfully improves visual comfort in S3D images by producing comfortable S3D images with high mean opinion score (MOS) while keeping the overall viewing image quality.     

DP3 signal as a neuro-indictor for attentional processing of stereoscopic contents in varied depths within the ‘comfort zone’

When viewing stereoscopic 3D (S3D) displays, excessive binocular disparity is one of the major sources for the induced visual fatigue. Limiting the disparity within a certain range termed ‘comfort zone’ is a useful approach to alleviate the negative impact of viewing the S3D displays. Using the event-related potential (ERP) approach, we have previously showed that the differential P3 (DP3) signal evoked in an oddball task could be a useful neural indicator of the involvement of high-level cognitive functions in viewing S3D contents within the comfort zone compared to viewing 2D contents, despite of the similar subjective comfort of viewing experience for the two display conditions. It remains unclear whether the DP3 signal is sensitive enough for examination of the high-level processing when viewing S3D contents in different disparities within the comfort zone. Here, the DP3 signals were recorded while subjects performed an oddball task presented in S3D modality for three disparities (0.15, 0.45 and 0.76 arcdeg) within the comfort zone. Whereas the oddball task was of similar comfort by subjective measurement for different disparities, the DP3 signal showed monotonic delay in latency for increasing disparity, suggesting the capability of the DP3 signal for establishing a ‘cognitive comfort zone’ in terms of the high-level cognitive functions.     

结合立体视觉舒适度的立体图像显著性检测

针对先前的立体图像显著性检测模型未充分考虑立体视觉舒适度和视差图分布特征对显著区域检测的影响,提出了一种结合立体视觉舒适度因子的显著性计算模型.该模型在彩色图像显著性提取中,首先利用SLIC算法对输入图像进行超像素分割,随后进行颜色相似区域合并后再进行二维图像显著性计算;在深度显著性计算中,首先对视差图进行预处理;然后基于区域对比度进行显著性计算;最后,结合立体视觉舒适度因子对二维显著图和深度显著图进行融合,得到立体图像显著图.在不同类型立体图像上的实验结果表明,该模型获得了85%的准确率和78%的召回率,优于现有常用的显著性检测模型,并与人眼立体视觉注意力机制保持良好的一致性.     

融合双目多维感知特征的立体视频显著性检测

目的 立体视频能提供身临其境的逼真感而越来越受到人们的喜爱,而视觉显著性检测可以自动预测、定位和挖掘重要视觉信息,可以帮助机器对海量多媒体信息进行有效筛选。为了提高立体视频中的显著区域检测性能,提出了一种融合双目多维感知特性的立体视频显著性检测模型。方法 从立体视频的空域、深度以及时域3个不同维度出发进行显著性计算。首先,基于图像的空间特征利用贝叶斯模型计算2D图像显著图;接着,根据双目感知特征获取立体视频图像的深度显著图;然后,利用Lucas-Kanade光流法计算帧间局部区域的运动特征,获取时域显著图;最后,将3种不同维度的显著图采用一种基于全局-区域差异度大小的融合方法进行相互融合,获得最终的立体视频显著区域分布模型。结果 在不同类型的立体视频序列中的实验结果表明,本文模型获得了80%的准确率和72%的召回率,且保持了相对较低的计算复杂度,优于现有的显著性检测模型。结论 本文的显著性检测模型能有效地获取立体视频中的显著区域,可应用于立体视频/图像编码、立体视频/图像质量评价等领域。     

Visual comfort in 2D/3D modes of glasses‐type stereoscopic displays

In this study, we compared visual comfort in 2D/3D modes of the pattern retarder (PR) and shutter glasses (SG) stereoscopic displays by changing viewing factors and image contents. The viewing factors include ambient illuminance/monitor luminance/background luminance and image contents mainly are determined with different disparity limits. The degrees of 2D/3D visual comfort were investigated by using various combinations of ambient illuminance, monitor luminance, background luminance, and disparity limit. A series of psychological experiments were also performed to compare 2D and 3D viewing experiences for the passive PR and active SG stereoscopic displays and to discover more comfortable conditions under various variable combinations. The experiment results show that the various variable combinations affecting visual comfort in the passive PR and active SG stereoscopic displays were significantly different. Finally, we suggest more comfortable conditions of viewing 2D and 3D images for the PR and SG stereoscopic displays.     

Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection

It is a challenging task to improve the visual comfort of a stereoscopic 3D (S3D) image with satisfactory viewing experience. In this paper, we propose a visual comfort improvement scheme by adjusting zero-disparity plane (ZDP) for projection. The degree of visual discomfort is predicted by considering three factors: spatial frequency, disparity response, and visual attention. Then, the selection of an optimal ZDP is guided by the predicted visual discomfort map. Finally, the disparity ranges of the crossed and uncrossed disparities are automatically adjusted according to the ZDP as requirements. Experiment results show that the proposed scheme is effective in improving visual comfort while preserving the unchanged depth sensation.     

Leveraging visual attention and neural activity for stereoscopic 3D visual comfort assessment

Visual comfort assessment (VCA) for stereoscopic three-dimensional (S3D) images is a challenging problem in the community of 3D quality of experience (3D-QoE). The goal of VCA is to automatically predict the degree of perceived visual discomfort in line with subjective judgment. The challenges of VCA typically lie in the following two aspects: 1) formulating effective visual comfort-aware features, and 2) finding an appropriate way to pool them into an overall visual comfort score. In this paper, a novel two-stage framework is proposed to address these problems. In the first stage, primary predictive feature (PPF) and advanced predictive feature (APF) are separately extracted and then integrated to reflect the perceived visual discomfort for 3D viewing. Specifically, we compute the S3D visual attention-weighted disparity statistics and neural activities of the middle temporal (MT) area in human brain to construct the PPF and APF, respectively. Followed by the first stage, the integrated visual comfort-aware features are fused with a single visual comfort score by using random forest (RF) regression, mapping from a high-dimensional feature space into a low-dimensional quality (visual comfort) space. Comparison results with five state-of-the-art relevant models on a standard benchmark database confirm the superior performance of our proposed method.     

Influence of depth and 3D crosstalk on blur in multi‐view 3D displays

Current 3D crosstalk equation was defined from the characteristics of 3D display using glasses. This equation is not suitable for multi‐view 3D display with larger view number as it gives the inappropriately large value. In 3D display using eyeglass, double images occur at large depth. But, in multi‐view 3D display with larger view number, blur occurs to larger width for the larger depth. Hence, blur phenomenon of multi‐view 3D display was investigated to understand the unique characteristics of multi‐view 3D display. For this purpose, ray tracing S/W was used to simulate 3D display image seen at the designed viewing distance, to calculate the relative luminance distribution, and to quantify the relation between blur and depth. Calculated results showed that incomplete image separation caused the overlap of multiple view images and the blur. Blur edge width (BEW) was proportional to the horizontal disparity and related to the depth. BEWR = (BEW) / (binocular disparity) was newly defined, and its usefulness for 3D characterization was investigated. BEW and BEWR might be useful as new measuring items to characterize multi‐view 3D display regarding 3D crosstalk.     

Research on the effect of contrast enhancement and illumination condition on 3D visual fatigue

3D visual comfort can be influenced by such common factors as binocular parallax, 3D image contrast, viewing distance, and illumination condition. This paper investigates the effect of contrast enhancement and illumination condition on 3D visual fatigue by performing a visual fatigue measurement experiment, of which subjects need to watch 3D images under two illumination conditions and finish a comfort judgment task. The proportion of discomfort images, the subjective evaluation of visual fatigue under 100 lux and 500 lux illumination conditions, the effect of image contrast enhancement on 3D visual fatigue, and the relationship between visual fatigue and proportion of discomfort images were analyzed, respectively. The experimental results showed that both the contrast enhanced by Laplacian algorithm and low illumination condition have positive influence on 3D visual perception of eyes; the 3D visual fatigue of contrast enhanced by Laplacian algorithm under low illumination condition was less than other situations in the experiment. Moreover, it was examined that the proportion of discomfort images can be potential prediction indicator of the 3D visual fatigue.     

Limits of fusion and depth judgment in stereoscopic color displays

The effective use of stereoscopic display systems is dependent, in part, on reliable data describing binocular fusion limits and the accuracy of depth discrimination for such visual display devices. These issues were addressed in three experiments, as were the effects of interocular cross talk. Results showed that limits of fusion were approximately 27.0 min arc for crossed disparity and 24.0 min arc for uncrossed disparity. Subjects were extremely accurate in distinguishing the relative distance among four groups of stimuli, were able to identify a pair of stimuli colocated at the same depth plane within each group, and were fairly accurate in scaling stimuli along the depth dimension. The mean error in using disparity as a depth cue was approximately 2.2 min arc. Interocular cross talk had little effect on fusion limits for 200-ms stimulus presentations but significantly affected fusion for longer (2 s) presentations that enabled vergence responses to be executed. Depth discrimination performance was essentially unaffected by interocular cross talk; however, cross talk significantly influenced subjective ratings of image quality and visual comfort.     

融合结构与非结构信息的自然图像恰可察觉失真阈值估计

目的 研究表明,图像的恰可察觉失真（JND）阈值主要与视觉系统的亮度适应性、对比度掩模、模块掩模以及图像结构等因素有关。为了更好地研究图像结构对JND阈值的影响,提出一种基于稀疏表示的结构信息和非结构信息分离模型,并应用于自然图像的JND阈值估计,使JND阈值模型与人眼视觉系统具有更好的一致性。方法 首先通过K-均值奇异值分解算法（K-SVD）得到过完备视觉字典。然后利用该过完备字典对输入的自然图像进行稀疏表示和重建,得到该图像对应的结构层和非结构层。针对结构层和非结构层,进一步设计基于亮度适应性与对比度掩模的结构层JND估计模型和基于亮度对比度与信息不确定度的非结构层JND估计模型。最后利用一个能够刻画掩模效应的非线性可加模型对以上两个分量的JND估计模型进行融合。结果 本文提出的JND估计模型利用稀疏表示将自然图像的结构/非结构信息进行分离,然后采用符合各自分量特点的JND模型进行计算,与视觉感知机理高度一致。实验结果表明,本文JND模型能够有效地预测自然图像的JND阈值,受污染图的峰值信噪比（PSNR）值比其他3个JND对比模型值高出35 dB。结论 与现有模型相比,该模型与人眼主观视觉感知具有更好的一致性,更能有效地预测自然图像的JND阈值。     

用户多维感知的3D图像体验质量评价

目的 符合用户视觉特性的3维图像体验质量评价方法有助于准确、客观地体现用户观看3D图像或视频时的视觉感知体验,从而给优化3维内容提供一定的思路。现有的评价方法仅从图像失真、深度感知和视觉舒适度中的一个维度或两个维度出发对立体图像进行评价,评价结果的准确性有待进一步提升。为了更加全面和准确地评价3D图像的视觉感知体验,提出了一种用户多维感知的3D图像体验质量评价算法。方法 首先对左右图像的差异图像和融合图像提取自然场景统计参数表示失真特征;然后对深度图像提取敏感区域,对敏感区域绘制失真前后深度变换直方图,统计深度变化情况以及利用尺度不变特征变换（SIFT）关键点匹配算法计算匹配点数目,两者共同表示深度感知特征;接下来对视觉显著区域提取视差均值、幅值表示舒适度特征;最后综合考虑图像失真、深度感知和视觉舒适度3个维度特征,将3个维度特征归一化后联合成体验质量特征向量,采用支持向量回归（SVR）训练评价模型,并得到最终的体验质量得分。结果 在LIVE和Waterloo IVC数据库上的实验结果表明,所提出的方法与人们的主观感知的相关性达到了0.942和0.858。结论 该方法充分利用了立体图像的特性,评价结果优于比较的几种经典算法,所构建模型的评价结果与用户的主观体验有更好的一致性。     

Impact of depth of field simulation on visual fatigue: Who are impacted? and how?

While stereoscopic content can be compelling, it is not always comfortable for users to interact with on a regular basis. This is because the stereoscopic content on displays viewed at a short distance has been associated with different symptoms such as eye-strain, visual discomfort, and even nausea. Many of these symptoms have been attributed to cue conflict, for example between vergence and accommodation. To resolve those conflicts, volumetric and other displays have been proposed to improve the user's experience. However, these displays are expensive, unduly restrict viewing position, or provide poor image quality. As a result, commercial solutions are not readily available. We hypothesized that some of the discomfort and fatigue symptoms exhibited from viewing in stereoscopic displays may result from a mismatch between stereopsis and blur, rather than between sensed accommodation and vergence. To find factors that may support or disprove this claim, we built a real-time gaze-contingent system that simulates depth of field (DOF) that is associated with accommodation at the virtual depth of the point of regard (POR). Subsequently, a series of experiments evaluated the impact of DOF on people of different age groups (younger versus older adults). The difference between short duration discomfort and fatigue due to prolonged viewing was also examined. Results indicated that age may be a determining factor for a user's experience of DOF. There was also a major difference in a user's perception of viewing comfort during short-term exposure and prolonged viewing. Primarily, people did not find that the presence of DOF enhanced short-term viewing comfort, while DOF alleviated some symptoms of visual fatigue but not all.     

Measuring visual fatigue and visual discomfort associated with 3‐D displays

Abstract— This research identifies methods for signs of visual complaints associated with stereoscopic displays. Two potential causes for contradictory results concerning these signs in previous research are hypothesized: (1) not all clinical tests are equally appropriate and (2) there is a natural variation in susceptibility amongst people with normal vision. An optometric screening differentiated participants with moderate binocular status (MBS) from those with good binocular status (GBS). In a 2 × 2 within‐subjects design (2‐D vs. 3‐D and MBS vs. GBS), a questionnaire and eight optometric tests (i.e., binocular acuity, aligning prism, fixation disparity, heterophoria, convergent and divergent fusional reserves, vergence facility, and accommodation response) were each administered before and immediately after a reading task. Results revealed that only participants with MBS in 3‐D conditions showed clinically meaningful changes in fusion range, experienced more visual discomfort, and performed worse on the reading task. Acombination of fusion range measurements and self‐report is appropriate for evaluating visual complaints of stereoscopic stills, and people with a MBS are more susceptible to visual complaints associated with stereoscopic displays. A simple measurement tool, i.e., the ratio of reading performance between 2‐D and 3‐D, to categorize people based on their binocular status is proposed.     

Attention-Aware Disparity Control in interactive environments

Our paper introduces a novel approach for controlling stereo camera parameters in interactive 3D environments in a way that specifically addresses the interplay of binocular depth perception and saliency of scene contents. Our proposed Dynamic Attention-Aware Disparity Control (DADC) method produces depth-rich stereo rendering that improves viewer comfort through joint optimization of stereo parameters. While constructing the optimization model, we consider the importance of scene elements, as well as their distance to the camera and the locus of attention on the display. Our method also optimizes the depth effect of a given scene by considering the individual user’s stereoscopic disparity range and comfortable viewing experience by controlling accommodation/convergence conflict. We validate our method in a formal user study that also reveals the advantages, such as superior quality and practical relevance, of considering our method.     

Video 2‐D—to—3‐D conversion based on hybrid depth cueing

Abstract— With the maturation of three‐dimensional (3‐D) technologies, display systems can provide higher visual quality to enrich the viewer experience. However, the depth information required for 3‐D displays is not available in conventional 2‐D recorded contents. Therefore, the conversion of existing 2‐D video to 3‐D video becomes an important issue for emerging 3‐D applications. This paper presents a system which automatically converts 2‐D videos to 3‐D format. The proposed system combines three major depth cues: the depth from motion, the scene depth from geometrical perspective, and the fine‐granularity depth from the relative position. The proposed system uses a block‐based method incorporating a joint bilateral filter to efficiently generate visually comfortable depth maps and to diminish the blocky artifacts. By means of the generated depth map, 2‐D videos can be readily converted into 3‐D format. Moreover, for conventional 2‐D displays, a 2‐D image/video depth perception enhancement application is also presented. With the depth‐aware adjustment of color saturation, contrast, and edge, the stereo effect of the 2‐D content can be enhanced. A user study on subjective quality shows that the proposed method has promising results on depth quality and visual comfort.     

Relationship between age differences and display parameters on visual comfort for autostereoscopic display

In order to investigate the age differences for viewing autostereoscopic display, we conduct a 3 × 3 × 3 × 4 mixed design with repeated measurement experiment by using three‐dimensional (3D) video clips. Visual comfort is compared between four age groups with a questionnaire. Results of subjective evaluation are compared through the chi‐square test to check if there is a statistical significance between groups with respect to the distribution of number and proportion according to five levels of visual comfort. Then we examine age differences of visual comfort under three key display parameters including crosstalk, ambient illumination, and disparity. The results indicate that the degree of comfort varies considerably among age groups as the parameters of the display system change. Although the seniors feel most discomfort and the children get best experience in general, there is no statistical difference among the subjects when the ambient illumination is medium or disparity is large. So, it is necessary to take account of the age differences in designing 3D display parameters for enhancing visual comfort.     

Visual comfort in autostereoscopic display

Autostereoscopic displays are likely to become widely used products in the future. However, certain physiological factors, especially visual comfort, limit their development. In this study, four observational parameters – ambient illuminance, image content, scaling ratio, and horizontal distance between major and minor objects – were evaluated to determine the degree of visual comfort offered by 3D computer‐generated images on an autostereoscopic display. Visual comfort score with the range of 0–1 is designed to represent the degree of visual comfort for the 3D images with different manipulations of ambient illuminance, image content, scaling ratio, and horizontal distance between major and minor objects in this study. Subjects were asked to indicate images that produced discomfort. The proportion of images for each condition where participants indicated that viewing the image was comfortable was computed. Images receiving a proportion of 0.5 or greater were classified as acceptable. The disparity ranges over which acceptable images were attained for each participant and for each condition were analyzed with analysis of variance. The analytical results indicate that ambient illuminance and image content have a significant effect on the acceptable disparity range, while scaling ratio and horizontal distance between major and minor objects did not.