Visual comfort in 2D/3D modes of glasses‐type stereoscopic displays

In this study, we compared visual comfort in 2D/3D modes of the pattern retarder (PR) and shutter glasses (SG) stereoscopic displays by changing viewing factors and image contents. The viewing factors include ambient illuminance/monitor luminance/background luminance and image contents mainly are determined with different disparity limits. The degrees of 2D/3D visual comfort were investigated by using various combinations of ambient illuminance, monitor luminance, background luminance, and disparity limit. A series of psychological experiments were also performed to compare 2D and 3D viewing experiences for the passive PR and active SG stereoscopic displays and to discover more comfortable conditions under various variable combinations. The experiment results show that the various variable combinations affecting visual comfort in the passive PR and active SG stereoscopic displays were significantly different. Finally, we suggest more comfortable conditions of viewing 2D and 3D images for the PR and SG stereoscopic displays.     

基于视差重映射的立体图像视觉舒适度提升

目的 针对人眼观看立体图像内容可能存在的视觉不舒适性,基于视差对立体图像视觉舒适度的影响,提出了一种结合全局线性和局部非线性视差重映射的立体图像视觉舒适度提升方法。方法 首先,考虑双目融合限制和视觉注意机制,分别结合空间频率和立体显著性因素提取立体图像的全局和局部视差统计特征,并利用支持向量回归构建客观的视觉舒适度预测模型作为控制视差重映射程度的约束;然后,通过构建的预测模型对输入的立体图像的视觉舒适性进行分析,就欠舒适的立体图像设计了一个两阶段的视差重映射策略,分别是视差范围的全局线性重映射和针对提取的潜在欠舒适区域内视差的局部非线性重映射;最后,根据重映射后的视差图绘制得到舒适度提升后的立体图像。结果 在IVY Lab立体图像舒适度测试库上的实验结果表明,相较于相关有代表性的视觉舒适度提升方法对于欠舒适立体图像的处理结果,所提出方法在保持整体场景立体感的同时,能更有效地提升立体图像的视觉舒适度。结论 所提出方法能够根据由不同的立体图像特征构建的视觉舒适度预测模型来自动实施全局线性和局部非线性视差重映射过程,达到既改善立体图像视觉舒适度、又尽量减少视差改变所导致的立体感削弱的目的,从而提升立体图像的整体3维体验。     

Exploring crosstalk perception for stereoscopic 3D head‐up displays in a crosstalk simulator

In today's upper class vehicles, head‐up displays (HUDs) are state of the art human–machine interfaces used to increase driving security and comfort. Working with a stereoscopically perceived viewing impression, a novel automotive 3D HUD would allow the simultaneous visualization of multiple contents in various depth planes. However, stereoscopic crosstalk is one of the most important parameters influencing the image quality of stereoscopic systems. We present a setup to simulate crosstalk including a specific software solution. It constitutes a platform to explore the effects of stereoscopic crosstalk in the augmented reality viewing conditions of a 3D HUD. As the setup allows the variation of numerous parameters, it provides a versatile research platform. By means of it, we investigate the variation of the visibility and acceptability thresholds of crosstalk over various parameters. Our results include the finding that they decrease with rising contrast affirming this behavior known for conventional stereoscopic displays for stereoscopic see‐through displays.     

Characterization of crosstalk in stereoscopic display devices

Many different types of stereoscopic display devices are used for commercial and research applications. Stereoscopic displays offer the potential to improve performance in detection tasks for medical imaging diagnostic systems. Due to the variety of stereoscopic display technologies, it remains unclear how these compare with each other for detection and estimation tasks. Different stereo devices have different performance trade‐offs due to their display characteristics. Among them, crosstalk is known to affect observer perception of 3D content and might affect detection performance. We measured and report the detailed luminance output and crosstalk characteristics for three different types of stereoscopic display devices. We recorded the effect of other issues on recorded luminance profiles such as viewing angle, use of different eye wear, and screen location. Our results show that the crosstalk signature for viewing 3D content can vary considerably when using different types of 3D glasses for active stereo displays. We also show that significant differences are present in crosstalk signatures when varying the viewing angle from 0 degrees to 20 degrees for a stereo mirror 3D display device. Our detailed characterization can help emulate the effect of crosstalk in conducting computational observer image quality assessment evaluations that minimize costly and time‐consuming human reader studies.     

Disparity-based just-noticeable-difference model for perceptual stereoscopic video coding using depth of focus blur effect

Human 3D perception provides an important clue to the removal of redundancy in stereoscopic 3D (S3D) videos. Because objects outside the binocular fusion limit cannot be fused on retina, the human visual system (HVS) makes them blur according to the depth-of-focus (DOF) effect to increase the binocular fusion limit and suppress diplopia, i.e. double vision. Based on human depth perception, we propose a disparity-based just-noticeable-difference model (DJND) to save bit-rate and improve visual comfort in S3D videos. We combine the DOF blur effect with conventional JND models in the pixel domain into DJND. Firstly, we use disparity information to get the average disparity value of each block. Then, we integrate the DOF blur effect into luminance JND (LJND) by a selective low pass Gaussian filter to minimize the visual stimulus in S3D videos. Finally, we incorporate disparity information into the filtered JND models to obtain DJND. Experimental results demonstrate that the proposed method successfully improves both image quality and visual comfort in viewing S3D videos without increasing the bit-rate.     

Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection

It is a challenging task to improve the visual comfort of a stereoscopic 3D (S3D) image with satisfactory viewing experience. In this paper, we propose a visual comfort improvement scheme by adjusting zero-disparity plane (ZDP) for projection. The degree of visual discomfort is predicted by considering three factors: spatial frequency, disparity response, and visual attention. Then, the selection of an optimal ZDP is guided by the predicted visual discomfort map. Finally, the disparity ranges of the crossed and uncrossed disparities are automatically adjusted according to the ZDP as requirements. Experiment results show that the proposed scheme is effective in improving visual comfort while preserving the unchanged depth sensation.     

时空联合视差优化的立体视频重定向

目的智能适配显示的图像/视频重定向技术近年受到广泛关注。与图像重定向以及2D视频重定向相比,3D视频重定向需要同时考虑视差保持和时域保持。现有的3D视频重定向方法虽然考虑了视差保持却忽略了对视差舒适度的调整,针对因视差过大和视差突变造成视觉不舒适度这一问题,提出了一种基于时空联合视差优化的立体视频重定向方法,将视频视差范围控制在舒适区间。方法在原始视频上建立均匀网格,并提取显著信息和视差,进而得到每个网格的平均显著值;根据相似性变化原理构建形状保持能量项,利用目标轨迹以及原始视频的视差变化构建时域保持能量项,并结合人眼辐辏调节原理构建视差舒适度调整能量项;结合各个网格的显著性,联合求解所有能量项得到优化后的网格顶点坐标,将其用于确定网格形变,从而生成指定宽高比的视频。结果实验结果表明,与基于细缝裁剪的立体视频重定向方法对比,本文方法在形状保持、时域保持及视差舒适度方面均具有更好的性能。另外,使用现有的客观质量评价方法对重定向结果进行评价,本文方法客观质量评价指标性能优于均匀缩放和细缝裁剪的视频重定向方法,时间复杂度较低,每帧的时间复杂度至少比细缝裁剪方法降低了98%。结论提出的时空联合...     

结合立体视觉舒适度的立体图像显著性检测

针对先前的立体图像显著性检测模型未充分考虑立体视觉舒适度和视差图分布特征对显著区域检测的影响,提出了一种结合立体视觉舒适度因子的显著性计算模型.该模型在彩色图像显著性提取中,首先利用SLIC算法对输入图像进行超像素分割,随后进行颜色相似区域合并后再进行二维图像显著性计算;在深度显著性计算中,首先对视差图进行预处理;然后基于区域对比度进行显著性计算;最后,结合立体视觉舒适度因子对二维显著图和深度显著图进行融合,得到立体图像显著图.在不同类型立体图像上的实验结果表明,该模型获得了85%的准确率和78%的召回率,优于现有常用的显著性检测模型,并与人眼立体视觉注意力机制保持良好的一致性.     

Visual experience for autostereoscopic 3D projection display

In order to investigate visual experience for watching the autostereoscopic three‐dimensional (3D) projection display, we conduct a subjective evaluation experiment by a questionnaire when viewing video clips. Factor analysis is adopted to classify the evaluation items for the perpetual constructs of visual experience. Then a mixed design with repeated measurement analysis of variance with dimension and display duration as factors is carried out on the evaluation data to check the factorial effects and interactions for statistical significance. The results of factor analysis extract five factors including visual comfort, image quality, distortion, naturalness, and presence, which can be used as comprehensive indicators to evaluate the autostereoscopic 3D projection display. The results of analysis of variance indicate that image quality, which is used to assess two‐dimensional contents, is no longer applicable. It is necessary to give consideration to depth when evaluating 3D visual experience. Although 3D scenes enhance the overall subjective performance such as naturalness and presence, the health issues and stereoscopic distortion related to the introduction of depth cannot be ignored.     

用户多维感知的3D图像体验质量评价

目的 符合用户视觉特性的3维图像体验质量评价方法有助于准确、客观地体现用户观看3D图像或视频时的视觉感知体验,从而给优化3维内容提供一定的思路。现有的评价方法仅从图像失真、深度感知和视觉舒适度中的一个维度或两个维度出发对立体图像进行评价,评价结果的准确性有待进一步提升。为了更加全面和准确地评价3D图像的视觉感知体验,提出了一种用户多维感知的3D图像体验质量评价算法。方法 首先对左右图像的差异图像和融合图像提取自然场景统计参数表示失真特征;然后对深度图像提取敏感区域,对敏感区域绘制失真前后深度变换直方图,统计深度变化情况以及利用尺度不变特征变换（SIFT）关键点匹配算法计算匹配点数目,两者共同表示深度感知特征;接下来对视觉显著区域提取视差均值、幅值表示舒适度特征;最后综合考虑图像失真、深度感知和视觉舒适度3个维度特征,将3个维度特征归一化后联合成体验质量特征向量,采用支持向量回归（SVR）训练评价模型,并得到最终的体验质量得分。结果 在LIVE和Waterloo IVC数据库上的实验结果表明,所提出的方法与人们的主观感知的相关性达到了0.942和0.858。结论 该方法充分利用了立体图像的特性,评价结果优于比较的几种经典算法,所构建模型的评价结果与用户的主观体验有更好的一致性。     

Impact of depth of field simulation on visual fatigue: Who are impacted? and how?

While stereoscopic content can be compelling, it is not always comfortable for users to interact with on a regular basis. This is because the stereoscopic content on displays viewed at a short distance has been associated with different symptoms such as eye-strain, visual discomfort, and even nausea. Many of these symptoms have been attributed to cue conflict, for example between vergence and accommodation. To resolve those conflicts, volumetric and other displays have been proposed to improve the user's experience. However, these displays are expensive, unduly restrict viewing position, or provide poor image quality. As a result, commercial solutions are not readily available. We hypothesized that some of the discomfort and fatigue symptoms exhibited from viewing in stereoscopic displays may result from a mismatch between stereopsis and blur, rather than between sensed accommodation and vergence. To find factors that may support or disprove this claim, we built a real-time gaze-contingent system that simulates depth of field (DOF) that is associated with accommodation at the virtual depth of the point of regard (POR). Subsequently, a series of experiments evaluated the impact of DOF on people of different age groups (younger versus older adults). The difference between short duration discomfort and fatigue due to prolonged viewing was also examined. Results indicated that age may be a determining factor for a user's experience of DOF. There was also a major difference in a user's perception of viewing comfort during short-term exposure and prolonged viewing. Primarily, people did not find that the presence of DOF enhanced short-term viewing comfort, while DOF alleviated some symptoms of visual fatigue but not all.     

Luminance effects influencing perception of 3‐D TV imagery

Abstract— This study investigates whether screen luminance or ambient illumination has a significant effect on the perception of 3‐D TV imagery for static images and dynamic films. Two types of stimuli were shown on a multi‐view stereoscopic display: the static image, which included computer‐generated and photographic images, and dynamic film, which contained real‐life and animation images. In each treatment with a different level of screen luminance, subjects completed psychophysical and physiological measurements and subjective comfort evaluations. The results showed that when subjects viewed 3‐D static images, the ambient illumination affected psychophysical visual fatigue and screen luminance had a significant effect on subjective comfort evaluation and visual discrimination performance. However, when subjects viewed 3‐D dynamic films, screen luminance was the major factor causing psychophysical visual fatigue, and ambient illumination significantly affected subjective comfort evaluation. The outcomes contribute to knowledge concerning the suitable viewing conditions for the 3‐D viewing experience. Future work will explore the intolerance threshold of the lowest display luminance or the effect of decomposition of the screen on other physiological measurements.     

Visual comfort in autostereoscopic display

Autostereoscopic displays are likely to become widely used products in the future. However, certain physiological factors, especially visual comfort, limit their development. In this study, four observational parameters – ambient illuminance, image content, scaling ratio, and horizontal distance between major and minor objects – were evaluated to determine the degree of visual comfort offered by 3D computer‐generated images on an autostereoscopic display. Visual comfort score with the range of 0–1 is designed to represent the degree of visual comfort for the 3D images with different manipulations of ambient illuminance, image content, scaling ratio, and horizontal distance between major and minor objects in this study. Subjects were asked to indicate images that produced discomfort. The proportion of images for each condition where participants indicated that viewing the image was comfortable was computed. Images receiving a proportion of 0.5 or greater were classified as acceptable. The disparity ranges over which acceptable images were attained for each participant and for each condition were analyzed with analysis of variance. The analytical results indicate that ambient illuminance and image content have a significant effect on the acceptable disparity range, while scaling ratio and horizontal distance between major and minor objects did not.     

Relationship between parallax and spatial resolution on visual comfort of an autostereoscopic display

A primary reason for visual discomfort caused by viewing 3D contents in autostereoscopic displays is the conflict between accommodation and convergence. Parallax directly affects the degree of conflict. In this paper, three factors are investigated—parallax magnitude, changing parallax, and spatial resolution—which are supposed to exert substantial influence upon visual comfort. The aforementioned three factors are integrated into stereoscopic sequences generated by software. With the data on the subjective evaluation of comfort from the viewers, statistical method is used to obtain both the main effects and interactions among the three factors. According to the experimental results, changing parallax might play a more crucial role in visual comfort than parallax magnitude. Spatial resolution might alleviate the negative effects in which changing parallax brings about. The experiment indicates that parallax for autostereoscopic displays should be within 50′ in consideration of comfort.     

融合双目多维感知特征的立体视频显著性检测

目的 立体视频能提供身临其境的逼真感而越来越受到人们的喜爱,而视觉显著性检测可以自动预测、定位和挖掘重要视觉信息,可以帮助机器对海量多媒体信息进行有效筛选。为了提高立体视频中的显著区域检测性能,提出了一种融合双目多维感知特性的立体视频显著性检测模型。方法 从立体视频的空域、深度以及时域3个不同维度出发进行显著性计算。首先,基于图像的空间特征利用贝叶斯模型计算2D图像显著图;接着,根据双目感知特征获取立体视频图像的深度显著图;然后,利用Lucas-Kanade光流法计算帧间局部区域的运动特征,获取时域显著图;最后,将3种不同维度的显著图采用一种基于全局-区域差异度大小的融合方法进行相互融合,获得最终的立体视频显著区域分布模型。结果 在不同类型的立体视频序列中的实验结果表明,本文模型获得了80%的准确率和72%的召回率,且保持了相对较低的计算复杂度,优于现有的显著性检测模型。结论 本文的显著性检测模型能有效地获取立体视频中的显著区域,可应用于立体视频/图像编码、立体视频/图像质量评价等领域。     

利用多核增强学习的立体图像舒适度评价模型

目的 传统的立体视觉舒适度评价模型,在学习阶段一般采用回归算法,且需要大量的包含主观测试数据的训练样本,针对这个问题,提出一种利用多核增强学习分类算法的立体图像舒适度评价模型。方法 首先,考虑人们在实际观测图像时,对于先后观测到的不同图像进行相互比较的情况,将评价模型看成是偏好分类器,构造包含偏好标签的偏好立体图像对（PSIP）,构成PSIP训练集;其次,提取多个视差统计特征和神经学模型响应特征;然后,利用基于AdaBoost的多核学习算法来建立偏好标签与特征之间的关系模型,并分析偏好分类概率（即相对舒适度概率）与最终的视觉舒适度之间的映射关系。结果 在独立立体图像库上,与现有代表性回归算法相比较,本文算法的Pearson线性相关系数（PLCC）在0.84以上,Spearman等级相关系数（SRCC）在0.80以上,均优于其他模型的各评价指标;而在跨库测试中,本文算法的PLCC、SRCC指标均优于传统的支持向量回归算法。结论 相比于传统的回归算法,本文算法具有更好的评价性能,能够更为准确地预测立体图像视觉舒适度。     

Measuring 3D crosstalk uniformity and its influence on contrast ratio

An array detector was employed to characterize the crosstalk and contrast ratio uniformity of three‐dimensional (3D) displays. The measurement method is described and demonstrated on a stereoscopic display with passive glasses. The resulting high resolution spatial uniformity maps enable a comprehensive statistical characterization of the display and provide a useful visual assessment tool. The statistical uniformity data are used to evaluate the crosstalk dependence on viewing conditions (such as viewing distance), and show the degradation in display performance that could not be captured with a discrete spot measurement. The measurement method was also employed to examine the influence of crosstalk on contrast ratio. It is shown that for some 3D displays, the crosstalk uniformity can dominate the perceived contrast. A metric is proposed that defines the maximal crosstalk contribution to the perceived contrast for 3D images with small features. The summary parameters extracted from the uniformity maps can be utilized to define the boundary viewing conditions needed to maintain minimum image quality.     

基于网格形变的立体图像内容重组

目的 近年来,随着数字摄影技术的飞速发展,图像增强技术越来越受到重视。图像构图作为图像增强中影响美学的重要因素,一直都是研究的热点。为此,从立体图像布局调整出发,提出一种基于Delaunay网格形变的立体图像内容重组方法。方法 首先将待重组的一对立体图像记为源图像,将用于重组规则确定的一幅图像记为参考图像;然后对源图像需要调整的目标、特征线和其他区域进行取点操作,建立Delaunay网格。将源图像的左图与参考图像进行模板匹配操作,得到源图像与参考图像在结构布局上的对应关系;最后利用网格形变的特性,移动和缩放目标对象,并对立体图像的深度进行自适应调整。结果 针对目标对象的移动、缩放和特征线调整几方面进行优化。当只涉及目标对象的移动或特征线调整时,立体图像视差保持不变;当目标对象缩放时,立体图像中目标对象的视差按照缩放比例变化而背景视差保持不变。实验结果表明,重组后的立体图像构图与参考图像一致且深度能自适应调整。与最新方法比较,本文方法在目标对象分割精度和图像语义保持方面具有优势。结论 根据网格形变相关理论,构建图像质量、布局匹配和视差适应3种能量项,实现了立体图像的内容重组。与现有需要提取和粘贴目标对象的重组方法不同,本文方法对目标对象的分割精度要求不高,不需要图像修复和混合技术,重组后的立体图像没有伪影和语义错误出现。用户可以通过参考图像来引导立体图像的布局调整,达到期望的图像增强效果。     

实时立体视觉系统中的深度映射

目的 为减少立体图像中由于水平视差过大引起的视觉疲劳。针对实时渲染的立体视觉系统,给出了一种非均匀深度压缩方法。方法 该方法在单一相机空间内,通过不同的投影变换矩阵生成双眼图像,水平视差由投影变换来控制。为减少深度压缩造成的模型变形而带来的瑕疵,将不同深度区域内物体施以不同的压缩比例;将相机轴距表示为深度的连续函数,通过相机轴距推导出在单一相机空间内获取双眼图像的坐标变换,将深度压缩转换为模型的坐标变换,从而保证压缩比例的连续变化。结果 实验结果表明,该方法能有效提高立体图像的质量。结论 该方法简单、高效,可应用于游戏、虚拟现实等实时立体视觉系统。     

3维立体动画制作的视觉效果评价

目的 视觉效果评价是3维立体动画制作过程中不可忽略的一环。评价过程主要依靠专业人员的行业经验,受人员知识水平、测试环境等因素的影响。针对该问题,提出了一个主客观结合的评价模型。方法 首先建立了一个面向前期制作的三维动画场景数据集用以训练和测试,针对视觉效果评价的两个重要指标:立体感和视觉舒适度,进行主观实验得到相应的分数;提取全局视觉舒适度特征和感兴趣区域立体感特征,使用支持向量回归(SVR)方法,经过训练和测试得到舒适度评价模型和立体感评价模型。结果 通过将性能验证实验得到验证场景的主观分数与评价模型给出的结果进行比对,结果表明,运用评价模型得到的预测分数与观众主观分数基本一致,该模型可以对影响视觉效果的视觉舒适度和立体感予以5级量化评分。结论 本文所提出的视觉舒适度和立体感评价方法,能建立影响视觉效果的特征与主观评分间的关系,用得到的模型预测分数给制作人员一个及时直观的调节依据标准。