基于视差重映射的立体图像视觉舒适度提升

目的 针对人眼观看立体图像内容可能存在的视觉不舒适性,基于视差对立体图像视觉舒适度的影响,提出了一种结合全局线性和局部非线性视差重映射的立体图像视觉舒适度提升方法。方法 首先,考虑双目融合限制和视觉注意机制,分别结合空间频率和立体显著性因素提取立体图像的全局和局部视差统计特征,并利用支持向量回归构建客观的视觉舒适度预测模型作为控制视差重映射程度的约束;然后,通过构建的预测模型对输入的立体图像的视觉舒适性进行分析,就欠舒适的立体图像设计了一个两阶段的视差重映射策略,分别是视差范围的全局线性重映射和针对提取的潜在欠舒适区域内视差的局部非线性重映射;最后,根据重映射后的视差图绘制得到舒适度提升后的立体图像。结果 在IVY Lab立体图像舒适度测试库上的实验结果表明,相较于相关有代表性的视觉舒适度提升方法对于欠舒适立体图像的处理结果,所提出方法在保持整体场景立体感的同时,能更有效地提升立体图像的视觉舒适度。结论 所提出方法能够根据由不同的立体图像特征构建的视觉舒适度预测模型来自动实施全局线性和局部非线性视差重映射过程,达到既改善立体图像视觉舒适度、又尽量减少视差改变所导致的立体感削弱的目的,从而提升立体图像的整体3维体验。     

True 3‐D scanned voxel displays using single or multiple light sources

Abstract— Conventional stereoscopic displays require viewers to unnaturally keep eye accommodation fixed at one focal distance while they dynamically change vergence to view objects at different distances. This forced decoupling of reflexively linked processes fatigues eyes, causes discomfort, compromises image quality, and may lead to pathologies in developing visual systems. Volumetric displays can overcome this conflict, but only for small objects placed within a limited range of viewing distances and accommodation levels, and cannot render occlusion cues correctly. Our multi‐planar True 3‐D displays generate accommodation cues that match vergence and stereoscopic retinal disparity demands and can display images and objects at viewing distances throughout the full range of human accommodation (from 6.25 cm to infinity), better mimicking natural vision and minimizing eye fatigue.     

Visual discomfort of stereoscopic 3D videos: Influence of 3D motion

Visual discomfort is one of the most frequent complaints of the viewers while watching 3D images and videos. Large disparity and large amount of motion are two main causes of visual discomfort. To quantify this influence, three objectives are set in this paper. The first one is the comparative analysis on the influence of different types of motion, i.e., static stereoscopic image, planar motion and in-depth motion, on visual discomfort. The second one is the investigation on the influence factors for each motion type, for example, the disparity offset, the disparity amplitude and velocity. The third one is to propose an objective model for visual discomfort. Thirty-six synthetic stereoscopic video stimuli with different types of motion are used in this study. In the subjective test, an efficient paired comparison method called Adaptive Square Design (ASD) was used to reduce the number of comparisons for each observer and keep the results reliable. The experimental results showed that motion does not always induce more visual discomfort than static conditions. The in-depth motion generally induces more visual discomfort than the planar motion. The relative disparity between the foreground and the background, and the motion velocity are identified as main factors for visual discomfort. According to the subjective results, an objective model for comparing visual discomfort induced by different types of motion is proposed which shows high correlation with the subjective perception.     

Impact of depth of field simulation on visual fatigue: Who are impacted? and how?

While stereoscopic content can be compelling, it is not always comfortable for users to interact with on a regular basis. This is because the stereoscopic content on displays viewed at a short distance has been associated with different symptoms such as eye-strain, visual discomfort, and even nausea. Many of these symptoms have been attributed to cue conflict, for example between vergence and accommodation. To resolve those conflicts, volumetric and other displays have been proposed to improve the user's experience. However, these displays are expensive, unduly restrict viewing position, or provide poor image quality. As a result, commercial solutions are not readily available. We hypothesized that some of the discomfort and fatigue symptoms exhibited from viewing in stereoscopic displays may result from a mismatch between stereopsis and blur, rather than between sensed accommodation and vergence. To find factors that may support or disprove this claim, we built a real-time gaze-contingent system that simulates depth of field (DOF) that is associated with accommodation at the virtual depth of the point of regard (POR). Subsequently, a series of experiments evaluated the impact of DOF on people of different age groups (younger versus older adults). The difference between short duration discomfort and fatigue due to prolonged viewing was also examined. Results indicated that age may be a determining factor for a user's experience of DOF. There was also a major difference in a user's perception of viewing comfort during short-term exposure and prolonged viewing. Primarily, people did not find that the presence of DOF enhanced short-term viewing comfort, while DOF alleviated some symptoms of visual fatigue but not all.     

Simultaneous measurement of lens accommodation and convergence in natural and artificial 3D vision

Recent advances in 3D technology have been accompanied by increasing complaints of visual fatigue. The usual explanation for such fatigue is that accommodation and convergence are mismatched during stereoscopic vision. The aim of this study was to measure fixation distances between lens accommodation and convergence in young subjects while they viewed real objects and 3D video clips. Measurements were made using an original instrument. The 3D video clips were presented to subjects using a liquid crystal shutter glass system. The results showed that when viewing real objects, the diopter values of subjects' accommodation and convergence were similar and changed periodically. This measurement method was thus considered to be appropriate for the measurement of stereoscopic vision. We also investigated lens accommodation and convergence when subjects viewed 3D video clips. Both accommodation and convergence were found to move along with the virtual position of 3D video clips. Therefore, there was little discrepancy between accommodation and convergence during the viewing of 3D images.     

Dual layered display that presents auto‐stereoscopic 3D images to multiple viewers in arbitrary positions

Dual layered display or also called tensor display that consists of two panels in a stack can present full‐parallax 3D images with high resolution and continuous motion parallax by reconstructing corresponding light ray field within a viewing angle. The depth range where the 3D images can be displayed with reasonable resolution, however, is limited around the panel stack. In this paper, we propose a dual layered display that can present stereoscopic images to multiple viewers located at arbitrary positions in observer space with high resolution and large depth range. Combined with the viewer tracking system, the proposed method provides a practical way to realize high‐resolution large‐depth auto‐stereoscopic 3D display for multiple observers without restriction on the observer position and the head orientation.     

Using Bold-fMRI to detect cortical areas and visual fatigue related to stereoscopic vision

PurposeThe location of cortical areas and visual fatigue related to stereoscopic vision were explored by two types of fMRI designs, namely block stimulation and event-related stimulation. The stimulations consist of 2D/3D images and different depths of the stereoscopic 3D images.Method20 normal subjects were randomly divided into the block group and the event-related group. Blood oxygenation level-dependent functional magnetic resonance imaging (Bold-fMRI) was performed in two groups. Functional data was preprocessed and statistically analyzed by SPM8. The result was reported by REST.ResultIn the block stimulation group, compared to 2D image stimulation, 3D image stimulation results in more activated brain areas, including frontal lobes, occipital lobes and limbic lobes, especially in the frontal eye field (Brodmann Area 8, BA8) and middle occipital gyrus (BA18/19). In the event-related group, compared to 2D images, viewing 3D images causes significant activations in temporal lobe, mainly represented in BA19/13/31/37. Additionally, 3D image stimulation with the focus set at front depth can lead to the activation of more brain areas compared to the back depth, including inferior parietal lobule and posterior central gyrus.ConclusionThe formation of the stereoscopic vision requires the collaboration of more brain areas, so that viewing stereoscopic videos for a long period may result in visual fatigue; meanwhile, the front depth of field can contribute to more activated brain areas than the back depth of field. As a parameter of stereoscopic images, it is valid to state that the depth of field may affect visual fatigue.     

时空联合视差优化的立体视频重定向

目的智能适配显示的图像/视频重定向技术近年受到广泛关注。与图像重定向以及2D视频重定向相比,3D视频重定向需要同时考虑视差保持和时域保持。现有的3D视频重定向方法虽然考虑了视差保持却忽略了对视差舒适度的调整,针对因视差过大和视差突变造成视觉不舒适度这一问题,提出了一种基于时空联合视差优化的立体视频重定向方法,将视频视差范围控制在舒适区间。方法在原始视频上建立均匀网格,并提取显著信息和视差,进而得到每个网格的平均显著值;根据相似性变化原理构建形状保持能量项,利用目标轨迹以及原始视频的视差变化构建时域保持能量项,并结合人眼辐辏调节原理构建视差舒适度调整能量项;结合各个网格的显著性,联合求解所有能量项得到优化后的网格顶点坐标,将其用于确定网格形变,从而生成指定宽高比的视频。结果实验结果表明,与基于细缝裁剪的立体视频重定向方法对比,本文方法在形状保持、时域保持及视差舒适度方面均具有更好的性能。另外,使用现有的客观质量评价方法对重定向结果进行评价,本文方法客观质量评价指标性能优于均匀缩放和细缝裁剪的视频重定向方法,时间复杂度较低,每帧的时间复杂度至少比细缝裁剪方法降低了98%。结论提出的时空联合...     

Effects of stereoscopic movies: The positions of stereoscopic objects and the viewing conditions

The present study investigates the effects of relative positions of stereoscopic objects (PSO) (‘Far’ (from viewers) vs. ‘Near’ (to viewers) vs. ‘Both’) and seat location on viewers’ psychological responses. People who watched a movie with ‘Both’ conditions reported more arousal and satisfaction compared to people who watched a movie with either the ‘Far’ or ‘Near’ condition. More importantly, interaction effects were reported such as more dizziness (a) with the ‘Near’ condition if sitting on the left or right side in the cinema and (b) with the ‘Far’ condition if sitting in the middle of the cinema. Additionally, people who wore glasses felt more eye fatigue than those who did not. Secondly, people felt less presence, sensation of depth, and arousal if they had prior experience viewing stereoscopic movies. The results indicate that viewing experience with different PSOs and/or seat locations can influence psychological response.     

用户多维感知的3D图像体验质量评价

目的 符合用户视觉特性的3维图像体验质量评价方法有助于准确、客观地体现用户观看3D图像或视频时的视觉感知体验,从而给优化3维内容提供一定的思路。现有的评价方法仅从图像失真、深度感知和视觉舒适度中的一个维度或两个维度出发对立体图像进行评价,评价结果的准确性有待进一步提升。为了更加全面和准确地评价3D图像的视觉感知体验,提出了一种用户多维感知的3D图像体验质量评价算法。方法 首先对左右图像的差异图像和融合图像提取自然场景统计参数表示失真特征;然后对深度图像提取敏感区域,对敏感区域绘制失真前后深度变换直方图,统计深度变化情况以及利用尺度不变特征变换（SIFT）关键点匹配算法计算匹配点数目,两者共同表示深度感知特征;接下来对视觉显著区域提取视差均值、幅值表示舒适度特征;最后综合考虑图像失真、深度感知和视觉舒适度3个维度特征,将3个维度特征归一化后联合成体验质量特征向量,采用支持向量回归（SVR）训练评价模型,并得到最终的体验质量得分。结果 在LIVE和Waterloo IVC数据库上的实验结果表明,所提出的方法与人们的主观感知的相关性达到了0.942和0.858。结论 该方法充分利用了立体图像的特性,评价结果优于比较的几种经典算法,所构建模型的评价结果与用户的主观体验有更好的一致性。     

Luminance effects influencing perception of 3‐D TV imagery

Abstract— This study investigates whether screen luminance or ambient illumination has a significant effect on the perception of 3‐D TV imagery for static images and dynamic films. Two types of stimuli were shown on a multi‐view stereoscopic display: the static image, which included computer‐generated and photographic images, and dynamic film, which contained real‐life and animation images. In each treatment with a different level of screen luminance, subjects completed psychophysical and physiological measurements and subjective comfort evaluations. The results showed that when subjects viewed 3‐D static images, the ambient illumination affected psychophysical visual fatigue and screen luminance had a significant effect on subjective comfort evaluation and visual discrimination performance. However, when subjects viewed 3‐D dynamic films, screen luminance was the major factor causing psychophysical visual fatigue, and ambient illumination significantly affected subjective comfort evaluation. The outcomes contribute to knowledge concerning the suitable viewing conditions for the 3‐D viewing experience. Future work will explore the intolerance threshold of the lowest display luminance or the effect of decomposition of the screen on other physiological measurements.     

The effects of motion and stereopsis on three-dimensional visualization

Previous studies have demonstrated that motion cues combined with stereoscopic viewing can enhance the perception of three-dimensional objects displayed on a two-dimensional computer screen. Using a variant of the mental rotation paradigm, subjects view pairs of object images presented on a computer terminal and judge whether the objects are the same or different. The effects of four variables on the accuracy and speed of decision performances are assessed: stereo vs. mono viewing, controlled vs. uncontrolled object motion, cube vs. sphere construction and wire frame vs. solid surface characteristic. Viewing the objects as three-dimensional images results in more accurate and faster decision performances. Furthermore, accuracy improves although response time increases when subjects control the object motion. Subjects are equally accurate comparing wire frame and solid images, although they take longer comparing wire frame images. The cube-based or sphere-based object construction has no impact on decision accuracy nor response time.     

Subjective measures of presence and discomfort in viewers of color‐separation‐based stereoscopic cinema

Abstract— The experience of various people related to a real 3‐D cinema visit has been studied, and the results will be discussed. This study has two distinct parts, which includes the comfort of the viewers and their sense of presence. Eighty‐four viewers filled out questionnaires about comfort and visual strain. Forty‐one subjects described their presence experience during the movie presentation. A majority of the people felt comfortable after the movie viewing; they experienced only mild‐eyestrain‐related symptoms. People evaluated the movie world as highly realistic, but they did not feel that they were actors in the virtual‐movie world. Most of the participants would recommend 3‐D cinema to friends because it was a very entertaining experience.     

User experience while viewing stereoscopic 3D television

3D display technologies have been linked to visual discomfort and fatigue. In a lab-based study with a between-subjects design, 433 viewers aged from 4 to 82 years watched the same movie in either 2D or stereo 3D (S3D), and subjectively reported on a range of aspects of their viewing experience. Our results suggest that a minority of viewers, around 14%, experience adverse effects due to viewing S3D, mainly headache and eyestrain. A control experiment where participants viewed 2D content through 3D glasses suggests that around 8% may report adverse effects which are not due directly to viewing S3D, but instead are due to the glasses or to negative preconceptions about S3D (the ‘nocebo effect’). Women were slightly more likely than men to report adverse effects with S3D. We could not detect any link between pre-existing eye conditions or low stereoacuity and the likelihood of experiencing adverse effects with S3D.     

Eye movements and visual discomfort when viewing stereoscopic 3D content

The visual brain fuses the left and right images projected onto the two eyes from a stereoscopic 3D (S3D) display, perceives parallax, and rebuilds a sense of depth. In this process, the eyes adjust vergence and accommodation to adapt to the depths and parallax of the points they gazed at. Conflicts between accommodation and vergence when viewing S3D content potentially lead to visual discomfort. A variety of approaches have been taken towards understanding the perceptual bases of discomfort felt when viewing S3D, including extreme disparities or disparity gradients, negative disparities, dichoptic presentations, and so on. However less effort has been applied towards understanding the role of eye movements as they relate to visual discomfort when viewing S3D. To study eye movements in the context of S3D viewing discomfort, a Shifted-S3D-Image-Database (SSID) is constructed using 11 original nature scene S3D images and their 6 shifted versions. We conducted eye-tracking experiments on humans viewing S3D images in SSID while simultaneously collecting their judgments of experienced visual discomfort. From the collected eye-tracking data, regions of interest (ROIs) were extracted by kernel density estimation using the fixation data, and an empirical formula fitted between the disparities of salient objects marked by the ROIs and the mean opinion scores (MOS). Finally, eye-tracking data was used to analyze the eye movement characteristics related to S3D image quality. Fifteen eye movement features were extracted, and a visual discomfort predication model learned using a support vector regressor (SVR). By analyzing the correlations between features and MOS, we conclude that angular disparity features have a strong correlation with human judgments of discomfort.     

Depth Mapping for Stereoscopic Videos

Stereoscopic videos have become very popular in recent years. Most of these videos are developed primarily for viewing on large screens located at some distance away from the viewer. If we watch these videos on a small screen located near to us, the depth range of the videos will be seriously reduced, which can significantly degrade the 3D effects of these videos. To address this problem, we propose a linear depth mapping method to adjust the depth range of a stereoscopic video according to the viewing configuration, including pixel density and distance to the screen. Our method tries to minimize the distortion of stereoscopic image contents after depth mapping, by preserving the relationship of neighboring features and preventing line and plane bending. It also considers the depth and motion coherences. While depth coherence ensures smooth changes of the depth field across frames, motion coherence ensures smooth content changes across frames. Our experimental results show that the proposed method can improve the stereoscopic effects while maintaining the quality of the output videos.     

Human fusion range for polarized 3D display

If discrepancy between accommodation and convergence caused by a stereoscopic display exceeds fusion range of human eyes, viewers will see ghosting image, which leads to the loss of correct depth information and even causes severe visual fatigue. In this paper, an experiment aiming to investigate the binocular fusion range is conducted for a polarized 3D display. Two experimental trials are arranged to examine two aspects of fusion range including outward depth and inward depth. 3D modeling software is used to generate the test stereoscopic image pairs, which vary in depth by adjusting the separation between the virtual cameras. Angular parallax corresponding to the limit of fusion range is obtained by determining critical point of ghosting images. The experimental results show deviation between theoretical fusion range calculated by formula and experimental one.?0.223° to 0.275° represent critical fusion range for the polarized 3D display to avoid ghosting images.     

A learning-based visual saliency prediction model for stereoscopic 3D video (LBVS-3D)

Saliency prediction models provide a probabilistic map of relative likelihood of an image or video region to attract the attention of the human visual system. Over the past decade, many computational saliency prediction models have been proposed for 2D images and videos. Considering that the human visual system has evolved in a natural 3D environment, it is only natural to want to design visual attention models for 3D content. Existing monocular saliency models are not able to accurately predict the attentive regions when applied to 3D image/video content, as they do not incorporate depth information. This paper explores stereoscopic video saliency prediction by exploiting both low-level attributes such as brightness, color, texture, orientation, motion, and depth, as well as high-level cues such as face, person, vehicle, animal, text, and horizon. Our model starts with a rough segmentation and quantifies several intuitive observations such as the effects of visual discomfort level, depth abruptness, motion acceleration, elements of surprise, size and compactness of the salient regions, and emphasizing only a few salient objects in a scene. A new fovea-based model of spatial distance between the image regions is adopted for considering local and global feature calculations. To efficiently fuse the conspicuity maps generated by our method to one single saliency map that is highly correlated with the eye-fixation data, a random forest based algorithm is utilized. The performance of the proposed saliency model is evaluated against the results of an eye-tracking experiment, which involved 24 subjects and an in-house database of 61 captured stereoscopic videos. Our stereo video database as well as the eye-tracking data are publicly available along with this paper. Experiment results show that the proposed saliency prediction method achieves competitive performance compared to the state-of-the-art approaches.

     

HoloTube: a low-cost portable 360-degree interactive autostereoscopic display

This paper proposes a novel, low cost, and portable 360-degree cylindrical interactive autostereoscopic 3D display system. The proposed system consists of three parts: the optical architecture (for back-projecting image correctly on the cylindrical screen), the projection image transformation workflow (for image rectifying and generating multi-view images), and the 360-degree motion detection module (for identifying viewers’ locations and providing the corresponding views). Based on the proposed design, only one commercial micro projector is employed for the proposed cylindrical screen. The proposed display offers great depth perception (stereoacuity) with a special designed thick barrier sheet attached to the screen. The viewers are not required to wear special glasses and within appropriate range (< 5m) the viewers can view the screen at any distance and angle. The user study verified that the proposed display offers satisfactory depth perception (binocular parallax, shading distribution, and linear perspective) for various viewing distances and angles without noticeable discomfort. The production cost of the current prototype is about USD$ 300. With mass production, the unit cost is expected to decline to within USD$60. The proposed display system has the advantages of ease of use, low production cost, high portability and mobility. The proposed system is suitable for application such as museum virtual exhibition, remote meeting, multi-user online game, etc. We believe that the proposed system is very promising for the market of low-cost portable 360-degree interactive autosereoscopic displays.     

Comparison of changes in visual fatigue and ocular surface after 3D and 2D viewing with augmented reality glasses

The extensive use of electronic devices commonly results in visual discomfort, particularly in stereoscopic display. Augmented Reality (AR) is a mixed technology that superimposes three-dimensional (3D) digital data onto an image of reality. Even though the visual health evaluation of 3D contents has been widely concerned, the difference between 3D and two-dimensional (2D) viewing using AR display was rarely studied. We aimed to compare the changes in visual fatigue and ocular surface parameters after 3D and 2D viewing with optical waveguide AR glasses for one hour (1hr). In the experiment, thirty young healthy participants watched the same film in 2D and 3D mode separately using AR glasses with one week interval. Subjective visual fatigue questionnaire and ocular examinations were evaluated at 3 time points: pre-watching, post-2D and post-3D visual task for 1hr. Lipid layer thickness (LLT) and total blink (TB) were evaluated with the LipiView interferometer. Tear meniscus height (TMH) and noninvasive tear breakup time (NIBUT) were measured using Keratograph 5M. The effect of viewing 2D and 3D videos for 1hr with AR glasses suggested slight visual discomfort except the symptom of pain and dullness. The lipid layer thickness and total blink showed increasing trends in both post-visual task but more obvious after 2D viewing. Blinking patterns and tear film stability are meaningful for video display terminal users in visual health education. Ocular parameters are needed to be considered in the design and usage of future AR applications.