Content-aware disparity adjustment for different stereo displays

In this paper, we present an effective disparity mapping method for binocular stereoscopic image. It is inspired by the observation that its displayed depth would change, when a stereoscopic image is displayed on different size screens. The phenomenon may bring an uncomfortable experience for viewers. To make a comfortable stereoscopic image for viewers, moreover to adapt a stereoscopic image to a target display screen, we propose a content-aware disparity adjustment method. Firstly, the disparity mapping is established to control and retarget the depth of a stereoscopic scene. Then, the relationship between the disparity editing and image content editing is established to guide the proposed warping model. At last, to implement the disparity mapping operator, we propose a content-aware stereoscopic mesh warping model, which can simultaneously avoid the salient region distortion and adjust disparity to a target range by establishing the relationship. Experimental results show that the proposed method can effectively adjust disparity of stereoscopic image, which not only avoids the salient region distortion and adjusts disparity to a target range.     

微距摄影的多聚焦图像拍摄和融合

目的 对于微距摄影来说,由于微距镜头的景深有限,往往很难通过单幅照片获得拍摄对象全幅清晰的图像.因此要想获取全幅清晰的照片,就需要拍摄多幅具有不同焦点的微距照片,并对其进行融合.方法 传统的微距照片融合方法一般都假定需要融合的图像是已经配准好的,也并没有考虑微距图像的自动采集.因此提出了一种用于微距摄影的多聚焦图像采集和融合系统,该系统由3个部分组成.第1部分是一种微距图像拍摄装置,该硬件能够以高精度的方式拍摄物体在不同焦距下的微距照片.第2部分是一个基于不变特征的图像配准组件,它可以对在多个焦点下拍摄的微距图像进行自动配准和对齐.第3部分是一个基于图像金字塔的多聚焦图像融合组件,这个组件能够对已经对齐的微距照片进行融合,使得合成的图像具有更大的景深.该组件对基于图像金字塔的融合方法进行了扩展,提出了一种基于滤波的权重计算策略.通过将该权重计算与图像金字塔相结合,得到了一种基于多分辨率的多聚焦图像融合方法.结果 论文使用设计的拍摄装置采集了多组实验数据,用以验证系统硬件设计和软件设计的正确性,并使用主观和客观的方法对提出的系统进行评价.从主观评价来看,系统合成的微距图像不仅具有足够的景深,而且在高分辨率下也能够清晰地呈现物体微小的细节.从客观评价来看,通过将系统合成的微距图像与其他方法合成的微距图像进行量化比较,在标准差、信息熵和平均梯度3种评价标准中都是最优的.结论 实验结果表明,该系统是灵活和高效的,不仅能够对多幅具有不同焦点的微距图像进行自动采集、配准和融合,并且在图像融合的质量方面也能和其他方法相媲美.     

树木结构层次细化的立体树木图像构建

目的 2维转3维技术可以将现有的丰富2维图像资源快速有效地转为立体图像,但是现有方法只能对树木的整体进行深度估计,所生成的图像无法表现出树木的立体结构。为此,提出一种树木结构层次细化的立体树木图像构建方法。方法 首先利用Lab颜色模型下的像素色差区别将2维树木图像的树干区域和树冠区域分割开来,并对树冠区域进行再分割;然后,在深度梯度假设思想基础上建立多种类型的深度模板,结合深度模板和树冠的区域信息为典型树木对象构建初始深度图,并通过基础深度梯度图组合的方式为非典型树木进行个性化深度构建;最后,根据应用场景对树木深度信息进行自适应调整与优化,将树木图像合成到背景图像中,并构建立体图像。结果 对5组不同的树木图像及背景图像进行了立体树木图像的构建与合成。结果表明,不同形态的树木图像都能生成具有层次感的深度图并自适应地合成到立体背景图像中,构建树木图像深度图的时间与原始树木图像的尺寸成正比,而构建立体树木图像并合成到背景中所需时间在24 s之间。对立体图像质量的主观评价测试中,这些图像的评分均达到良好级别以上,部分立体图像达到了优秀级别。结论 该方法充分利用了树木的形态结构特征,能同时适用于典型和非典型树木,所构建的立体树木图像质量较高,具有丰富的层次感,并具有舒适的立体观看效果。     

基于网格形变的立体图像内容重组

目的 近年来,随着数字摄影技术的飞速发展,图像增强技术越来越受到重视。图像构图作为图像增强中影响美学的重要因素,一直都是研究的热点。为此,从立体图像布局调整出发,提出一种基于Delaunay网格形变的立体图像内容重组方法。方法 首先将待重组的一对立体图像记为源图像,将用于重组规则确定的一幅图像记为参考图像;然后对源图像需要调整的目标、特征线和其他区域进行取点操作,建立Delaunay网格。将源图像的左图与参考图像进行模板匹配操作,得到源图像与参考图像在结构布局上的对应关系;最后利用网格形变的特性,移动和缩放目标对象,并对立体图像的深度进行自适应调整。结果 针对目标对象的移动、缩放和特征线调整几方面进行优化。当只涉及目标对象的移动或特征线调整时,立体图像视差保持不变;当目标对象缩放时,立体图像中目标对象的视差按照缩放比例变化而背景视差保持不变。实验结果表明,重组后的立体图像构图与参考图像一致且深度能自适应调整。与最新方法比较,本文方法在目标对象分割精度和图像语义保持方面具有优势。结论 根据网格形变相关理论,构建图像质量、布局匹配和视差适应3种能量项,实现了立体图像的内容重组。与现有需要提取和粘贴目标对象的重组方法不同,本文方法对目标对象的分割精度要求不高,不需要图像修复和混合技术,重组后的立体图像没有伪影和语义错误出现。用户可以通过参考图像来引导立体图像的布局调整,达到期望的图像增强效果。     

Viewpoint image generation for head tracking 3D display using multi‐camera and approximate depth information

A simple and high image quality method for viewpoint image synthesis from multi‐camera images for a stereoscopic 3D display using head tracking is proposed. In this method, slices of images for depth layers are made using approximate depth information, the slices are linearly blended corresponding to the distance between the viewpoint and cameras at each layer, and the layers are overlaid from the perspective of viewpoint. Because the linear blending automatically compensates for depth error because of the visual effects of depth‐fused 3D (DFD), the resulting image is natural to observer's perception. Smooth motion parallax of wide depth range objects induced by viewpoint movement for left‐and‐right and front‐and‐back directions is achieved using multi‐camera images and approximate depth information. Because the calculation algorithm is very simple, it is suitable for real time 3D display applications.     

Optimized Generation of Stereoscopic CGI Films by 3D Image Warping

The generation of a stereoscopic animation film requires doubling the rendering times and hence the cost. In this paper, we address this problem and propose an automatic system for generating a stereo pair from a given image and its depth map. Although several solutions exist in the literature, the high standards of image quality required in the context of a professional animation studio forced us to develop specially crafted algorithms that avoid artefacts caused by occlusions, anti‐aliasing filters, etc. This paper describes all the algorithms involved in our system and provides their GPU implementation. The proposed system has been tested with real‐life working scenarios. Our experiments show that the second view of the stereoscopic pair can be computed with as little as 15% of the effort of the original image while guaranteeing a similar quality.     

Image correction based on homography in stereoscopic projection display

Abstract— Stereoscopic and autostereoscopic projection‐display systems use projector arrays to present stereoscopic images, and each projector casts one parallax image of a stereoscopic scene. Because of the position shift of the projectors, the parallax images have geometric deformation, which influences the quality of the displayed stereoscopic images. In order to solve this problem, a method based on homography is proposed. The parallax images are pre‐transformed before they are projected, and then the stereoscopic images without geometric distortion can be obtained. An autostereoscopic projection‐display system is developed to present the images with and without calibration. Experimental results show that this method works effectively.     

基于视差重映射的立体图像视觉舒适度提升

目的 针对人眼观看立体图像内容可能存在的视觉不舒适性,基于视差对立体图像视觉舒适度的影响,提出了一种结合全局线性和局部非线性视差重映射的立体图像视觉舒适度提升方法。方法 首先,考虑双目融合限制和视觉注意机制,分别结合空间频率和立体显著性因素提取立体图像的全局和局部视差统计特征,并利用支持向量回归构建客观的视觉舒适度预测模型作为控制视差重映射程度的约束;然后,通过构建的预测模型对输入的立体图像的视觉舒适性进行分析,就欠舒适的立体图像设计了一个两阶段的视差重映射策略,分别是视差范围的全局线性重映射和针对提取的潜在欠舒适区域内视差的局部非线性重映射;最后,根据重映射后的视差图绘制得到舒适度提升后的立体图像。结果 在IVY Lab立体图像舒适度测试库上的实验结果表明,相较于相关有代表性的视觉舒适度提升方法对于欠舒适立体图像的处理结果,所提出方法在保持整体场景立体感的同时,能更有效地提升立体图像的视觉舒适度。结论 所提出方法能够根据由不同的立体图像特征构建的视觉舒适度预测模型来自动实施全局线性和局部非线性视差重映射过程,达到既改善立体图像视觉舒适度、又尽量减少视差改变所导致的立体感削弱的目的,从而提升立体图像的整体3维体验。     

All-in-focus and multi-focus color image reconstruction from a database of color and depth image pairs

Owing to recent advances in depth sensors and computer vision algorithms, depth images are often available with co-registered color images. In this paper, we propose a simple but effective method for obtaining an all-in-focus (AIF) color image from a database of color and depth image pairs. Since the defocus blur is inherently depth-dependent, the color pixels are first grouped according to their depth values. The defocus blur parameters are then estimated using the amount of the defocus blur of the grouped pixels. Given a defocused color image and its estimated blur parameters, the AIF image is produced by adopting the conventional pixel-wise mapping technique. In addition, the availability of the depth image disambiguates the objects located far or near from the in-focus object and thus facilitates image refocusing. We demonstrate the effectiveness of the proposed algorithm using both synthetic and real color and depth images.     

Depth Mapping for Stereoscopic Videos

Stereoscopic videos have become very popular in recent years. Most of these videos are developed primarily for viewing on large screens located at some distance away from the viewer. If we watch these videos on a small screen located near to us, the depth range of the videos will be seriously reduced, which can significantly degrade the 3D effects of these videos. To address this problem, we propose a linear depth mapping method to adjust the depth range of a stereoscopic video according to the viewing configuration, including pixel density and distance to the screen. Our method tries to minimize the distortion of stereoscopic image contents after depth mapping, by preserving the relationship of neighboring features and preventing line and plane bending. It also considers the depth and motion coherences. While depth coherence ensures smooth changes of the depth field across frames, motion coherence ensures smooth content changes across frames. Our experimental results show that the proposed method can improve the stereoscopic effects while maintaining the quality of the output videos.     

Visual comfort enhancement study based on visual attention detection for stereoscopic displays

Although numerous potential causes may lead to visual discomfort when viewing content on three‐dimensional (3D) displays, vergence–accommodation conflict is a particular cause of binocular parallax‐based stereoscopic displays, and it is unavoidable. Based on the study of 3D content visual attention, we proposed a novel stereoscopic depth adjustment method to improve the visual comfort and enhance perceived naturalness. The proposed method combined the 3D image saliency and specific viewing condition to establish a novel model for computing the optimum zero‐disparity plane of stereoscopic image. The results of perception experiments, focused on visual comfort and stereoscopic sensation, supported that the proposed method can significantly enhance stereoscopic viewing comfort and even can improve the stereoscopic sensation by insuring the 3D image fusion.     

基于结构失真分析的立体图像质量客观评价模型

在立体图像?视频系统编码、传输、解码等环节中,立体图像质量评价是图像失真的一个重要评判准则.考虑到图像结构特性能较好地反映立体图像质量变化,提出一种基于结构失真的立体图像质量客观评价模型,该模型包括左右视点图像质量和深度感知质量评价两部分.首先根据人眼对图像中不同区域的敏感度存在差异且自然图像具有一定方向奇异性的特性,得到左右视点图像质量评价值;然后利用左右视点的绝对差值图像,采用奇异值向量差异和均值偏差率来描述立体图像深度感的畸变;最后融合前面两部分的评价值得到最终的立体图像质量评价值.实验结果表明,文中模型与主观感知存在较好的一致性,对于高斯模糊、高斯白噪声、JPEG压缩、JP2000压缩以及H.264编码,其评价结果与主观感知之间的相关系数高于0.93,Spearman秩相关系数高于0.92,均方误差低于6.6.     

The effect of depth map up-sampling on the overall quality of stereopairs

The quality of depth maps affects the quality of generated 3D content. Practically, the depth maps often have lower resolution than that of color images, thus, Depth map Up-sampling (DU) is needed in various 3D applications. DU can yield specific artifacts which can degrade the quality of depth maps as well as constructed stereoscopic (color plus depth map) images. This paper investigates the effect of DU on 3D perception. The depth maps were up-sampled using seven approaches and the quality of stereoscopic images obtained from up-sampled depth maps was estimated through subjective and objective tests. The objective quality prediction was performed using a depth map quality assessment framework. The method is able to predict the quality of stereoscopic images through evaluation of their corresponding up-sampled depth maps using 2D Image Quality Metrics (IQMs). In order to improve the quality estimation, the framework selects the 2D IQMs with highest correlation to subjective test. Furthermore, motivated by previous researches on multiple metrics combination, a new metric fusion method is proposed. Experimental results show that the combined metric delivers higher performance than single metrics in 3D quality prediction.     

FPGA-based methodology for depth-of-field extension in a single image

Computer vision applications rely upon high resolution images with extended depth of field (DoF). Most approaches contain arrays of lenses and computing intensive algorithms that must be calibrated every time, to reach in-focus images; however, by changing directly the system focal length, resolution and information are lost. Traditional methods consist in taking a great number of images varying the optical system pupil aperture, whereas, the post processing system demands a great amount of computational resources with long processing time and high implementation cost. In this work a novel methodology for DoF extension that applies a complex-amplitude mask during a single image pre-processing taken at full pupil aperture, and a Wiener filter for the image recovery without focalization errors, during post-processing, is introduced. An FPGA-based implementation shows the feasibility of the proposed methodology for real-time DoF extension. Obtained results demonstrate qualitatively and quantitatively the effectiveness of the proposed FPGA-based method, which offers a reconfigurable solution for online DoF extension on a single image, in real time.     

3D reconstruction of the blood vessels of the brain from a stereoscopic pair of subtraction angiograms

From physiological considerations about human stereoscopic depth perception we derive an algorithm for automatic image matching of a stereoscopic pair of subtraction angiograms. In a first stage, the blood vessels are seen as strings of micropatterns, characterized by an orientation, width and intensity. Similar micropatterns on the left and the right image of the stereo pair are selected. In a second step, this result is iteratively improved using the consistency property of the blood vessels as a priori knowledge.     

A new depth image quality metric using a pair of color and depth images

Typical depth quality metrics require the ground truth depth image or stereoscopic color image pair, which are not always available in many practical applications. In this paper, we propose a new depth image quality metric which demands only a single pair of color and depth images. Our observations reveal that the depth distortion is strongly related to the local image characteristics, which in turn leads us to formulate a new distortion assessment method for the edge and non-edge pixels in the depth image. The local depth distortion is adaptively weighted using the Gabor filtered color image and added up to the global depth image quality metric. The experimental results show that the proposed metric closely approximates the depth quality metrics that use the ground truth depth or stereo color image pair.     

Stereoscopic image generation with depth image based rendering

In this paper, we proposed stereoscopic image generation methods of adjusting the depth value of edge pixels and improved hole filling procedures. For the conventional system, the smooth of depth map can reduce the incidence of image holes, but cause geometric distortions of the image depth. To solve the problems, the depth map is first expanded to refine the accuracy of image depth and the quality of images. Next, we derive a hardware-oriented method for 3D warping. Finally, appropriate blocks are searched to enhance the performance of image by improving hole-filling procedures. The experimental results demonstrate the proposed methods have great performance and practicability.     

彩色图约束的二阶广义总变分深度图超分辨率重建

目的 针对深度图像分辨率非常低的问题,结合同场景高分辨率彩色图像,提出一种基于彩色图约束的二阶广义总变分深度图超分辨率重建方法。方法 首先将低分辨率深度图映射到高分辨率彩色空间;然后利用二阶广义总变分模型,将带有边缘指示函数的高分辨率彩色约束项作为正则项,使得深度图像超分辨率重建问题变成最优求解问题;最后通过迭代重加权和原—对偶方法进行求解。结果 实验结果表明,本文方法可以有效地保护图像的边缘结构,在定性和定量两个方面都可达到很好的效果。结论 本文方法可以有效地解决深度图分辨率非常低的问题。     

沿法线方向传播的深度图像补洞算法

密集的深度信息在计算机视觉的各种任务中有广泛的应用,然而深度相机在有光泽、透明、较远处的物体表面通常无法探测到深度信息,映射到深度图片上形成了大小不一的孔洞。因此,提出了一种沿法线方向传播的单一深度图像补洞算法。本文方法沿着物体本身的曲面变化进行扩散,把深度图片的补全问题转化成几何完成的问题。首先把2D的深度图像扩展到3D点云,然后3D点云沿着孔洞边界的法线方向向内收缩。收缩的过程中加入类似正态滤波器的约束函数来模拟深度的变化,使填充的点云更加适合整个物体的结构。最后把3D点云重新映射到2D图片上。本文的算法在NYU-Depth-v2数据集上进行测试,实验证明本文算法对孔洞的填充有较好的效果。     

一种基于矩不变的深度获取计算方法

针对传统矩不变深度信息获取方法在实际应用中普适性不足,且边缘检测效果不理想的问题,提出一种改进的矩不变深度信息获取方法。首先通过改变像距和光圈参数来获取不同位置的两幅图像,然后用基于图像梯度和灰度标准差相结合的方法检测出每幅图像的边缘,最后根据梯度图像占整幅图像的比值来计算深度信息。实验结果表明该算法提升了边缘检测的效果,减少了计算量,提高了距离估测的准确性和实时性。