基于分割的离焦图像深度图提取方法

针对影视作品中的大量离焦图像,提出了一种离焦图像的深度图提取方法。将离焦图像的聚焦前景和离焦背景进行分离。对离焦背景提出了深度图模型匹配的方法,构建深度图模型并结合人眼视觉对场景深度的敏锐判断,将背景与对应的深度图模型进行匹配,实现背景深度图的构建;提出了基于颜色分割的深度图再处理,来进一步提高场景深度图的精度。对前景采用单深度赋值,并结合背景深度图融合生成最终深度图。实验表明采用该方法提取的深度图在深度跳跃和深度平滑区域都得到了好的效果。     

加权SIFT流深度迁移的单幅图像2D转3D

2D视频转3D视频是解决3D片源不足的主要手段,而单幅图像的深度估计是其中的关键步骤.提出基于加权SIFT流深度迁移和能量模型优化的单幅图像深度提取方法.首先利用图像的全局描述符从深度图数据库中检索出近邻图像;其次通过SIFT流建立输入图像和近邻图像之间像素级稠密对应关系;再次由SIFT流误差计算迁移权重,将近邻图像对应像素点的深度乘以权重后迁移到输入图像上;然后利用均值滤波对迁移后的近邻图像深度进行融合;最后建立深度图优化能量模型,在尽量接近迁移后近邻图像深度的前提下,平滑梯度较小区域的深度.实验结果表明,该方法降低了估计深度图的平均相对误差,增强了深度图的均匀性.     

基于边缘增强的深度图超分辨率重建

准确的深度图像获取是计算机视觉中的一个难题 。传统的立体匹配得到深度的方法不仅计算量大,而且在纹理稀疏与重复区域往往存在较大 的误差。主动式深度传感器虽然解决了这些问题, 但其获取的深度图存在着分辨率低和易受噪声干扰的问题。因此,本文提出一种结合彩色图 像信息 的深度图超分辨率(SR)重建方法来提高深度图的质量与分辨率。首先运用自回归(AR)模型 下的非局部均值(NLM)算 法获取初始的上采样深度图;然后利用边缘提取与边缘修复算法优化深度图。实验结果表 明,本文提出的方法能够生成误差更小、主观质量更好的高分辨率深度图。     

基于区域融合的单视点图像深度信息提取

将2D转换为3D是目前3D多媒体成功应用的重要解决方法之一.而在2D至3D转换过程中,单视点图像中深度信息提取是最具挑战的任务.提出一种基于区域融合的单视点图像深度信息提取方法,利用区域融合对单视点图像进行区域融合,用先验假设的深度梯度图对区域进行深度分配,得到基于区域融合的深度图.实验结果表明通过所提方法得到的深度图...     

面向虚拟视点图像绘制的深度图编码算法

针对不同区域对绘制虚拟视点图像质量产生不同的影响,以及深度估计不准确导致时域抖动效应影响压缩效率的问题,提出了一种面向虚拟视点图像绘制的深度图压缩算法。通过彩色图像帧差、深度图边缘提取等相关处理过程,提取深度图的静态区域、边缘区域以及动态区域。对深度图边缘区域使用了较低的量化系数,以提高深度图边缘区域编码质量;根据深度图各个区域的编码模式特点,仅对部分编码模式而不是所有模式进行率失真优化搜索,以提高深度图的编码速度;对于深度图P帧的静态区域,合理地采用了SKIP模式,以消除由于深度估计算法的局限性导致时域抖动效应对深度图压缩的影响。实验结果表明,与传统的H.264编码方案相比,本文方案在传输码流大小基本不变的前提下提高了最终虚拟视点图像边缘区域的绘制质量,其余区域主观质量相近,而深度图编码时间则节省了约77～87%。     

单幅图像的深度标签流形学习

图像中背景与前景对象的空间位置决定了场景在图像中的相对深度,利用图像的局部特征相似性和流形结构的降维性能,并应用salient区域DCT高频系数分布的深度排序索引性能,定义出图像深度的马尔科夫概率图模型MRF。通过划分场景对象检测salient区域模糊度,最后估计得出图像场景的相对深度图。通过学习图像数据的流形嵌入对数据流形分布概率密度函数进行迁移,得出遵循相似流形分布的对象特征类别标记概率密度分布。进一步检测空间变化salient区的模糊程度,融合多尺度梯度幅度的高频离散余弦变换DCT系数特征,依据模糊变化高频特征计算深度标记索引确定深度标签的层级次序,融合类别标签以生成深度图。这种模型框架下检测单个图像中模糊和未模糊的区域,可获得图像中场景的相对深度,而无需了解相机设置或模糊类型的先验参数。在典型的深度图估计数据集中应用MRF深度图模型评测图像的深度估计性能,实验结果给出该方法在检测场景分布和划分场景深度次序上的准确率,验证了方法的有效性。     

基于深度学习的鱼类摄食状态识别算法

针对传统评估鱼类摄食状态的算法易受养殖环境光照和水质条件限制,图像特征提取困难、识别效率低的问题,提出了一种利用卷积神经网络模型识别鱼类摄食深度图像的方法,使用深度相机获取鱼群摄食深度图像,采用距离特征的方法实现鱼群目标提取和背景图像消除,通过HSV颜色转换算法将鱼类的距离信息线性转换为颜色信息,解决了深度图中目标深度信息表示困难的问题。经过预处理的深度图轮廓清晰、颜色鲜明、特征提取容易,设计一种结构简单的神经网络模型对鱼类摄食深度图进行识别,实验结果表明,模型识别的平均准确率为97.81%,相比于其他复杂模型,权重空间降低了90%,训练速度提高了三倍,有效降低了识别模型的复杂度。     

一种用于深度图编码的虚拟视失真估计模型

多视视频加深度(MVD,multi-view video plus depth)的3D视频格式中,深度图提供视频的场景几何信息,其不在终端成像显示而是通过基于深度图像的绘制(DIBR)技术用于绘制虚拟视图像。在深度图的压缩编码过程中,深度图的失真会引起绘制的虚拟视图像的失真。深度图用于绘制而不用于显示的特性使得准确估计深度图绘制的虚拟视失真可以提高深度图编码的率失真性能。本文分析了不同的深度图失真引起的不同的虚拟视失真,进而提出一种估计深度图失真引起虚拟视失真的指数模型,并将模型用于深度图编码的率失真优化(RDO)中。实验结果表明,本文提出的模型可以准确估计深度图失真引起的虚拟视失真,提高深度图编码性能,相比于HTM的VSO可以降低约10%的编码时间,并且虚拟视质量略优于HTM。     

深度成像理论与实现

传统成像获取信息不足,成像质量有一定局限性。为此,提出了一种深度成像模型。模型包含深度矩阵、分解函数、散焦算子、自适应正则项等部分。深度矩阵的获取有双目立体视觉、结构光或飞行时间法等实现方法;分解函数用于将图像按深度值的不同分割为若干子图像;散焦算子可以通过深度散焦法来计算;自适应正则项的引入能减少图像的阶梯效应,增强图像的光滑性。通过局部标准差和局部平均梯度这两个评价指标检验深度成像模型的效果。实验结果表明,深度成像模型效果显著。     

基于图割的高质量深度图获取方法

3DTV作为下一代视频广播系统,还有许多技术难点有待解决,其中深度估计是3DTV的关键技术之一。为了获取高质量的深度图,提出基于图割(graph cut)的深度估计方法。该算法在构建能量函数的数据项时,通过对窗口内各个像素赋予自适应权重,引入梯度信息以抑制因亮度差异导致的误匹配问题并保护边缘信息。然后,经过交叉检测将深度图像素分为可靠点与不可靠点两类。对检测后的深度图进行后处理迭代优化,从而提高所获取深度值的可靠性。实验表明此算法估计出的深度图用VSRS绘制虚拟合成视时比标准的深度估计软件DERS5.1可有效提高虚拟视质量。     

Single-image depth estimation using relative depths

Depth estimation from a single RGB image is a challenging task. It is ill-posed since a single 2D image may correspond to various 3D scenes at different scales. On the other hand, estimating the relative depth relationship between two objects in a scene is easier and may yield more reliable results. Thus, in this paper, we propose a novel algorithm for monocular depth estimation using relative depths. First, using a convolutional neural network, we estimate two types of depths at multiple spatial resolutions: ordinary depth maps and relative depth tensors. Second, we restore a relative depth map from each relative depth tensor. A relative depth map is equivalent to an ordinary depth map with global scale information removed. For the restoration, sparse pairwise comparison matrices are constructed from available relative depths, and missing entries are filled in using the alternative least square (ALS) algorithm. Third, we decompose the ordinary and relative depth maps into components and recombine them to yield a final depth map. To reduce the computational complexity, relative depths at fine spatial resolutions are directly used to refine the final depth map. Extensive experimental results on the NYUv2 dataset demonstrate that the proposed algorithm provides state-of-the-art performance.     

Stereoscopic view synthesis based on region-wise rendering and sparse representation

Depth image-based rendering (DIBR), which is used to render virtual views with a color image and the corresponding depth map, is one of the key techniques in the 2D to 3D conversion process. One of the main problems in DIBR is how to reduce holes that occur on the generated virtual view images. In this paper, we make two main contributions to deal with the problem. Firstly, a region-wise rendering framework, which divides the original image regions into three special classes and renders each with optimal adaptive process respectively, is introduced. Then, a novel sparse representation-based inpainting method, which can yield visually satisfactory results with less computational complexity for high quality 2D to 3D conversion, is proposed. Numerical experimental results demonstrate the good performance of the proposed methods.     

Depth completion towards different sensor configurations via relative depth map estimation and scale recovery

Depth completion, which combines additional sparse depth information from the range sensors, substantially improves the accuracy of monocular depth estimation, especially using the deep-learning-based methods. However, these methods can hardly produce satisfactory depth results when the sensor configuration changes at test time, which is important for real-world applications. In this paper, the problem is tackled by our proposed novel two-stage mechanism, which decomposes depth completion into two subtasks, namely relative depth map estimation and scale recovery. The relative depth map is first estimated from a single color image with our designed scale-invariant loss function. Then the scale map is recovered with the additional sparse depth. Experiments on different densities and patterns of the sparse depth input show that our model always produces satisfactory depth results. Besides, our approach achieves state-of-the-art performance on the indoor NYUv2 dataset and performs competitively on the outdoor KITTI dataset, demonstrating the effectiveness of our method.     

软分割约束边缘保持插值的半自动2D转3D

半自动2D转3D将用户标注的稀疏深度转换成稠密深度,是解决3D片源不足的主要手段之一.针对现有方法利用硬分割增强深度边缘引入误差的问题,提出像素点与超像素深度一致性约束的边缘保持插值方法.首先,建立像素点深度和超像素深度传播的能量模型,通过像素点与所属超像素间深度差异的约束项将二者关联起来;其次,利用矩阵表示形式将两个能量模型的最优化转换成一个稀疏线性方程组的求解问题.通过超像素提供的约束项,可避免深度传播穿过低对比度边缘区域,从而能保持对象边缘.实验结果表明,本文方法对象边缘处深度恢复的准确性优于融合图割的随机游走方法,PSNR改善了1.5dB以上.     

Fast Extraction of Objects of Interest from Images with Low Depth of Field

In this paper, we propose a novel unsupervised video object extraction algorithm for individual images or image sequences with low depth of field (DOF). Low DOF is a popular photographic technique which enables the representation of the photographer's intention by giving a clear focus only on an object of interest (OOI). We first describe a fast and efficient scheme for extracting OOIs from individual low‐DOF images and then extend it to deal with image sequences with low DOF in the next part. The basic algorithm unfolds into three modules. In the first module, a higher‐order statistics map, which represents the spatial distribution of the high‐frequency components, is obtained from an input low‐DOF image. The second module locates the block‐based OOI for further processing. Using the block‐based OOI, the final OOI is obtained with pixel‐level accuracy. We also present an algorithm to extend the extraction scheme to image sequences with low DOF. The proposed system does not require any user assistance to determine the initial OOI. This is possible due to the use of low‐DOF images. The experimental results indicate that the proposed algorithm can serve as an effective tool for applications, such as 2D to 3D and photorealistic video scene generation.     

Adaptive depth truncation filter for MVC based compressed depth image

In multiview video plus depth (MVD) format, virtual views are generated from decoded texture videos with corresponding decoded depth images through depth image based rendering (DIBR). 3DV-ATM is a reference model for the H.264/AVC based multiview video coding (MVC) and aims at achieving high coding efficiency for 3D video in MVD format. Depth images are first downsampled then coded by 3DV-ATM. However, sharp object boundary characteristic of depth images does not well match with the transform coding based nature of H.264/AVC in 3DV-ATM. Depth boundaries are often blurred with ringing artifacts in the decoded depth images that result in noticeable artifacts in synthesized virtual views. This paper presents a low complexity adaptive depth truncation filter to recover the sharp object boundaries of the depth images using adaptive block repositioning and expansion for increasing the depth values refinement accuracy. This new approach is very efficient and can avoid false depth boundary refinement when block boundaries lie around the depth edge regions and ensure sufficient information within the processing block for depth layers classification. Experimental results demonstrate that the sharp depth edges can be recovered using the proposed filter and boundary artifacts in the synthesized views can be removed. The proposed method can provide improvement up to 3.25 dB in the depth map enhancement and bitrate reduction of 3.06% in the synthesized views.     

Disparity field and depth map coding for multiview 3D image generation

In the present paper techniques are examined for the coding of the depth map and disparity fields for stereo or multiview image communication applications. It is assumed that both the left and right channels of the multiview image sequence are coded using block- or object-based methods. A dynamic programming algorithm is used to estimate a disparity field between each stereo image pair. Depth is then estimated and occlusions are optionally detected, based on the estimated disparity fields. Spatial interpolation techniques are examined based on the disparity/depth information and the detection of occluded regions using either stereoscopic or trinocular camera configurations. It is seen that the presence of a third camera at the transmitter site improves the estimation of disparities, the detection of occlusions and the accuracy of the resulting spatial interpolation at the receiver. Various disparity field and depth map coding techniques are then proposed and evaluated, with emphasis given to the quality of the resulting intermediate images at the receiver site. Block-based and wireframe modeling techniques are examined for the coding of isolated depth or disparity map information. Further, 2D and 3D motion compensation techniques are evaluated for the coding of sequences of depth or disparity maps. The motion fields needed may be available as a byproduct of block-based or object-based coding of the intensity images. Experimental results are given for the evaluation of the performance of the proposed coding and spatial interpolation methods.     

3D modeling and rendering from multiple wide-baseline images by match propagation

In this paper, we present a method for modeling a complex scene from a small set of input images taken from widely separated viewpoints and then synthesizing novel views. First, we find sparse correspondences across multiple input images and calibrate these input images taken with unknown cameras. Then one of the input images is chosen as the reference image for modeling by match propagation. A sparse set of reliably matched pixels in the reference image is initially selected and then propagated to neighboring pixels based on both the clustering-based light invariant photoconsistency constraint and the data-driven depth smoothness constraint, which are integrated into a pixel matching quality function to efficiently deal with occlusions, light changes and depth discontinuity. Finally, a novel view rendering algorithm is developed to fast synthesize a novel view by match propagation again. Experimental results show that the proposed method can produce good scene models from a small set of widely separated images and synthesize novel views in good quality.