Edge-preserving interpolation of depth data exploiting color information

The extraction of depth information associated to dynamic scenes is an intriguing topic, because of its perspective role in many applications, including free viewpoint and 3D video systems. Time-of-flight (ToF) range cameras allow for the acquisition of depth maps at video rate, but they are characterized by a limited resolution, specially if compared with standard color cameras. This paper presents a super-resolution method for depth maps that exploits the side information from a standard color camera: the proposed method uses a segmented version of the high-resolution color image acquired by the color camera in order to identify the main objects in the scene and a novel surface prediction scheme in order to interpolate the depth samples provided by the ToF camera. Effective solutions are provided for critical issues such as the joint calibration between the two devices and the unreliability of the acquired data. Experimental results on both synthetic and real-world scenes have shown how the proposed method allows to obtain a more accurate interpolation with respect to standard interpolation approaches and state-of-the-art joint depth and color interpolation schemes.     

Reliable fusion of ToF and stereo data based on joint depth filter

To obtain reliable depth images with high resolution, a novel method is proposed in this study that fuses data acquired from time-of-flight (ToF) and stereo cameras, through which the advantages of both active and passive sensing are utilised. Based on the classic error model of the ToF, gradient information is introduced to establish the likelihood distribution for all disparity candidates. The stereo likelihood is estimated in parallel based on a 3D adaptive support-weight approach. The two independent likelihoods are unified using a maximum likelihood estimation, a process also referred to as a joint depth filter herein. Conventional post-processing methods such as a mutual consistency check are also used after applying a joint depth filter. We also propose a novel hole-filling method based on the seed-growing algorithm to retrieve missing disparities. Experiment results show that the proposed fusion method can produce reliable high-resolution depth maps and outperforms other compared methods.     

基于深度融合的深度图像修复算法

提出了一种基于深度融合的深度图像修 复算法。对于单幅深度图像,首先利用形态学操作进行空洞区域优化,消除深度图像中的间 隙和随机噪声;然后针对迭 代滤波过程,提出一种新的深度融合策略计算深度值,并通过对空洞区域的分析,判断深度 图像中空洞区 域类型,自适应选择结构元进行迭代操作;最后利用局部深度值重建方法对受损的边缘处深 度值进行修复。 实验结果表明,本文算法在较好的修复深度图像中存在空洞和间隙的同时,能够保持原始深 度图深度值分 布规律,克服修复过程中存在的深度值失真,边缘模糊等不足。基于标准数据集Middlebury 的对比试验结果表明,本文算法与其它算法相比,获得了良好的效果。     

基于图像边缘特征的深度上采样算法

针对低分辨率深度图像上采样容易导致边缘模糊问 题,提出了一种基 于图像边缘特征的深度上采样算法。一方面,利用相同低分辨率深度和彩色图像的相关性系 数,自适应调 节深度图像边缘上采样过程中深度和彩色的权重;另一方面,结合上采样值和低分辨率深度 图像中邻近像 素值,对低分辨率深度图像的不连续区域进行求精操作以进一步减少边缘模糊现象。实验结 果表明,本文算法 的性能优于近年文献中提出的算法。本算法上采样深度图像的平均坏点率(B PR)为2.07%,均方根误差(RMSE )为3.46,峰值信噪比(PSNR)为38.58dB,绘制 虚拟视点的平均PSNR为39.58dB。     

基于梯度重建与形态学分水岭算法的图像分割

由于分水岭算法存在着过分割的问题,文章提出了一个有效解决该问题的方法。首先,在图像预处理过程中先对图像进行形态学滤波,消除部分噪声;其次,采用形态学求梯度的方法得到原始图像的梯度图并对其进行开闭重建,在保留区域重要轮廓的同时去除噪声和图像细节;第三,对重建后的梯度图像进行基于标记约束的分水岭分割。试验结果表明：该方法能够很好地抑制过分割,同时通过结构元素的选择而具备一定的灵活性,整个过程无需进行合并处理,从而降低了分割的复杂性。     

DIBR中基于平面扫描的深度重建方法

提出一种 DIBR 中基于平面扫描法的深度重建方法,与立体深度重建算法和基于图像的视觉壳算法不同,本文进一步改进平面扫描算法,无需任何场景的几何先验知识,而是利用每个像素点的深度信息合成真实场景的虚拟视点.当输入图像映射至相互平行的虚拟深度平面时,采用"动态判决方法"来计算像素间的色彩一致度;并在虚拟视合成中采用了基于视向权重策略的新视点重建方法.本文算法获取的深度信息更为精确,虚拟新视点的质量得到较大提高.     

红外图像超分辨率重建技术研究

利用图像超分辨率重建技术可以在不改进硬件的情况下提升现有成像系统的图像分辨率。为提升红外图像质量,提出了一种基于NSCT变换与自适应正则化重建相结合的图像超分辨率重建算法。算法充分考虑了实际红外图像中噪声特点,利用NSCT变换特点在尽可能减少图像信息损失的条件下,对加性噪声与乘性噪声采用不同策略进行了抑制,并对预处理后的红外图像序列进行自适应正则化重建。实验结果表明本算法处理后的红外图像在主观视觉效果与客观指标上较传统图像处理方法均有改善。     

Reconstruction of lost depth data in multiview video-plus-depth communications using geometric transforms

This paper addresses depth data recovery in multiview video-plus-depth communications affected by transmission errors and/or packet loss. The novel aspects of the proposed method rely on the use of geometric transforms and warping vectors, capable of capturing complex motion and view-dependent deformations, which are not efficiently handled by traditional motion and/or disparity compensation methods. By exploiting the geometric nature of depth information, a region matching approach combined with depth contour reconstruction is devised to achieve accurate interpolation of arbitrary shapes within lost regions of depth maps. The simulation results show that, for different packet loss rates, up to 20%, the depth maps recovered by the proposed method produce virtual views with better quality than existing methods based on motion information and spatial interpolation. An average PSNR gain of 1.48 dB is obtained in virtual views synthesised from depth maps using the proposed method.     

噪声对大气痕量气体分布重建的影响

针对大气痕量气体浓度的二维分布重建中的噪声问题,采用数值模拟的方法研究了噪声对差分吸收光谱断层扫描技术重构大气痕量气体空间分布的影响,建立了噪声的模型,并研究了噪声对代数迭代重建算法(ART)和联合迭代重建算法(SIRT)两种重建算法的重建效果的影响,ART算法重建后,误差Ea 由0.1080降低到了0.0182; SIRT算法重建后误差Ea 由0.0476降低到了0.0102。结果表明,随着随机噪声的增加,重建图像变态程度越来越严重。由重建相对误差指标可以看出,随着信噪比的增加,重建相对误差越来越小。因此,在外场重建试验中应该尽量避免噪声。     

基于Kinect的实时深度提取与多视绘制算法

提出了一种基于Kinect的实时深度提取算法和单纹理+深度的多视绘制方法。在采集端,使用Kinect提取场景纹理和深度,并针对Kinect输出深度图的空洞提出一种快速修复算法。在显示端,针对单纹理+深度的基于深度图像的绘制(DIBR,depth image based rendering)绘制产生的大空洞,采用一种基于背景估计和前景分割的绘制方法。实验结果表明,本文方法可实时提取质量良好的深度图,并有效修复了DIBR绘制过程中产生的大空洞,得到质量较好的多路虚拟视点图像。以所提出的深度获取和绘制算法为核心,实现了一种基于深度的立体视频系统,最终的虚拟视点交织立体显示的立体效果良好,进一步验证了本文算法的有效性。本文系统可用于实景的多视点立体视频录制与播放。     

基于边缘增强的深度图超分辨率重建

准确的深度图像获取是计算机视觉中的一个难题 。传统的立体匹配得到深度的方法不仅计算量大,而且在纹理稀疏与重复区域往往存在较大 的误差。主动式深度传感器虽然解决了这些问题, 但其获取的深度图存在着分辨率低和易受噪声干扰的问题。因此,本文提出一种结合彩色图 像信息 的深度图超分辨率(SR)重建方法来提高深度图的质量与分辨率。首先运用自回归(AR)模型 下的非局部均值(NLM)算 法获取初始的上采样深度图;然后利用边缘提取与边缘修复算法优化深度图。实验结果表 明,本文提出的方法能够生成误差更小、主观质量更好的高分辨率深度图。     

基于L曲线的自适应时空正则化视频超分辨率重建

为了解决视频超分辨率重建的病态问题,以得到良好的重建效果,提出了一种新颖的视频超分辨率重建算法。在算法中引入了时空联合正则化算子,通过视频帧本身的空间平滑信息和视频相邻帧的帧间相关先验信息的引入,提高了解的质量;同时,为了选择合适的时空正则化系数,提出了基于L曲线的自适应时空正则化系数计算方法,可以自适应地计算合适的正则化系数。通过对模拟图像序列和真实视频序列的实验结果表明,算法能得到较为精确的解,重建出具有良好视觉效果的高分辨率视频。     

TMSO-Net: Texture adaptive multi-scale observation for light field image depth estimation

Light field can record the four-dimensional information of light rays, i.e. the position and direction information in which depth information is implied. To improve the depth estimation accuracy, we propose a depth estimation algorithm based on convolutional neural network (CNN). First, a single image super resolution algorithm is adopted to spatially super resolve the sub-aperture images (SAIs). Second, to adapt the texture complexity, the SAIs are partitioned into two regions, i.e., simple texture region and complex texture region, based on the texture analysis of the central SAI. Third, the epipolar plane images (EPIs) in horizontal, vertical, 45 degree diagonal, and 135 degree diagonal directions for both complex and simple texture regions are extracted, and the corresponding EPIs for the simple and complex texture regions are fed into the specified network branches. Finally, a fusion module is designed to generate the depth map. Experimental results show that the quality of the estimated depth maps by the proposed method is better than the state-of-the-art methods in terms of both objective quality and subjective quality. Moreover, the proposed method is more robust to noise.     

基于联合代数重建技术的介观荧光分子层析成像

针对介观荧光分子层析成像重建问题,提出了一种基于联合代数重建技术的介观荧光分子层析成像重建方法.首先应用主成分分析算法对敏感矩阵进行双重降维操作,以消除敏感矩阵中的冗余信息;其次为保持重建结果与目标数据的一致性,对降维后的矩阵进行零填充;最后应用荧光光强测量数据和填充后的敏感矩阵,经带有总变差正则化项的联合代数重建技术...     

Color Noise Reduction Method in Non‐constant Luminance Signal for High Dynamic Range Video Service

A high dynamic range (HDR) video service is an upcoming issue in the broadcasting industry. For compatibility with legacy devices receiving a non‐constant luminance (NCL) signal, new tools supporting an HDR video service are required. The current pre‐processing chain of HDR video can produce color noise owing to the chroma component down‐sampling process for video encoding. Although a luma adjustment method has been proposed to solve this problem, some disadvantages still remain. In this paper, we present an adaptive color noise reduction method for an NCL signal of an HDR video service. The proposed method adjusts the luma component of an NCL signal adaptively according to the information of the luma component from a constant luminance signal and the level of color saturation. Experiment results show that the color noise problem is resolved by applying our proposed method. In addition, the speed of the pre‐processing is increased more than two‐fold compared to a previous method.     

于形态学梯度重建的分水岭分割

提出一种基于形态学梯度重建的分水岭图像分割方法.该方法在形态学梯度图像的基础上,利用形态学开闭重建运算对梯度图像进行重建,在保留重要区域轮廓的同时去除了细节和噪声.避免了标准分水岭存在的过分割现象及传统形态学开闭运算先平滑原始图像,后进行分水岭变换而造成的区域轮廓位置偏移.仿真实验证明,无论从消除过分割还是区域轮廓定位等性能方面,该方法均具有较好的分割效果.整个分割过程无需进行分割后的区域合并处理,降低了分割的复杂性;且分割过程只需选择合适的结构元素大小,增强了算法的灵活性.     

基于无监督学习的ToF点云目标提取方法

飞行时间(Time of Flight,ToF)三维成像技术在人工智能领域具有重要的应用价值。间接ToF三维成像是通过向目标发射调制的光强信号,再经过目标反射到相位解调图像传感器获得相位差,通过计算获得目标的深度信息。由于间接ToF成像技术会受到背景界面多次反射产生的多路径干扰,因此在复杂环境中目标物体深度测量数据会受到侧面和背景界面的多次反射的回波信号影响,降低边缘处深度测量的精度水平,因此需要对原始点云数据进行目标提取和多路径去除的预处理。本文针对该问题提出一种多界面场景中基于点云矢量的目标提取方法,能够实现复杂多目标的快速提取和多路径强干扰的去除。首先基于kmeans提出一种FVPkmeans算法,完成目标点云数据的全局全矢量提取处理。再基于K NN提出一种迭代滤波算法,实现局部多路径干扰数据的滤除。通过与其它方法的比较研究,该方法能够有效去除TOF点云目标数据的多路径干扰,目标提取性能提高了40,实验表明本文提出的全局点云数据全矢量目标提取和多路径干扰去除算法能够实现对目标点云数据的无监督学习智能提取与滤波要求。     

A new fuzzy color correlated impulse noise reduction method.

A new impulse noise reduction method for color images is presented. Color images that are corrupted with impulse noise are generally filtered by applying a grayscale algorithm on each color component separately or using a vector-based approach where each pixel is considered as a single vector. The first approach causes artefacts especially on edge and texture pixels. Vector-based methods were successfully introduced to overcome this problem. Nevertheless, they tend to cluster the noise and to receive a lower noise reduction performance. In this paper, we discuss an alternative technique which gives a good noise reduction performance while much less artefacts are introduced. The main difference between the proposed method and other classical noise reduction methods is that the color information is taken into account to develop (1) a better impulse noise detection method and (2) a noise reduction method that filters only the corrupted pixels while preserving the color and the edge sharpness. Experimental results show that the proposed method provides a significant improvement on other existing filters.     

基于向量化的稀疏重构解相干算法

针对稀疏重构中正交匹配追踪（Orthogonally Matched Pursuit,OMP）算法解相干问题,利用矢量化的接收数据自相关矩阵,提出一种改进解相干方法——矢量化正交匹配追踪（Vectorized OMP,VO）算法.改进方法只通过矢量化后的一维矢量来重构角度,无需知道信号源的数目,即可降低噪声影响,实现解相干.相对于经典OMP算法,稀疏重构效果更优.理论分析和仿真结果都验证了算法的良好性能.