融合一致性与差异性约束的光场深度估计

光场图像深度估计是光场三维重建、目标检测、跟踪等应用中十分关键的技术.虽然光场图像的重聚焦特性为深度估计提供了非常有用的信息,但是在处理遮挡区域、边缘区域、噪声干扰等情况时,光场图像深度估计仍然存在很大的挑战.因此,提出了一种基于极平面图(Epipolar plane image,EPI)斜线像素一致性和极平面图区域差异性的深度估计算法用于解决遮挡和噪声问题.EPI斜线像素的一致性采用旋转线性算子(Spinning linear operator,SLO)的颜色熵度量,能够提高深度图边缘的准确性以及抗噪能力;EPI区域的差异性采用旋转平行四边形算子(Spinning parallelogram operator,SPO)的卡方x2度量,能够提高深度图深度渐变区域的准确性,并使用置信度加权的方法将两种度量进行融合,可以减少遮挡区域和噪声的干扰.另外,充分利用像素邻域的颜色相似性,使用引导保边滤波器和马尔科夫随机场(Markov random field,MRF)全局优化策略进行后处理,进一步减少深度图的边缘错误,得到遮挡边缘准确的深度图.在HCI光场数据集上进行了实验,并与经典光场深度估计算法进行了对比,结果表明该算法在主观质量和客观指标两方面都有明显提升.     

融合一致性与差异性约束的光场深度估计

光场图像深度估计是光场三维重建、目标检测、跟踪等应用中十分关键的技术.虽然光场图像的重聚焦特性为深度估计提供了非常有用的信息,但是在处理遮挡区域、边缘区域、噪声干扰等情况时,光场图像深度估计仍然存在很大的挑战.因此,提出了一种基于极平面图(Epipolar plane image,EPI)斜线像素一致性和极平面图区域差异性的深度估计算法用于解决遮挡和噪声问题.EPI斜线像素的一致性采用旋转线性算子(Spinning linear operator,SLO)的颜色熵度量,能够提高深度图边缘的准确性以及抗噪能力;EPI区域的差异性采用旋转平行四边形算子(Spinning parallelogram operator,SPO)的卡方x2度量,能够提高深度图深度渐变区域的准确性,并使用置信度加权的方法将两种度量进行融合,可以减少遮挡区域和噪声的干扰.另外,充分利用像素邻域的颜色相似性,使用引导保边滤波器和马尔科夫随机场(Markov random field,MRF)全局优化策略进行后处理,进一步减少深度图的边缘错误,得到遮挡边缘准确的深度图.在HCI光场数据集上进行了实验,并与经典光场深度估计算法进行了对比,结果表明该算法在主观质量和客观指标两方面都有明显提升.     

面向三维高效视频编码的深度图错误隐藏

基于多视点视频序列视点内、视点间存在的相关性,并结合视点间运动矢量共享技术,该文提出一种面向3维高效视频编码中深度序列传输丢包的错误隐藏算法。首先,根据3D高效视频编码(3D-HEVC)的分层B帧预测(HBP)结构和深度图纹理特征,将深度图丢失块分成运动块和静止块;然后,对于受损运动块,使用结合纹理结构的外边界匹配准则来选择相对最优的运动/视差矢量进行基于位移矢量补偿的错误掩盖,而对受损静止块采用参考帧直接拷贝进行快速错误隐藏;最后,使用参考帧拆分重组来获取新的运动/视差补偿块对修复质量较差的重建块进行质量提升。实验结果表明:相较于近年提出的对比算法,该文算法隐藏后的深度帧平均峰值信噪比(PSNR)能提升0.25～2.03 dB,结构相似度测量值(SSIM)能提升0.001～0.006,且修复区域的主观视觉质量与原始深度图更接近。     

基于运动和视差信息的立体视频质量客观评价

在研究人类 立体视觉特性及现有立体图像/视频质量评价算法的基础上,提出了一种基于运动信息和视 差信息的立 体视频质量的客观评价方法。方法包括视频质量评价(VQA)和视频立体感评价(VSSA)两个指 标,其中VQA的估计基于梯度的结构相似度(GSSIM) 算法,并充分考虑了帧内的亮度信息和结构信息、帧间运动信息以及人眼的感知特性对视频 质量的影响, 特别是根据人类的视觉特性,对左右视点的质量赋予了不同的权重;VSSA的估计 是通过计算参考 视频的绝对差值图和降质视频的绝对差值图之间的峰值信噪比(PSNR)而得到。实验结果表明,本文方法对基于H.264 编码的失真视频的评价结果与主观测试有较高的一致性,很好地体现人眼的视觉特性。     

基于特征分析的深度视频处理算法

在自由视点视频(FVV)系统中,为了提供更真 实的感受,需要通过基于深度的虚拟视点绘制技术获 取足够多的视点,然而由于深度视频获取技术的限制,深度视频的质量很差。为了提高深 度视频的质量, 本文提出了一种基于区域特征的深度视频处理算法。首先,使用模糊C均值(FC M)聚类算 法将深度视频自适应地把深度视频像素分为5类,同时检测深度图的边缘并将边缘块化;其 次,对于非边 缘像素,根据其所在类的特点分别采用改进的三边滤波器及水平方向均值滤波器进行处理; 最后,对彩色 和深度视频分别进行角点检测并块化,并根据边缘像素是否在彩色和深度的角点交集区域内 而采用不同窗 口的中值滤波。实验结果表明,与使用原始深度视频绘制的虚拟视点相比,本文算法能够在 将虚拟视点的客 观质量平均提高0.43dB的同时,显著提高虚拟视点的主观质量;与M artin算法相比,无论是主观质量还是客观质量,本文算法的效果都更加显著。     

压缩视频感觉质量的计算

论文提出了基于视觉特性、视频内容和模糊学的压缩视频感觉质量计算方法.方法的核心是利用视觉掩蔽特性.论文分析了与压缩视频质量有关的视觉特性及视频图像内容特性,提出了视觉掩蔽计算结构及用模糊学方法进行视觉阈值提升的计算方法,并给出了感觉峰值信噪比的压缩视频质量的计算方法,因而该方法使压缩视频质量的度量更符合人的视觉特性.     

基于纹理平滑度的视点合成失真优化快速算法

针对3D-HEVC中深度图编码采用的视点合成失真优化方法的高复杂度问题,提出一种基于纹理平滑度的快速算法。首先结合帧内DC预测特性和统计学方法分析平坦纹理图中像素规律并设定基于纹理图平坦度的跳过准则;然后在深度图编码采用视点合成失真优化方法时提前分离出纹理图平坦区域所对应的深度图区域,并终止该区域像素基于虚拟视点合成的视点合成失真计算过程。实验结果证明该算法的有效性,能在保持编码质量的同时减少大量编码时间。     

一种低复杂度立体视频错误隐藏新方法

针对立体视频流传输中右视 点整帧丢失,提出 了一种低复杂度的错误隐藏算法。首先,为了高效地感知立体视频的时域质量和视点间质量 ,定义了时域相似尺度(TSM)、 视间相似尺度(ISM)的概念;将前一时刻右视点图像进行时域和视点间匹配,分别求取 其以像素为单位的TSM和 ISM映射图;然后,计算前一时刻右视点图像当前宏块的TSM和ISM值,通过比较得 到当前宏块的预测模式;最后,根据视频序列的时域一致性,将前一时刻右视点图像宏块 的预测模式作为丢失图像宏 块的预测模式,从而使用运动补偿预测(MCP)或者视差补偿预测(DCP )的方法恢复丢失信息。研究结果表明,与已有错误隐藏 算法相比,本文算法获得更好主客观视觉效果;同时与基于图像结构相似度(SSIM)的错误隐藏算法相比,在保持主观视觉质量情况下,错误隐藏时间节省20%左右。     

基于中心凹的双目恰可察觉编码失真模型

针对立体视频中存在着大量的感知冗余,首先通过主观实验研究梯度和纹理因素对立体掩蔽 效应的影响,建立双目恰可察觉编码失真(BJNCD)模型;同时考虑到人 类视觉特性(HVS)的视觉敏感度并非恒定不变,当视网膜离心率变大 时,像素的视觉阈值也随 之变大,因此结合HVS的视网膜中心凹感知特性建立了基于中心凹的双目恰可察觉编码失真(FBJNCD)模型。为验证所提出模型的有效性,将其 应用于多视点高效视频编码(MV-HEVC)测试平台中对立体视频进行非 对称编码。实验结果表明,所提出模型在保持立体视频感知质量的同时能够有效节省编码码 率,提高了立体视频压缩效率。     

基于视频质量评价的时域失真测量

视频质量评价(VQA)对于视频处理应用有着重要影响.人眼视觉特性的时域掩蔽效应和时域失真波动是视频质量感知评价的关键因子.在已有的视频质量评价研究中,很少有考虑到时域失真波动对视频主观感知质量的影响.改进了传统的时域分析算法,并证明了视频质量评价算法中时域失真的有效性.     

Joint-adaptive bilateral depth map upsampling

In this paper, we present a new depth upsampler in which an upsampled depth map is computed at each pixel as the average of neighboring pixels, weighted by color and depth intensity filters. The proposed method features two parameters, an adaptive smoothing parameter and a control parameter. The adaptive smoothing parameter is determined based on the ratio between a depth map and its corresponding color image. The adaptive smoothing parameter is used to control the dynamic range of the color-range filter. The control parameter assigns a larger weighting factor to pixels in the object to which a missing pixel belongs. In a comparison with five existing upsamplers, the proposed method outperforms all five in terms of both objective and subjective quality.     

Vision-model-based impairment metric to evaluate blocking artifactsin digital video

In this paper investigations are conducted to simplify and refine a vision-model-based video quality metric without compromising its prediction accuracy. Unlike other vision-model-based quality metrics, the proposed metric is parameterized using subjective quality assessment data recently provided by the Video Quality Experts Group. The quality metric is able to generate a perceptual distortion map for each and every video frame. A perceptual blocking distortion metric (PBDM) is introduced which utilizes this simplified quality metric. The PBDM is formulated based on the observation that blocking artifacts are noticeable only in certain regions of a picture. A method to segment blocking dominant regions is devised, and perceptual distortions in these regions are summed up to form an objective measure of blocking artifacts. Subjective and objective tests are conducted and the performance of the PBDM is assessed by a number of measures such as the Spearman rank-order correlation, the Pearson correlation, and the average absolute error The results show a strong correlation between the objective blocking ratings and the mean opinion scores on blocking artifacts     

A new reduced reference metric for color plus depth 3D video

A new reduced reference (RR) objective quality metric for 3D video is proposed that incorporates spatial neighboring information. The contrast measures from gray level co-occurrence matrices (GLCM) for both color and depth sections are main parts of spatial information. Side information is extracted from edge properties of reference 3D video and sent through an auxiliary channel. The other important factor in the proposed metric is the unequal weight of color and depth sections, which can maximize the performance of the proposed metric for some specific values. Performance of the proposed metric is validated through series of subjective tests. For validations, compression and transmission artifacts are considered. The average correlation of the proposed metric and subjective quality scores is 0.82 for compressed 3D videos when color to depth importance ratio is near 0.8. This measure for transmitted 3D videos is 0.857 for the same value of color to depth importance ratio.     

加权SIFT流深度迁移的单幅图像2D转3D

2D视频转3D视频是解决3D片源不足的主要手段,而单幅图像的深度估计是其中的关键步骤.提出基于加权SIFT流深度迁移和能量模型优化的单幅图像深度提取方法.首先利用图像的全局描述符从深度图数据库中检索出近邻图像;其次通过SIFT流建立输入图像和近邻图像之间像素级稠密对应关系;再次由SIFT流误差计算迁移权重,将近邻图像对应像素点的深度乘以权重后迁移到输入图像上;然后利用均值滤波对迁移后的近邻图像深度进行融合;最后建立深度图优化能量模型,在尽量接近迁移后近邻图像深度的前提下,平滑梯度较小区域的深度.实验结果表明,该方法降低了估计深度图的平均相对误差,增强了深度图的均匀性.     

Depth image enlargement using an evolutionary approach

Accurate depth map at high resolution is required in many 3D video display concepts. Given a low-resolution depth map, this paper studies how to enhance its resolution with a registered high-resolution color image. The idea of the proposed approach is that pixels with similar color values and small distances should have similar depth values. Therefore, the known depth values in input depth map can be propagated to estimate the unknown depth values of their neighboring pixels with similar color values and small distances in high-resolution depth map. Different from conventional approaches, the proposed approach utilizes the ant colony optimization (ACO) technique to dispatch artificial ants moving on a coupled graph, which consists of a depth map and a color image. Then these artificial ants propagate the known depth information from the observed low-resolution depth map to its up-sampled counterpart. Experimental results show that the proposed approach achieves high-resolution depth map with more desirable quality than that of several conventional approaches.     

一种基于最小可察觉失真的立体图像质量客观评价方法

立体图像质量是评价立体视频系统性能的有效途径,而如何利用人类视觉特性对立体图像质量进行有效的评价是目前的研究难点。本文通过分析最小可察觉失真(JND,just noticeable distortion)视觉感知模型,并结合反映图像结构信息的奇异值矢量,提出了一种基于JND的立体图像质量客观评价方法。评价方法由图像质量评价和深度感知评价两部分组成,首先提取反映图像质量和深度感知的特征信息作为立体图像特征信息,然后根据立体图像的不同失真类型情况对其特征进行融合,通过支持向量回归(SVR,support vector Regression)预测得出立体图像质量的客观评价值。实验结果表明,采用本文提出的客观评价方法对立体数据测试库进行评价,在不同失真类型或混合失真评价结果中,Pearson线性相关系数(CC)值均在0.94以上,Spearman等级相关系数(SROCC)值均在0.92以上,符合人眼视觉特性,能够很好地预测人眼对立体图像的主观感知。     

利用人眼视觉特性的图像结构差异性杂波度量

针对光电图像杂波效应对目标获取性能的影响,提出一种基于人眼视觉特性的图像结构差异性杂波度量。依据人眼对图像结构的高度自适应性以及图像质量评估领域广泛认可的图像结构的定义方法,计算目标和杂波的结构相似性;并根据人眼的视觉显著性原理,给出两种适用于结构相似性的显著性加权信息度量。采用外场实验对该杂波度量的合理性进行验证,得出基于该杂波度量的目标预测性能与外场目标获取性能,在均方根误差、相关性方面都优于现有的杂波度量的预测性能。     

Reduced Reference Quality Metric for Synthesized Virtual Views in 3DTV

Multi‐view video plus depth (MVD) has been widely used owing to its effectiveness in three‐dimensional data representation. Using MVD, color videos with only a limited number of real viewpoints are compressed and transmitted along with captured or estimated depth videos. Because the synthesized views are generated from decoded real views, their original reference views do not exist at either the transmitter or receiver. Therefore, it is challenging to define an efficient metric to evaluate the quality of synthesized images. We propose a novel metric—the reduced‐reference quality metric. First, the effects of depth distortion on the quality of synthesized images are analyzed. We then employ the high correlation between the local depth distortions and local color characteristics of the decoded depth and color images, respectively, to achieve an efficient depth quality metric for each real view. Finally, the objective quality metric of the synthesized views is obtained by combining all the depth quality metrics obtained from the decoded real views. The experimental results show that the proposed quality metric correlates very well with full reference image and video quality metrics.     

基于差值扩展和余数调整的可逆数据隐藏

将Tian差值扩展技术应用于彩色图像,提出一种基于预测误差差值扩展(PEDE)和余数调整的彩色图像可逆数据隐藏算法。针对传统差值扩展技术存在过分修改像素灰度值、须嵌入定位图等缺点,首先利用色彩分量间的相关性减小差值,并将差值扩展量分散到2个色彩分量中;其次,由2个色彩分量中像素的预测平均值决定可用于扩展嵌入的差值,对不能用于扩展嵌入的差值则用可逆对比图RCM变换嵌入数据,无需保存溢出定位图;最后,对直方图平移技术进行改进,实现嵌入容量和失真控制,提取端在提取信息时可无损地恢复原始图像。实验结果表明,与其他算法相比,本文算法在同等嵌入率下可取得更好的图像质量,算法复杂度更低。     

Noise reduction method using a variance map of the phase differences in digital holographic microscopy

The phase reconstruction process in digital holographic microscopy involves a trade-off between the phase error and the high-spatial-frequency components. In this reconstruction process, if the narrow region of the sideband is windowed in the Fourier domain, the phase error from the DC component will be reduced, but the high-spatial-frequency components will be lost. However, if the wide region is windowed, the 3D profile will include the high-spatial-frequency components, but the phase error will increase. To solve this trade-off, we propose the high-variance pixel averaging method, which uses the variance map of the reconstructed depth profiles of the windowed sidebands of different sizes in the Fourier domain to classify the phase error and the high-spatial-frequency components. Our proposed method calculates the average of the high-variance pixels because they include the noise from the DC component. In addition, for the nonaveraged pixels, the reconstructed phase data created by the spatial frequency components of the widest window are used to include the high-spatial-frequency components. We explain the mathematical algorithm of our proposed method and compare it with conventional methods to verify its advantages.