联合彩色信息的多视点深度视频视差矢量预测

多视点加深度(MVD)格式可以利用虚拟视点绘制技术生成任意视点的视频,能高效地向观看者提供立体感知。由于MVD中的深度视频与对应的彩色视频采集于同一场景,具有高度的相关性。因此,深度视频可以利用已编码的彩色信息辅助编码。提出一种联合彩色信息的多视点深度视频视差矢量预测方法。该方法首先获得当前编码深度块对应位置的彩色块,提取彩色块预测得到的视差矢量作为深度块的一个候选视差矢量。然后,利用深度块信息计算得到另一个候选视差矢量。最后,对两个候选视差矢量通过计算下采样均方差,选择更精确的一个作为当前深度块的视差矢量预测值。实验结果表明,所提出的方法性能优于3D-HEVC编码平台HTM。     

基于深度学习的多视点视频编码方法综述

在当今这个数据爆炸增长的时代,深度学习强大的非线性建模能力进一步提高了多视点视频编码的率失真性能.对此,介绍多视点视频的特点及应用,重点对深度学习背景下现有的多视点视频编码方法进行介绍和总结,并展望多视点视频编码未来的发展趋势,旨在进一步发挥深度学习的能力,为用户提供更佳的观看体验.     

面向六自由度视频应用的多视点视频码率分配

六自由度(Six Degree of Freedom,6DoF)视频系 统允许用户从全方位、以任意视角身 临其境地体验场景,是沉浸式视频技术的发展方向。根据6DoF视频系统中用户观看位置的变 化,对多视点彩色与深度视频的码率分配是高质量场景生成的关键。本文从虚拟视点的失真 出发,提出一种基于虚拟视点质量预测 (Quality prediction model of virtual view,QP MVV)模型的视点级码率分配方法。首先理论分析了彩色和深度视频编码失真和虚拟视点失 真的关系,然后通过实验统计分析了虚拟视点质量与彩色和深度视频编码量化参数的关系, 建立了多视点彩色和深度视频的QPMVV模型,最后推导出多视点彩色和深度视频的相关视点 的码率分配比例。实验表明,与平均分配的方法相比,本文码率分配方法能显著提升虚拟视 点的主观和客观质量。虚拟视点越偏离中心位置,质量改善越明显。     

3D-HEVC深度视频编码MERGE模式快速选择

针对3D-HEVC中多视点深度视频编码进行研究,提出了一种对深度视频参考视点的MERGE模式快速选择算法。考虑到多视点深度视频相邻视点间相关性和MVD格式中深度视频与彩色视频间相关性,在两个方向上的参考CU采用相同编码深度时,借鉴深度视频主视点PU模式信息以加快当前CU帧间模式选择速度。在深度主视点采用MERGE模式时,当前视点跳过其他的预测模式而直接采用MERGE模式。实验结果表明,在码率和图像质量基本不变的情况下,提出的算法可以有效地提高帧间模式选择速度。     

基于自适应窗口的深度视频预处理算法

在自由视点视频系统中,由深度估计算法得 到的深度视频并不精确,对虚拟视点的绘制质量和深度视频的编码效率有很大的影响。本文 提出了一种基于自适应窗口的深度视频预处理算法来提高深度视频的压缩效率和虚拟视点的 绘制质量。首先,将深度视频划分为不连续区域、连续区域中的前景区域和连续区域中的背 景区域。然后,提取出深度视频各区域中的对象边界。最后对不连续区域中的非边界区域进 行高斯滤波,降低深度值骤变对虚拟视点绘制质量的影响,对连续区域中的非边界区域采用 自适应窗口滤波平滑处理以提高压缩效率。实验结果表明,提出的算法可以使深度视频的编 码码率节省8.33~34.39%,同时可使绘制的虚拟视点质量平均提高0.21dB。和Silva的算法相比,在更具实际意义的低码率 端编码效率更显著,绘制结果中对象边界得到了更好的保护。     

3D多视点立体显示及其关键技术

作为基于 DTV/HDTV 的二维(2D)显示之后的下一代视频显示技术,三维(3D)多视点立体显示已成为国际上的研究热点之一.为建立多视点立体显示系统,阐述了相关的关键技术,包括:光场表示模型和光场获取系统、高效的与现行视频标准兼容的多视点编码和传输方法、解码端任意位置视点的高效绘制方法、3D显示技术以及多视点自由立体显示.针对上述关键技术,分析了当前国际上的发展趋势及存在的问题,同时提出了一种基于交互式自由立体显示的 3D 视频处理系统的解决方案.     

一种用于多视视频加深度联合编码的错误隐藏算法

ATM(AVC-based test model)测试模型实现了多视视频加深度(MVD)格式的联合编码,使得数据的压缩效率更高。然而,较高的压缩效率使得码流对传输错误非常敏感,极易产生错误扩散现象。针对ATM测试模型的编码顺序,提出一种用于MVD联合编码的错误隐藏算法。算法充分利用视点内、视点间及纹理视频与深度视频间的相关性,针对每个视点的不同特征提出了适应其视频特性的不同隐藏算法。实验表明,本文提出的算法可以在不增加算法复杂度的情况下,有效提高视频的主客观质量。     

结合预处理的深度视频帧内快速编码算法

多视点彩色加深度(MVD)视频是三维(3D)视频的 主流格式。在3D高效视频编码中,深度视频帧内编码 具有较高的编码复杂度;深度估计软件获取的深度视频由于不够准确会使深度图平坦 区域纹理增加, 从而进一步增加帧内编码复杂度。针对以上问题,本文提出了一种联合深度处理的深度视频 帧内低复杂度 编码算法。首先,在编码前对深度视频进行预处理,减少由于深度图不准确而出现的纹理信 息;其次,运 用反向传播神经网络(BPNN,backpropagation neural network)预测最大编码单元 (LCU,la rgest coding unit)的最大划分深度;最后联合深度视频的边缘信 息及对应的彩色LCU最大划分深度进行CU提前终止划分和快速模式选取。实验结果表明, 本文算法在保证 虚拟视点质量的前提下,BDBR下降0.33% ,深度视频编码时间平均节省50.63%。     

一种新的多视点视频编码预测结构

多视点视频编码的参考预测结构虽然具有高效的压缩效率,但是随机访问性能较差.在综合考虑了编码效率和随机访问等因素的基础上,提出了一种改进的多视点视频预测结构编码方法.首先,分析视点与视点间的相关性来调整Ⅰ视点的位置,然后在此基础上有效地增加B视点的数目,以获得较好的综合性能.实验结果表明,该方法与传统的参考预测结构编码方法相比,提高了编码效率,并且具有较好的随机访问性能.     

多视视频编码标准MVC

近年,三维电视、自由视点电视等应用得到了相关工业界与产业界的极大关注与推动。国际标准化组织ITU-T与ISO,IEC联合制定了多视视频编码标准（MVC,Multi—view Video Coding standard）,为三维电视、自由视点电视等应用提供了高效的信源编码方案。MVC标准以H．264／AVC标准补篇（Amendment）的形式出现,它继承并发展了H．264／AVC标准的高效编码技术,并后向兼容H．264／AVC标准。这确保了MVC标准技术的先进性与应用实施的可行性。本文对MVC标准的目标应用、技术需求、技术情况、性能、应用前景等方面进行介绍和分析。     

Adaptive depth truncation filter for MVC based compressed depth image

In multiview video plus depth (MVD) format, virtual views are generated from decoded texture videos with corresponding decoded depth images through depth image based rendering (DIBR). 3DV-ATM is a reference model for the H.264/AVC based multiview video coding (MVC) and aims at achieving high coding efficiency for 3D video in MVD format. Depth images are first downsampled then coded by 3DV-ATM. However, sharp object boundary characteristic of depth images does not well match with the transform coding based nature of H.264/AVC in 3DV-ATM. Depth boundaries are often blurred with ringing artifacts in the decoded depth images that result in noticeable artifacts in synthesized virtual views. This paper presents a low complexity adaptive depth truncation filter to recover the sharp object boundaries of the depth images using adaptive block repositioning and expansion for increasing the depth values refinement accuracy. This new approach is very efficient and can avoid false depth boundary refinement when block boundaries lie around the depth edge regions and ensure sufficient information within the processing block for depth layers classification. Experimental results demonstrate that the sharp depth edges can be recovered using the proposed filter and boundary artifacts in the synthesized views can be removed. The proposed method can provide improvement up to 3.25 dB in the depth map enhancement and bitrate reduction of 3.06% in the synthesized views.     

Overview of the MVC + D 3D video coding standard

3D video services are emerging in various application domains including cinema, TV broadcasting, Blu-ray discs, streaming and smartphones. A majority of the 3D video content in market is still based on stereo video, which is typically coded with the multiview video coding (MVC) extension of the Advanced Video Coding (H.264/AVC) standard or as frame-compatible stereoscopic video. However, the 3D video technologies face challenges as well as opportunities to support more demanding application scenarios, such as immersive 3D telepresence with numerous views and 3D perception adaptation for heterogeneous 3D devices and/or user preferences. The Multiview Video plus Depth (MVD) format enables depth-image-based rendering (DIBR) of additional viewpoints in the decoding side and hence helps in such advanced application scenarios. This paper reviews the MVC + D standard, which specifies an MVC-compatible MVD coding format.     

Recent MPEG Standardization Activities on 3D Video Coding

The Moving Picture Experts Group(MPEG) has been developing a 3D video(3DV) coding standard for depth-based 3DV data representations,especially for multiview video plus depth(MVD) format.With MVD,depth-image-based rendering(DIBR) is used to synthesize virtual views that are based on a few transmitted pairs of texture and depth data.In this paper,we discuss ongoing 3DV standardization and summarize coding tools proposed in the responses to MPEG’s call for proposals on 3DV coding.     

新一代基于HEVC的3D视频编码技术

HEVC标准出台后,新一代基于HEVC的多视点加深度编码也将正式推出。基于HEVC的3D视频编码作为HEVC标准的扩展部分,主要面向立体电视和自由立体视频。从该编码方式的基本结构出发,较全面地介绍了视频编码方式、深度图编码方式和对深度图的编码控制三个方面的关键技术,包括视点间运动预测、深度图建模模式和视点合成优化等技术。     

用于多视视频加深度的错误隐藏算法

提出了一种针对多视视频加深度(MVD)的错误隐藏( EC)算法。算法充分利用MVD特有的深度 信息及当前丢失宏块周围正确解码的宏块信息,将丢失宏块分为3类不同属性的宏块。针对3类宏块的特 点,分别提出了基于候选运动矢量修正(CMVR)、基于深度的外边界匹配(DOBMA)以及自适应 权 值的EC(AWEC)等模式。实验表明,本文提出的算法在保证相同的视频主客观质量情况下, 能够快速有效地实现EC。     

一种基于立体视觉显著性的多视点视频比特分配方法

针对多视点立体视频压缩编码,提出了一种基于立 体视觉显著性的比 特分配方法。研究综合利用多视点立体视频数据中场景的运动、深度以及深度边缘信息提取 人眼感兴趣区 域(ROI)的方法;然后根据ROI的划分结果优化区域比特分配。实验结果表 明,本文提出的算法能有效提 高ROI区域的编码性能,同时整体视频的率失真性能有一定程度的提高。     

Reduced Reference Quality Metric for Synthesized Virtual Views in 3DTV

Multi‐view video plus depth (MVD) has been widely used owing to its effectiveness in three‐dimensional data representation. Using MVD, color videos with only a limited number of real viewpoints are compressed and transmitted along with captured or estimated depth videos. Because the synthesized views are generated from decoded real views, their original reference views do not exist at either the transmitter or receiver. Therefore, it is challenging to define an efficient metric to evaluate the quality of synthesized images. We propose a novel metric—the reduced‐reference quality metric. First, the effects of depth distortion on the quality of synthesized images are analyzed. We then employ the high correlation between the local depth distortions and local color characteristics of the decoded depth and color images, respectively, to achieve an efficient depth quality metric for each real view. Finally, the objective quality metric of the synthesized views is obtained by combining all the depth quality metrics obtained from the decoded real views. The experimental results show that the proposed quality metric correlates very well with full reference image and video quality metrics.     

Fast depth map mode decision based on depth–texture correlation and edge classification for 3D-HEVC

The 3D extension of High Efficiency Video Coding (3D-HEVC) has been adopted as the emerging 3D video coding standard to support the multi-view video plus depth map (MVD) compression. In the joint model of 3D-HEVC design, the exhaustive mode decision is required to be checked all the possible prediction modes and coding levels to find the one with least rate distortion cost in depth map coding. Furthermore, new coding tools (such as depth-modeling mode (DMM) and segment-wise depth coding (SDC)) are exploited for the characteristics of depth map to improve the coding efficiency. These achieve the highest possible coding efficiency to code depth map, but also bring a significant computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast depth map mode decision algorithm for 3D-HEVC by jointly using the correlation of depth map-texture video and the edge information of depth map. Since the depth map and texture video represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes and coding levels in depth map coding. Therefore, we can skip some specific prediction modes and depth coding levels rarely used in corresponding texture video. Meanwhile, the depth map is mainly characterized by sharp object edges and large areas of nearly constant regions. By fully exploiting these characteristics, we can skip some prediction modes which are rarely used in homogeneity regions based on the edge classification. Experimental results show that the proposed algorithm achieves considerable encoding time saving while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.