基于中心凹恰可觉察失真模型的多视点深度视频编码方法

针对深度视频压缩中存在的大量视觉冗余,提出了一种基于中心凹恰可觉察失真(FJND)模型的深度视频编码方法。首先通过左右通道的彩色和深度视频绘制虚拟视点图像,并利用FJND模型得到虚拟视点图像的FJND,然后根据深度视频中几何偏移和深度值失真之间的关系确定深度视频左通道的可允许失真,将深度视频左通道分区域采用自适应量化参数进行编码,并对右通道的深度视频根据与左通道量化参数的关系进行编码。实验结果表明,本文方法在相同码率下,虚拟视点图像质量平均提高0.48dB;在相同虚拟视点图像的绘制质量下,深度视频编码码率平均减少26%。     

基于多视点视频特征的3D-HEVC深度视频信息隐藏算法

为了更安全地传输秘密信息和保护3D视频,提 出基于多视点视频特征的3D-HEVC深度视频信息隐 藏算法。首先结合彩色视频的纹理特征,考虑深度视频不同区域对绘制视点的质量和编码效 率的影响,对 深度视频进行区域分割。然后针对不同的区域,采用不同的方式调制最大编码单元(LCU) 的QP值嵌入 秘密信息。最后,用修改后的QP值进行编码压缩,传输视频信息。实验仿真结果表明,相 比于原始HTM13.0, 本算法编码重建深度视频绘制视点质量的PSNR平均下降0.0015dB, 码率平均增加0.035%。本文算法 能较好地保证绘制视点的质量,对视频流的码率影响较小,且能实现秘密信息的盲提取。     

联合彩色信息的多视点深度视频视差矢量预测

多视点加深度(MVD)格式可以利用虚拟视点绘制技术生成任意视点的视频,能高效地向观看者提供立体感知。由于MVD中的深度视频与对应的彩色视频采集于同一场景,具有高度的相关性。因此,深度视频可以利用已编码的彩色信息辅助编码。提出一种联合彩色信息的多视点深度视频视差矢量预测方法。该方法首先获得当前编码深度块对应位置的彩色块,提取彩色块预测得到的视差矢量作为深度块的一个候选视差矢量。然后,利用深度块信息计算得到另一个候选视差矢量。最后,对两个候选视差矢量通过计算下采样均方差,选择更精确的一个作为当前深度块的视差矢量预测值。实验结果表明,所提出的方法性能优于3D-HEVC编码平台HTM。     

结合预处理的深度视频帧内快速编码算法

多视点彩色加深度(MVD)视频是三维(3D)视频的 主流格式。在3D高效视频编码中,深度视频帧内编码 具有较高的编码复杂度;深度估计软件获取的深度视频由于不够准确会使深度图平坦 区域纹理增加, 从而进一步增加帧内编码复杂度。针对以上问题,本文提出了一种联合深度处理的深度视频 帧内低复杂度 编码算法。首先,在编码前对深度视频进行预处理,减少由于深度图不准确而出现的纹理信 息;其次,运 用反向传播神经网络(BPNN,backpropagation neural network)预测最大编码单元 (LCU,la rgest coding unit)的最大划分深度;最后联合深度视频的边缘信 息及对应的彩色LCU最大划分深度进行CU提前终止划分和快速模式选取。实验结果表明, 本文算法在保证 虚拟视点质量的前提下,BDBR下降0.33% ,深度视频编码时间平均节省50.63%。     

基于Kinect的实时深度提取与多视绘制算法

提出了一种基于Kinect的实时深度提取算法和单纹理+深度的多视绘制方法。在采集端,使用Kinect提取场景纹理和深度,并针对Kinect输出深度图的空洞提出一种快速修复算法。在显示端,针对单纹理+深度的基于深度图像的绘制(DIBR,depth image based rendering)绘制产生的大空洞,采用一种基于背景估计和前景分割的绘制方法。实验结果表明,本文方法可实时提取质量良好的深度图,并有效修复了DIBR绘制过程中产生的大空洞,得到质量较好的多路虚拟视点图像。以所提出的深度获取和绘制算法为核心,实现了一种基于深度的立体视频系统,最终的虚拟视点交织立体显示的立体效果良好,进一步验证了本文算法的有效性。本文系统可用于实景的多视点立体视频录制与播放。     

一种用于多视视频加深度联合编码的错误隐藏算法

ATM(AVC-based test model)测试模型实现了多视视频加深度(MVD)格式的联合编码,使得数据的压缩效率更高。然而,较高的压缩效率使得码流对传输错误非常敏感,极易产生错误扩散现象。针对ATM测试模型的编码顺序,提出一种用于MVD联合编码的错误隐藏算法。算法充分利用视点内、视点间及纹理视频与深度视频间的相关性,针对每个视点的不同特征提出了适应其视频特性的不同隐藏算法。实验表明,本文提出的算法可以在不增加算法复杂度的情况下,有效提高视频的主客观质量。     

基于自适应窗口的深度视频预处理算法

在自由视点视频系统中,由深度估计算法得 到的深度视频并不精确,对虚拟视点的绘制质量和深度视频的编码效率有很大的影响。本文 提出了一种基于自适应窗口的深度视频预处理算法来提高深度视频的压缩效率和虚拟视点的 绘制质量。首先,将深度视频划分为不连续区域、连续区域中的前景区域和连续区域中的背 景区域。然后,提取出深度视频各区域中的对象边界。最后对不连续区域中的非边界区域进 行高斯滤波,降低深度值骤变对虚拟视点绘制质量的影响,对连续区域中的非边界区域采用 自适应窗口滤波平滑处理以提高压缩效率。实验结果表明,提出的算法可以使深度视频的编 码码率节省8.33~34.39%,同时可使绘制的虚拟视点质量平均提高0.21dB。和Silva的算法相比,在更具实际意义的低码率 端编码效率更显著,绘制结果中对象边界得到了更好的保护。     

面向立体视频传输的右视点图像错误隐藏

利用立体视频序列视点间相关性及单视点内相关性,提出了一种面向立体视频传输的错误隐藏算法.从基于H.264/AVC立体视频编码结构出发,推断出受损块的参考模式;然后基于出错块的内容特征,根据块视差活力度(TDA)或块运动活力度(TMA)的大小,内容自适应地选择恰当的视点间及时域错误隐藏方法对受损块进行错误掩盖.实验结果表...     

面向六自由度视频应用的多视点视频码率分配

六自由度(Six Degree of Freedom,6DoF)视频系 统允许用户从全方位、以任意视角身 临其境地体验场景,是沉浸式视频技术的发展方向。根据6DoF视频系统中用户观看位置的变 化,对多视点彩色与深度视频的码率分配是高质量场景生成的关键。本文从虚拟视点的失真 出发,提出一种基于虚拟视点质量预测 (Quality prediction model of virtual view,QP MVV)模型的视点级码率分配方法。首先理论分析了彩色和深度视频编码失真和虚拟视点失 真的关系,然后通过实验统计分析了虚拟视点质量与彩色和深度视频编码量化参数的关系, 建立了多视点彩色和深度视频的QPMVV模型,最后推导出多视点彩色和深度视频的相关视点 的码率分配比例。实验表明,与平均分配的方法相比,本文码率分配方法能显著提升虚拟视 点的主观和客观质量。虚拟视点越偏离中心位置,质量改善越明显。     

基于帧内帧间联合预测的深度视频编码方法

在基于多视点加深度(MVD)格式的视频编码方案中,深度视频的编码性能直接影响最终绘制的虚拟视点的质量。对于具有边界的深度块而言,传统的帧内预测和帧间预测模式仍存在一定的提升空间。因此,文中提出一种基于帧内帧间联合预测的深度视频编码方法。该方法首先获取当前深度块的最优帧内预测模式和最优帧间预测模式。然后,将这两种模式应用于边界深度块的不同区域。最后,自适应地调整预测结果的加权系数,实现联合预测。实验结果表明,相对于3D-HEVC平台的传统预测模式,本方法实现了更好的编码性能。     

一种基于MPEG-4的感兴趣区域视频编码新方法

本文提出了一种基于MPEG-4的感兴趣区域视频编码新方法.通过对感兴趣区边沿宏块采用强制帧内编码及宏块内预测编码时参考位置的自适应选择,该方法有效地抑制了数据噪声的扩散和传播.通过采用自适应宏块大小的运动估计/补偿算法,提高了感兴趣区特别是图像运动复杂的感兴趣区的编码效率及质量.在码率分配部分,本文方法通过计算不同区域的图像复杂性和能量,依据用户可设定的感兴趣权重因子不等重地分配可用码率资源.实验证明,本文方法较大程度改善了感兴趣区视频编码的压缩效率,提高了码率分配地灵活性和有效性.     

Frame-level bit allocation for hierarchical coding of H.265/HEVC considering dependent rate-distortion characteristics

Rate control plays an important role in video coding for rectifying the coded bit stream to meet the transmission bandwidth, which is related to two parts of bit allocation and quantization parameter acquirement. Considering that the bit allocation is a critical step due to its effect on the subsequent coding operation and rate-distortion performance, therefore in this paper an accurate H.265/HEVC frame-level bit allocation algorithm is proposed for higher coding efficiency. Concretely the feature regarding the inter-frame dependency is firstly analyzed based on the theoretical derivation. Then the hierarchically dependent characteristics are extracted and utilized to establish the bit allocation model. Experimental results show that the proposed method can guarantee a good consistency between the target bits and coded bits, where a PSNR improvement of about 0. 131 dB can be observed compared with the original H.265/HEVC bit allocation while keeping the same bit rate.     

Convex Optimization-Based Bit Allocation for Video Coding

This paper presents a new convex optimization-based bit allocation framework for video coding. Motivated by the statistical analysis of the relationship between the rate and the corresponding distortion in typical video sequences, an improved empirical rate-distortion model with more flexibility in explaining the experimental observations is proposed. The convexity and monotonicity properties of the proposed model allow us to formulate the bit allocation problem as a convex programming problem, which can be solved using interior-point methods. Coupled with the region of interest (ROI) functionality and the technique of MB classification, the proposed bit allocation scheme is then applied to the problem of frame-level bit allocation for various video coding standards employing motion compensated hybrid DCT/DPCM technique. Similar bit allocation scheme is also developed for MPEG-4 video object coding at the object level. The relevant model parameters are determined using previously encoded data by means of linear regression. Simulation results show that the proposed algorithms can achieve considerably better PSNR performance than the conventional approaches for the tested video sequences, at the expenses of slight increase in coding complexity.     

多视点纹理加深度编码的联合码率控制方法

码率控制技术是多视点视频编码和传输中一个关键的问题。为了提高三维(3D)视频的整体显示质量,包括虚拟视点质量和编码视点质量,提出一种多视点纹理加深度编码的联合码率控制方法。该算法研究了纹理和深度的关系,采用基于模型方法确定最优的纹理和深度之间的码率比例。根据各个视点编码结果的统计规律,不同的视频序列采用不同的视点间比特分配比例。实验结果表明,与目前流行的多视点码率控制算法相比,该算法在计算复杂度基本保持不变的情况下,平均码率控制误差在0.6%以内,客观质量PSNR最高可提高0.65 dB。     

Extraction technique of region of interest from stereoscopic video

A feature fusion approach is presented to extract the region of interest (ROI) from the stereoscopic video. [0]Based on human vision system (HVS), the depth feature, the color feature and the motion feature are chosen as vision features. [0]The algorithm is shown as follows. Firstly, color saliency is calculated on superpixel scale. Color space distribution of the superpixel and the color difference between the superpixel and background pixel are used to describe color saliency and color salient region is detected. Then, the classic visual background extractor (Vibe) algorithm is improved from the update interval and update region of background model. The update interval is adjusted according to the image content. The update region is determined through non-obvious movement region and background point detection. So the motion region of stereoscopic video is extracted using improved Vibe algorithm. The depth salient region is detected by selecting the region with the highest gray value. Finally, three regions are fused into final ROI. Experiment results show that the proposed method can extract ROI from stereoscopic video effectively. In order to further verify the proposed method, stereoscopic video coding application is also carried out on the joint model (JM) encoder with different bit allocation in ROI and the background region.     

Depth perceptual region-of-interest based multiview video coding

MultiView Video (MVV) has attracted considerable attention recently since it is capable of providing users with three-dimensional perception and interactive functionalities. However, these MVV data require large mount of storage and bandwidth in network transmission. In this paper, we present a novel Depth Perceptual Region-Of-Interest (DP-ROI) based Multiview Video Coding (RMVC) scheme to extensively improve data compression efficiency by exploiting redundancies in depth perception. Firstly, we define DP-ROI according to the three-dimensional depth sensation of human visual system. Then, a framework of RMVC is developed to improve compression efficiency by properly segmenting the MVV into different macroblock wise DP-ROIs and encoding them separately. And then, we propose three fast depth based DP-ROI extraction and tracking algorithms by jointly using motion, texture, depth as well as previous extracted DP-ROIs. Finally, on the basis of the extracted DP-ROI, bit allocation optimization model is proposed to allocate more bits on DP-ROIs for high image quality and fewer bits on background regions for high compression ratio. Experimental results show that the presented RMVC scheme achieves significant coding gains at high rate while comparing with original joint multiview video model. To be specific, up to 14.22–23.32% bit-rate are saved while 0.16–0.68 dB coding gains are achieved in DP-ROIs at the cost of the image quality degradation in background.     

An adaptive quantization algorithm without side information for depth map coding

This paper presents a novel block-adaptive quantization scheme for efficient bit allocation without side information in depth map coding. Since the type of distortion in a depth map causes different effects in terms of the visual artifacts in a synthesized view, the proposed method adaptively assigns the number of bits according to the characteristics of the corresponding texture block. I have studied the details of the depth map and its rendered view distortion, modeled these analytically, and then proposed a new rate and distortion model for depth map coding. Finally, I derived a simple closed-form solution based on my proposed rate and distortion model, which determines the block-adaptive quantization parameter without any side information. Experimental results show that the proposed scheme can achieve coding gains of more than 0.6% and 1.4% for quarter- and full-resolution depth maps, respectively, in a multi-view-plus-depth 3D system.     

An attention emphasized bit arrangement in 3-D SPIHT video coding for human vision

In recent years, Set Partitioning in Hierarchical Trees (SPIHT) algorithm has performed efficiently on image and video coding. However, in low bit rate situations, on the whole the videos usually have relatively low visual quality while attention is paid on the videos. To improve the problem, this paper proposes Attention Emphasis bit arrangement in 3-D SPIHT video coding algorithm (AE-SPIHT) to rearrange bits efficiently from eye-insensitive regions to eye-sensitive regions, so that the whole video can obtain a better visual condition. AE-SPIHT properly exploits a bit-arrangement method rearranging bits foveately from eye-insensitive regions, according to texture content and bit-rate, to eye-sensitive ones. Experiments show that AE-SPIHT algorithm improves the visual quality of attention regions in conventional 3-D SPIHT in most cases. Compared with other bit arrangement methods, the proposed method also achieves a better visual condition in arranging the bits.     

Dynamic computational complexity and bit allocation for optimizing H.264/AVC video compression

In this work, we present a novel approach for optimizing H.264/AVC video compression by dynamically allocating computational complexity (such as a number of CPU clocks) and bits for encoding each coding element (basic unit) within a video sequence, according to its predicted MAD (mean absolute difference). Our approach is based on a computational complexity–rate–distortion (C–R–D) analysis, which adds a complexity dimension to the conventional rate–distortion (R–D) analysis. Both theoretically and experimentally, we prove that by implementing the proposed approach for the dynamic allocation better results are achieved. We also prove that the optimal computational complexity allocation along with optimal bit allocation is better than the constant computational complexity allocation along with optimal bit allocation. In addition, we present a method and system for implementing the proposed approach, and for controlling computational complexity and bit allocation in real-time and off-line video coding. We divide each frame into one or more basic units, wherein each basic unit consists of at least one macroblock (MB), whose contents are related to a number of coding modes. We determine how much computational complexity and bits should be allocated for encoding each basic unit, and then allocate a corresponding group of coding modes and a quantization step-size, according to the estimated distortion (calculated by a linear regression model) of each basic unit and according to the remaining computational complexity and bits for encoding remaining basic units. For allocating the corresponding group of coding modes and the quantization step-size, we develop computational complexity–complexity step–rate (C–I–R) and rate–quantization step-size–computational complexity (R–Q–C) models.     

Region-of-interest based rate control for UAV video coding

To meet the requirement of high-quality transmission of videos captured by unmanned aerial vehicles (UAV) with low bandwidth, a novel rate control (RC) scheme based on region-of-interest (ROI) is proposed. First, the ROI information is sent to the encoder with the latest high efficient video coding (HEVC) standard to generate an ROI map. Then, by using the ROI map, bit allocation methods are developed at frame level and large coding unit (LCU) level, to avoid inaccurate bit allocation produced by camera movement. At last, by using a better robustness R-λ model, the quantization parameter (QP) for each LCU is calculated. The experimental results show that the proposed RC method can get a lower bitrate error and a higher quality for reconstructed video by choosing appropriate pixel weight on the HEVC platform.