面向立体视频传输的右视点图像错误隐藏

利用立体视频序列视点间相关性及单视点内相关性,提出了一种面向立体视频传输的错误隐藏算法.从基于H.264/AVC立体视频编码结构出发,推断出受损块的参考模式;然后基于出错块的内容特征,根据块视差活力度(TDA)或块运动活力度(TMA)的大小,内容自适应地选择恰当的视点间及时域错误隐藏方法对受损块进行错误掩盖.实验结果表...     

A joint encoder–decoder error control framework for stereoscopic video coding

Stereoscopic video coding (SSVC) plays an important role in various 3D video applications. In SSVC, robust stereoscopic video transmission over error-prone networks is still a challenge problem to be solved. In this paper, we propose a joint encoder–decoder error control framework for SSVC, where error-resilient source coding, transmission network conditions, and error concealment scheme are jointly considered to achieve better error robustness performance. The proposed joint encoder–decoder error control framework includes two parts: an error concealment algorithm at the decoder side and a rate–distortion optimized error resilience algorithm at the encoder side. For error concealment at the decoder side, an overlapped block motion and disparity compensation based error concealment scheme is proposed to adaptively utilize inter-view correlations and temporal correlations. For error resilience at the encoder side, first, the inter-view refreshment is proposed for SSVC to suppress error propagations. Then, an end-to-end distortion model for SSVC is derived, which jointly considers the transmission network conditions, inter-view refreshment, and error concealment tools at the decoder side. Finally, based on the derived end-to-end distortion model, the rate–distortion optimized error resilience algorithm is presented to adaptively select inter-view, inter- or intra-coding for SSVC. The experimental results show that the proposed joint encoder–decoder error control framework has superior error robustness performance for stereoscopic video transmission over error-prone networks.     

Multi-view video coding with view interpolation prediction for 2D camera arrays

An efficient compression algorithm for multi-view video sequences, which are captured by two-dimensional (2D) camera arrays, is proposed in this work. First, we propose a novel prediction structure, called three-dimensional hierarchical B prediction (3DHBP), which can efficiently reduce horizontal inter-view redundancies, vertical inter-view redundancies, and temporal redundancies in multi-view videos. Second, we develop a view interpolation scheme based on the bilateral disparity estimation. The interpolation scheme yields high quality view frames by adapting disparity estimation and compensation procedures using the information in neighboring frames. Simulation results demonstrate that the proposed multi-view video coding algorithm provides significantly better rate–distortion (R–D) performance than the conventional algorithm, by employing the 3DHBP structure and using interpolated view frames as additional reference frames.     

一种低复杂度立体视频错误隐藏新方法

针对立体视频流传输中右视 点整帧丢失,提出 了一种低复杂度的错误隐藏算法。首先,为了高效地感知立体视频的时域质量和视点间质量 ,定义了时域相似尺度(TSM)、 视间相似尺度(ISM)的概念;将前一时刻右视点图像进行时域和视点间匹配,分别求取 其以像素为单位的TSM和 ISM映射图;然后,计算前一时刻右视点图像当前宏块的TSM和ISM值,通过比较得 到当前宏块的预测模式;最后,根据视频序列的时域一致性,将前一时刻右视点图像宏块 的预测模式作为丢失图像宏 块的预测模式,从而使用运动补偿预测(MCP)或者视差补偿预测(DCP )的方法恢复丢失信息。研究结果表明,与已有错误隐藏 算法相比,本文算法获得更好主客观视觉效果;同时与基于图像结构相似度(SSIM)的错误隐藏算法相比,在保持主观视觉质量情况下,错误隐藏时间节省20%左右。     

用于多视视频加深度的错误隐藏算法

提出了一种针对多视视频加深度(MVD)的错误隐藏( EC)算法。算法充分利用MVD特有的深度 信息及当前丢失宏块周围正确解码的宏块信息,将丢失宏块分为3类不同属性的宏块。针对3类宏块的特 点,分别提出了基于候选运动矢量修正(CMVR)、基于深度的外边界匹配(DOBMA)以及自适应 权 值的EC(AWEC)等模式。实验表明,本文提出的算法在保证相同的视频主客观质量情况下, 能够快速有效地实现EC。     

Spatiotemporal error concealment with optimized mode selection and application to H.264

In this paper we propose an error concealment algorithm for video transmission over lossy packet networks. The proposed technique is based on temporal and spatial interpolation. A sophisticated mode selection algorithm decides whether to employ the temporal or the spatial part, or a combination thereof, to estimate a missing macroblock; the selection does not rely on knowledge of the original coding modes. The resulting error concealment algorithm is designed so as to optimize both PSNR and visual quality of the restored video sequence, and employs directional interpolation and texture analysis/synthesis. The technique has been applied to H.264 coded video, providing satisfactory results on a number of test sequences.     

TZSearch算法分析和性能改进方法

研究和改进多视点视频平台（JMVC,Joint Multi-view Video Coding）中的TZSearch快速搜索算法。首先介绍TZSearch算法原理、宏块最优匹配点分布特点及阈值设置方法,然后根据多视点视频帧间和视间的不同特性,结合钻石搜索算法和光栅搜索算法的特点及TZSearch中阈值设置规律,提出了两种改进的TZSearch快速搜索算法。实验结果表明,针对不同特性的序列进行测试,在保证重建视频质量与编码比特率的前提下,相比原算法,两种改进后的算法效果明显,总编码时间可减少30%-50%。     

An efficient rate–distortion optimization method for dependent view in MV-HEVC based on inter-view dependency

Rate–distortion optimization (RDO) is utilized to select the optimal coding parameters in multi-view video coding (MVC), which employs a Lagrange multiplier to balance the relationship between the distortion and the bitrate. In this paper, an efficient RDO method for the dependent view (DV) in multi-view video (MVV) is proposed based on inter-view dependency. First of all, by investigating the sources of the distortion in the DV, a new distortion model for the DV is established. In addition, based on the proposed distortion model, an efficient Lagrangian multiplier decision for B frame is proposed by considering the inter-view dependency. Finally, the optimized Lagrangian multiplier for P frame is designed using the scaling factor which is deduced to have a linear relationship with the disparity between I frame and P frame. Experiment results demonstrate that compared with the original HTM-16.0 encoder, the proposed overall method reduces 12.19% BD-rate for the DV on average, bringing 0.40 dB BD-PSNR gain.     

Improved inter-view correlations for low complexity MV-HEVC

The more advanced multi-view extension, MV-HEVC, effectively exploits visual similarities between multi-view videos and enables high compression efficiency. Each view in the multi-view sequence depends on the captured scene, the distance between cameras and recording angles. Increasing the distance between dependent viewpoints generates an inter-view disparity. This impacts the inter-view similarities, affects the disparity estimation and further increases the computational complexity of the MV-HEVC encoder. In this paper, an efficient earlier disparity estimation is proposed for low complexity MV-HEVC. This algorithm is based on reducing the complexity of disparity estimation by eliminating the inter-view offset. Moreover, the inter-view similarities are controlled by considering the reliability of each coding unit size in the search range. This reliability is estimated by reducing the number of searching points within a new limited window. For reliable motion estimation, we further proposed an earlier decision of coding units splitting in the dependent views according to those in the reference views. Experimental results show that the proposed algorithm can achieve an average encoding time saving of 20.37%–40,61% with marginal performance degradation.     

A motion vector prediction method for multi-view video coding

This paper proposes a new motion vector (MV) prediction method in multi-view video coding (MVC). In order to exploit the information in adjacent views, inter-view MVs as well as temporal MVs are used in conventional MVC. Since the inter-view MVs are usually uncorrelated with the temporal MVs and most neighboring partitions have temporal MVs only, the conventional DPCM coding gain of inter-view MV is very low and thus the inter-view MVs are seldom selected. In order to increase the probability of inter-view MV selection, we define a virtual inter-view MV which can be generated from temporal MVs. Then, an inter-view MV is predicted using these neighboring virtual inter-view MVs, leading to less prediction error than using the temporal MVs. As a result, bit-rates are decreased by up to 9% for the view-temporal prediction structure.     

地面数字电视广播系统中的视频传输容错算法

本文提出了一套针对MPEG-2视频压缩数据传输的自适应容错机制,用于支持地面数字电视广播系统中的可靠电视传输.具体地,本文首先提出了一种基于时域隐藏和块匹配的差错隐藏算法,在保持较好隐藏效果的同时,通过有效地减小搜索窗口而显著降低了算法复杂度.同时,针对独立I帧图像还提出了独特的差错隐藏方法.将这两种差错隐藏算法与低运算复杂度的空域/时域运动检测方法相结合,又衍生出一种自适应空/时域结合差错隐藏算法.仿真结果表明,本文提出的算法即使在很高的数据差错率情况下,仍然可以实现地面数字电视广播系统中可靠而高效的视频传输.     

一种基于多视点视频的低复杂度自适应环路滤波算法

自适应环路滤波(ALF)是面向高清的多视点视频编码(MVC)中的一项新的滤波技术,虽然能提高视频压缩的主观质量和客观质量,但是其复杂度过高阻碍了其实时应用。本文提出了一种低复杂度的ALF算法,充分利用了视点间相关性和层间相关性以减少亮度和色度分量的分割次数,并且能自适应跳过大部分帧的块控制(DBC)处理过程。实验结果表明,所提出的算法能在保证视频主观质量和客观质量基本不变的情况下减少61%左右的ALF处理时间。因此,所提出的方法在编码效率和复杂度上做到了很好的折中处理,并且能很好地改善ALF的性能。     

Fast motion and disparity estimation for HEVC based 3D video coding

The emerging international standard for high efficiency video coding (HEVC) based 3D video coding (3D-HEVC) is an extension of HEVC. In the test model of 3D-HEVC, variable size motion estimation (ME) and disparity estimation (DE) are both employed to select the best coding mode for each treeblock in the encoding process. This technique achieves the highest possible coding efficiency, but it brings extremely high computational complexity which limits 3D-HEVC from practical applications. In this paper, a fast ME/DE algorithm based on inter-view and spatial correlations is proposed to reduce 3D-HEVC computational complexity. Since the multi-view videos represent the same scene with similar characteristic, there is a high correlation among the coding information from inter-view prediction. Besides, the homogeneous regions in texture video have a strong spatial correlation, and thus spatially neighboring treeblocks have similar coding information. Therefore, we can determine ME search range and skip some specific ME and DE rarely used in the previously coded view frames and spatially neighboring coding unit. Experimental results demonstrate that the proposed algorithm can significantly reduce computational complexity of 3D-HEVC encoding while maintaining almost the same rate-distortion performance.     

多视点视频编码中图像分组技术的研究

在多视点视频编码中,引入了视点间运动补偿预测技术,而在视点间预测过程中,由于视差矢量的使用,使得GoP长度对编码效率的影响与一般H.264/AVC编码有所不同。在分析了可分级B帧的预测结构对多视点视频编码性能的影响基础上,通过对不同视频序列设置不同GoP的大小,经过多次反复实验,而后对实验结果进行分析,找到了在一定数值范围内的GoP。通过该GoP得到了较好的PSNR和模值较小的MV或者DV,节约了一定的编码时间。     

基于3D视频的错误隐藏技术分析

随着3D视频产品的逐步问世,3D视频技术逐渐成为当前研究的热点。由于3D视频编码是利用多个视点的空间、时间及视点间的相关性进行压缩编码,导致当3D视频遭遇不可靠信道差错时,其受损的影响比传统的2D视频更严重。因此,研究基于3D视频的差错控制技术十分必要,针对3D视频的3种视频格式,初步探讨了目前已经出现或正在研究中的错误隐藏技术。     

一种基于率失真优化的关键参考帧选择算法

为抑制压缩视频流的传输错误扩散,提出了一种新的基于关键帧的无反馈参考帧选择算法,通过采用一种适用各种掩盖策略的端到端失真度估计方法,评价出选择不同参考帧情况下的解码器失真度期望,在率失真框架内选择最合适的参考帧,使解码端的失真度达到最小.基于H. 263+ TMN8测试模型和IP包丢失的模拟实验表明,提出的率失真优化关键参考帧算法比传统的预测编码方法,PSNR值大约提高了1~3dB.当数据丢包率严重时,优势尤为明显.     

HEVC中基于前后景区域的错误隐藏

新一代视频编码标准HEVC采用四叉树编码结构,能够高效地进行视频编码,尤其适用于高清视频。然而,网络信道的不可靠性会导致视频数据的受损或丢失,HEVC的编码特性决定了视频数据的受损或丢失可能会发生在不同尺寸编码块中。因此,在分析各种不同尺寸编码块自身特性的基础上,提出首先对视频进行前景、后景区分,其中利用OBMA算法区分出边界处与运动物体,然后对于不同区域的错误块分别进行隐藏。     

基于多方向自适应加权的空间错误隐藏算法

为了提高H.264/AVC压缩视频码流在无线信道传输过程中的抗误码性能,提出了一种基于多方向自适应加权的空间错误隐藏算法.该算法首先应用边缘算子检测受损宏块的相邻宏块信息,然后通过多方向自适应加权插值得到受损宏块像素近似值.实验表明,该算法相对传统空间错误隐藏算法,对于不同视频序列图像重构质量有很大的改善,具有较高的应用价值.     

一种多参考帧时域视频错误隐藏算法

提出一种自适应扩大搜索范围的多参考帧错误隐藏算法。该算法以视频序列的运动剧烈程度为基础,在多个参考帧中根据视频序列运动的剧烈程度的不同自适应地扩大搜索范围以寻找最优匹配块,用最优匹配块替换受损块。实验结果表明,与多帧恢复的时域错误隐藏算法相比,错误隐藏算法无论是在PSNR值上还是主观视觉效果上都有所提高。     

High‐Performance Spatial and Temporal Error‐Concealment Algorithms for Block‐Based Video Coding Techniques

A compressed video bitstream is sensitive to errors that may severely degrade the reconstructed images even when the bit error rate is small. One approach to combat the impact of such errors is the use of error concealment at the decoder without increasing the bit rate or changing the encoder. For spatial‐error concealment, we propose a method featuring edge continuity and texture preservation as well as low computation to reconstruct more visually acceptable images. Aiming at temporal error concealment, we propose a two‐step algorithm based on block matching principles in which the assumption of smooth and uniform motion for some adjacent blocks is adopted. As simulation results show, the proposed spatial and temporal methods provide better reconstruction quality for damaged images than other methods.