面向三维高效视频编码的深度图错误隐藏

基于多视点视频序列视点内、视点间存在的相关性,并结合视点间运动矢量共享技术,该文提出一种面向3维高效视频编码中深度序列传输丢包的错误隐藏算法。首先,根据3D高效视频编码(3D-HEVC)的分层B帧预测(HBP)结构和深度图纹理特征,将深度图丢失块分成运动块和静止块;然后,对于受损运动块,使用结合纹理结构的外边界匹配准则来选择相对最优的运动/视差矢量进行基于位移矢量补偿的错误掩盖,而对受损静止块采用参考帧直接拷贝进行快速错误隐藏;最后,使用参考帧拆分重组来获取新的运动/视差补偿块对修复质量较差的重建块进行质量提升。实验结果表明:相较于近年提出的对比算法,该文算法隐藏后的深度帧平均峰值信噪比(PSNR)能提升0.25～2.03 dB,结构相似度测量值(SSIM)能提升0.001～0.006,且修复区域的主观视觉质量与原始深度图更接近。     

立体视频右视点图像宏块丢失错误隐藏算法

根据宏块的运动特性,提出了一种立体视频右视 点图像宏块丢失的错误隐藏算法。根 据视点内时域相邻帧及 视点间相邻帧的相关性估计丢失宏块的运动静止特性,将丢失宏块分为运动块和静止块。对 于静止块,直接使用时 域帧拷贝的方法恢复;对于运动块,使用外边界匹配准则选择候选块并赋予相应的权重得到 加权候选块,根据最优 候选块和加权候选块的边界匹配值自适应地选择最优块或空域插值方法恢复丢失信息。实验 表明,本文算法提高了 隐藏图像的主客观质量。在不同量化参数(QP)和不同宏块丢包率(PLR) 的情况下,经本文算 法恢复后图像的峰值信噪比(PSNR)与普通JM算法相比,平均提高0. 77～3.22dB。     

一种低复杂度立体视频错误隐藏新方法

针对立体视频流传输中右视 点整帧丢失,提出 了一种低复杂度的错误隐藏算法。首先,为了高效地感知立体视频的时域质量和视点间质量 ,定义了时域相似尺度(TSM)、 视间相似尺度(ISM)的概念;将前一时刻右视点图像进行时域和视点间匹配,分别求取 其以像素为单位的TSM和 ISM映射图;然后,计算前一时刻右视点图像当前宏块的TSM和ISM值,通过比较得 到当前宏块的预测模式;最后,根据视频序列的时域一致性,将前一时刻右视点图像宏块 的预测模式作为丢失图像宏 块的预测模式,从而使用运动补偿预测(MCP)或者视差补偿预测(DCP )的方法恢复丢失信息。研究结果表明,与已有错误隐藏 算法相比,本文算法获得更好主客观视觉效果;同时与基于图像结构相似度(SSIM)的错误隐藏算法相比,在保持主观视觉质量情况下,错误隐藏时间节省20%左右。     

面向HBP编码格式的立体视频B帧整帧丢失分层错误隐藏算法

针对分层B帧(HBP)编码格式的立体视频B帧整帧丢失的问题,该文分析了双视点视频中存在的视点间运动矢量相关性,提出一种分层错误隐藏算法。该算法与当前主流的方法有两大不同:一是该算法采用分级隐藏,根据B帧的重要性等级不同采用不同的错误隐藏方法;二是该算法考虑了相邻视点序列之间的宏块运动矢量相关性。实验表明,该算法的性能优于当前常用的H.264多视点视频整帧丢失错误隐藏方法。     

分布式多视点视频编码中的立体边信息生成

新兴的分布式多视点视频编码通过在解码端使用立体边信息可充分挖掘传统单视点的时间相关性和多视点特有的视间相关性.研究了一种多视点运动预测的立体边信息生成算法,提出了近似视差估计和编码端掩模融合算法.实验证明,解码端采用近似的视差矢量,编码端传递某些先验信息,都能有效提高立体边信息精度,降低计算复杂度.     

一种基于立体视邻接帧时空相关性的最小代价函数帧估计算法

为了在立体视频序列编码中获得高的压缩率,需要对立体视频序列中一个视的序列按传统方法进行独立编码;另一个视的序列中,只对其中一些参考帧(I帧或P帧)按视差补偿预测的方法进行编码,其余帧不进行编码和传输,而在解码端用立体视帧估计的方法得到重建.本文提出了一种基于立体视中邻接帧在图像、视差场和运动矢量场之间高度相关性的方法.对于因遮挡而缺乏估计的区域,则结合了图像强度的连续性和运动,视差矢量的分布特性,构造了代价方程并估计出该部分的运动矢量及强度值.实验证明,重建出来的帧图像在视觉和信噪比意义上均具有较好的效果.     

基于视差补偿预测的立体视频图像压缩编码

本文介绍了立体视频编码方法,并对其关键技术-视差补偿预测技术进行深入研究.本文所提出的基于视差分割的视差补偿预测算法是建立在可变尺寸块匹配算法的基础上,充分利用视差信息实现对目标图像帧的有效分割,并采用相适应的视差向量编码方案.与传统算法相比,在相同预测精度下,明显降低了视差信息编码开销.     

基于多视点视频整帧丢失的自适应错误掩盖算法

为解决多视点视频压缩(MVC)导致视频整帧丢失风险增大的问题,基于MVC的特点提出了一种4方向和多参考帧的错误掩盖算法。首先,在同一视点内,将相邻帧沿其各自运动矢量外推,得到视点内的最佳运动矢量;在视点间采用同样的方法,得到最佳视差;以外推的重叠面积为判断依据,在视点内错误掩盖和视点间错误掩盖之间选择较优的掩盖方式;最后,在MVC已有块模式的基础上引入新的块组合模式,合并具有相同错误掩盖方式、运动矢量的绝对差值小于阈值的相邻块,以减小图像的块效应。实验结果表明,与已有算法相比,本文算法有效利用了MVC的内部机制,改善了重建帧的客观指标,增强了主观质量。     

一种基于场一致性的H.264整帧丢失掩盖方法

针对H.264视频序列在传输中可能发生的整帧丢失现象,提出了一种新型的整帧丢失掩盖算法,该算法可用于P帧和B帧.对于P帧,该算法需要修改解码器的解码流程,使得掩盖丢失帧时能获取后一帧的运动矢量信息.然后,利用运动矢量映射的方法获取重建丢失帧的参考运动矢量列表,采用基于场一致性的判决方式分别选取前一帧中和后一帧中的最佳运动矢量以取得最终重建矢量.对于B帧,利用后P帧的运动矢量信息,采用基于场一致性的判决方式得到最终重建矢量.实验结果说明此算法和传统的整帧掩盖算法相比能获得更高的PSNR,并且有更好的视觉显示效果.所提算法在H.264视频发生整帧丢失时有较好的掩盖效果.     

基于运动特性分析的立体视频差错掩盖

传统的视频差错掩盖技术在处理含有剧烈或复杂运动的丢失宏块时效果不理想,该文在对立体视频编码预测方式统计分析的基础上,利用丢失宏块周围正确接收块和已掩盖块的运动和视差信息,将当前丢失宏块进行判别分类,根据不同类别做不同的处理,这就对丢失宏块的恢复更具有针对性,进而提高差错掩盖的质量和效率。实验结果表明,所提方法能够改善视频剧烈或复杂运动区域的掩盖质量。     

Fuzzy Logic Based Temporal Error Concealment for H.264 Video

In this paper, a new error concealment algorithm is proposed for the H.264 standard. The algorithm consists of two processes. The first process uses a fuzzy logic method to select the size type of lost blocks. The motion vector of a lost block is calculated from the current frame, if the motion vectors of the neighboring blocks surrounding the lost block are discontinuous. Otherwise, the size type of the lost block can be determined from the preceding frame. The second process is an error concealment algorithm via a proposed adapted multiple‐reference‐frames selection for finding the lost motion vector. The adapted multiple‐reference‐frames selection is based on the motion estimation analysis of H.264 coding so that the number of searched frames can be reduced. Therefore the most accurate mode of the lost block can be determined with much less computation time in the selection of the lost motion vector. Experimental results show that the proposed algorithm achieves from 0.5 to 4.52 dB improvement when compared to the method in VM 9.0.     

Selective VS-MRF-ME and intra coding in H.264 based on spatiotemporal continuity of motion field

The new video coding standard, H.264 uses variable size motion estimation (VS-ME), multiple reference frame motion estimation (MRF-ME) and spatial-based intra prediction with selectable block size in inter frame coding. These tools have achieved significant coding efficiency compared to coding a macroblock (MB) only based on motion-compensation in regular size with single reference frame. However, these new features also give rise to an exhaustive computation in the coding procedure since there are so many combinations of coding modes and reference frames to be tried. In this paper, a fast motion estimation algorithm based on the selective VS-MRF-ME and intra prediction is proposed to reduce H.264 coding computational complexity. The basic idea of the method is to utilize the spatiotemporal property of motion field in predicting where VS-MRF-ME and intra prediction are needed, and only in these regions VS-MRF-ME and intra coding are enabled. The motion field is generated by motion vectors from 16×16 motion estimation on the nearest reference frame. Simulation results show that the proposed algorithm can save 50% computational complexity on average, with negligible loss of coding efficiency.     

一种高精度的压缩域视频目标分割算法

该文提出了一种工作于MPEG压缩域的快速视频目标分割算法。该算法以从MPEG1/2码流中部分解码提取的特征为输入,提取P帧中的运动目标。针对一般的压缩域算法目标边界精度不高的特点,算法采用I帧和P帧中每个块的直流DCT系数和3个交流DCT系数,以及运动补偿信息,重建出P帧的原图像1/16大小的子图像,采用快速平均移聚类得到具有较高边界精度的亮度一致的区域;针对运动向量的噪声容易造成错误检测的缺点,算法结合聚类分析结果和运动块的分布,采用基于马尔可夫随机场的统计标号方法对目标和背景区域进行分类,得到每个P帧的目标掩模。该算法可以得到44子块的边界精度,对于CIF格式的码流,在Pentium IV 2GHz平台上可以达到每秒40帧的处理速度。     

Side information refinement using motion estimation in dc domain for transform-based distributed video coding

The key to good decoding performance in distributed video coding systems lies in the efficient prediction of frames using side information. Previously, this process has been performed without considering the progressive nature of the synthesis of the final frame. Observing the fact that incorrectly predicted areas of the current frame can be detected at different levels of final frame synthesis, an algorithm is proposed for refinement of side information synthesis based on an additional stage of motion estimation. The additional stage is performed on correctly decoded DC frames and is used to significantly improve the motion prediction of the final resolution frames, which leads to enhanced performance of the overall system.     

Two-dimensional mesh-based mosaic representation for manipulationof video objects with occlusion

We present a two-dimensional (2-D) mesh-based mosaic representation, consisting of an object mesh and a mosaic mesh for each frame and a final mosaic image, for video objects with mildly deformable motion in the presence of self and/or object-to-object (external) occlusion. Unlike classical mosaic representations where successive frames are registered using global motion models, we map the uncovered regions in the successive frames onto the mosaic reference frame using local affine models, i.e., those of the neighboring mesh patches. The proposed method to compute this mosaic representation is tightly coupled with an occlusion adaptive 2-D mesh tracking procedure, which consist of propagating the object mesh frame to frame, and updating of both object and mosaic meshes to optimize texture mapping from the mosaic to each instance of the object. The proposed representation has been applied to video object rendering and editing, including self transfiguration, synthetic transfiguration, and 2-D augmented reality in the presence of self and/or external occlusion. We also provide an algorithm to determine the minimum number of still views needed to reconstruct a replacement mosaic which is needed for synthetic transfiguration. Experimental results are provided to demonstrate both the 2-D mesh-based mosaic synthesis and two different video object editing applications on real video sequences.     

Straight-Line Approximation for the Boundary of the Left Ventricular Chamber from a Cardiac Cineangiogram

A cardiac cineangiogram is an X-ray motion picture of the heart. Boundary detection is an essential task to compute valuable quantitative information such as the temporal volume change and the wall velocity. Previously, boundary detection methods treated frames independent of each other. Consecutive frames of a motion picture are, however, highly correlated and therefore this property can be exploited. The method described takes advantage of the known location of the boundary on previous frames. The previous boundary is divided into a set of segments along which local rectangular regions are set up on the present frame. This boundary is approximated by a set of straight lines which minimize the square error in each rectangular region with the spatial derivative as its weight. The method is a fast computing algorithm and relatively insensitive to white noise because the resultant positive and negative weights tend to cancel this effect. An experiment was carried out, and good agreement with boundaries detected by humans was obtained.     

Efficient disparity vector prediction schemes with modified P frame for 2D camera arrays

An efficient disparity estimation algorithm for multi-view video sequences, recorded by a two-dimensional camera array in which the cameras are spaced equidistantly, is presented. Because of the strong geometrical relationship among views, the disparity vectors of a certain view can for most blocks be derived from the disparity vectors of other views. A frame constructed using that idea is called a D frame in this work. Three new prediction schemes which contain D frames are proposed for encoding 5 × 3 multi-view video sequences. The schemes are applied to several multi-view image sequences taken from a camera-array and they are compared in terms of quality, bit-rate and complexity. The experimental results show that the proposed prediction schemes significantly decrease the complexity of the encoder at a very low cost of quality and/or bit-rate.     

一种整帧丢失时的视频差错掩盖算法

压缩视频对于信道误码十分敏感,可导致重组视频的质量严重下降。在低码率的情况下,当网络传输中数据包丢失时,通常对应于一整帧图像内容的丢失。为了恢复丢失帧,提出了一种针对整帧图像丢失时的基于运动向量外推法的空时域相结合的差错掩盖算法。试验结果表明,该算法不仅能够恢复整个丢失帧,而且在主观上与客观上其掩盖效果均优于传统的掩盖算法。