Efficient multi-view video coding using inter-view information

Multi-view video coding (MVC) has been extended from H.264/AVC to improve the coding efficiency of multi-view video. This paper proposes a fast mode decision algorithm which can make an early decision on the correct mode partition to solve the issue of the enormous computational complexity. The best modes of the reference views are utilized to determine the complexity of the macroblock (MB) in the current view, the mode candidates needed to be calculated can then be obtained according to the complexity. If the complexity is low or medium, the search range can be reduced. The threshold of the rate-distortion cost for the current MB is calculated using the co-located and neighboring MBs in previously coded view and is utilized as the criterion for early termination. The motion vector difference in the reference view is applied to dynamically adjust the search range in the current MB. Experimental results prove that the proposed algorithm achieves a time saving of 81.05% for a fast TZ search and 87.85% for full search, and still maintains quality performance and bitrate.     

Error concealment analysis for H.264/advanced video coding encoded video sequences

Error concealment methods have become very important in particular when transmitting video streams over error prone wireless links. Often a retransmission of corrupted sequences is not possible and thus the receiver has to make the best out of the received stream. The contributions of this article are the following: firstly, a performance comparison of various error concealment strategies (straight decoding, slice level concealment and macroblock level concealment) is presented based on the detection of errors, the exact location of which is unknown. Secondly, an analytical treatment of the slice level concealment, resulting in a precise mathematical model is provided. Finally, further improvements are proposed by subjective methods based on visual inspection and comparison of their performance by means of simulations.     

用于多视视频加深度的错误隐藏算法

提出了一种针对多视视频加深度(MVD)的错误隐藏( EC)算法。算法充分利用MVD特有的深度 信息及当前丢失宏块周围正确解码的宏块信息,将丢失宏块分为3类不同属性的宏块。针对3类宏块的特 点,分别提出了基于候选运动矢量修正(CMVR)、基于深度的外边界匹配(DOBMA)以及自适应 权 值的EC(AWEC)等模式。实验表明,本文提出的算法在保证相同的视频主客观质量情况下, 能够快速有效地实现EC。     

多视点立体视频传输系统与错误隐藏算法设计

针对多视点立体视频数据量大、网络带宽受限等问题,提出了一种分组传输多路复用的多视点立体视频实时传输系统。系统将多视点数据通过H.264/AVC压缩编码后,在两个IP链路中进行分组复用,实现了立体视频在带宽受限网络中的实时传输;同时,为了解决IP网络传输中的丢包问题,提出一种联合时域视点间预测的错误隐藏算法。最后通过实验表明,本文错误隐藏算法可以提高多视点立体视频的解码质量。     

一种用于多视视频加深度联合编码的错误隐藏算法

ATM(AVC-based test model)测试模型实现了多视视频加深度(MVD)格式的联合编码,使得数据的压缩效率更高。然而,较高的压缩效率使得码流对传输错误非常敏感,极易产生错误扩散现象。针对ATM测试模型的编码顺序,提出一种用于MVD联合编码的错误隐藏算法。算法充分利用视点内、视点间及纹理视频与深度视频间的相关性,针对每个视点的不同特征提出了适应其视频特性的不同隐藏算法。实验表明,本文提出的算法可以在不增加算法复杂度的情况下,有效提高视频的主客观质量。     

Error concealment for SNR scalable video coding

This paper proposes an efficient error concealment method for SNR scalable coded video. The algorithm adaptively selects a proper concealment candidate from the base or the enhanced pictures to conceal the artifact of a lost enhancement block. To determine the best concealment candidate, we propose a trial process in which the concealment candidates are examined based on two criteria: (1) picture continuity at the border of concealed macroblocks, and (2) to satisfy the coding distortion bound of the base layer coefficients when they are available. For the latter, requantization of the concealed picture with the base layer quantizer step size and its dequantized pixels should result in zero distortion. We have implemented the method on a proposed SNR scalable H.264 video codec and compared the decoded video quality against just copying the base layer pixels into the enhanced picture. Simulation results show that the proposed method can achieve a considerable improvement by up to 3 dB especially in situations where the enhancement layer contains a large portion of the picture information. This will make scalable video transmission more successful over unreliable channels.     

Error control and concealment for video communication: a review

The problem of error control and concealment in video communication is becoming increasingly important because of the growing interest in video delivery over unreliable channels such as wireless networks and the Internet. This paper reviews the techniques that have been developed for error control and concealment. These techniques are described in three categories according to the roles that the encoder and decoder play in the underlying approaches. Forward error concealment includes methods that add redundancy at the source end to enhance error resilience of the coded bit streams. Error concealment by postprocessing refers to operations at the decoder to recover the damaged areas based on characteristics of image and video signals. Last, interactive error concealment covers techniques that are dependent on a dialogue between the source and destination. Both current research activities and practice in international standards are covered     

Error concealment aware rate shaping for wireless video transport

Streaming of video, which is both source- and channel-coded, over wireless networks faces the challenge of time-varying packet loss rate and fluctuating bandwidth. Rate shaping (RS) has been proposed to reduce the bit-rate of a precoded video bitstream to adapt to the real-time bandwidth variation. In our earlier work, rate shaping was extended to consider not only the bandwidth but also the packet loss rate variations. Rate-distortion optimized rate adaptation is performed on the precoded video that is a scalable coded bitstream protected by forward error correction codes. In this paper, we propose an RS scheme that further takes into account the error concealment (EC) method used at the receiver. We refer to this scheme as EC aware RS (ECARS). When performing ECARS, first ECARS needs to know the benefit/gain of sending each part of the precoded video, as opposed to not sending it but reconstructing it by EC. Then given a certain packet loss probability, the expected gain can be derived and be included in the rate-distortion optimization problem formulation. Finally, ECARS performs rate-distortion optimization to adapt the rate of the precoded video. A two-stage rate-distortion optimization approach is proposed to solve the ECARS rate-distortion optimization problem. In addition to ECARS, the precoding process can be EC aware to prioritize the precoded video based on the gain. We present an example EC aware precoding process by means of macroblock prioritization. Experiment results of ECARS together with EC aware precoding are shown to have excellent performance.     

Error concealment algorithms for robust decoding of MPEG compressed video

This paper provides some research results on the topic of error resilience for robust decoding of MPEG (Moving Picture Experts Group) compressed video. It introduces and characterizes the performance of a general class of error concealment algorithms. Such receiver-based error concealment techniques are essential for many practical video transmission scenarios such as terrestrial HDTV broadcasting, packet network based teleconferencing/multimedia, and digital SDTV/HDTV delivery via ATM (asynchronous transfer mode). Error concealment is intended to ameliorate the impact of channel impairments (i.e., bit-errors in noisy channels, or cell-loss in ATM networks) by utilizing available picture redundancy to provide a subjectively acceptable rendition of affected picture regions. The concealment process must be supported by an appropriate transport format which helps to identify image pixel regions which correspond to lost or damaged data. Once the image regions (i.e., macroblocks, slices, etc.) to be concealed are identified, a combination of spatial and temporal replacement techniques may be applied to fill in lost picture elements. A specific class of spatio-temporal error concealment algorithms for MPEG video is described and alternative realizations are compared via detailed end-to-end simulations for both one- or two-tier transmission media. Several algorithm enhancements based on directional interpolation, ‘I-picture motion vectors’, and use of MPEG-2 ‘scalability’ features are also presented. In each case, achievable performance improvements are estimated via simulation. Overall, these results demonstrate that the proposed class of error concealment algorithms provides significant robustness for MPEG video delivery in the presence of channel impairments, permitting useful operation at ATM cell-loss rates in the region of 10⁻⁴ to 10⁻³ and 10⁻² to 10⁻¹ for one- and two-tier transmission scenarios, respectively.     

Error prevention and concealment for scalable video coding with dual-priority transmission

In this work, we present an efficient error resilient system against ATM cell loss using a hybrid error concealment and error propagation prevention (ECP) technique with dual-priority transmission scheme (DPTS). DPTS performs traffic policing to form dual-priority cells in ATM connections and manages to make most cell losses occur in a low priority layer. However, cell loss may still occur in the high priority layer if the bandwidth is not reserved enough for the usually variable bitrate video traffic. Therefore, the ECP technique can still be utilized to reduce the error damage and limit the impact of cell loss to the erroneous slices. Simulation results of two-layer MPEG-2 coding over DPTS in ATM networks demonstrate that ECP with feedback over DPTS can effectively isolate errors and reduce the damage to yield a satisfactory performance, even when the cell-loss rate is as high as 8%.     

Error concealment for shape in MPEG-4 object-based video coding.

In asynchronus transfer mode networks, cell loss or channel errors can cause data to be dropped in the channel. When digital images/videos are transmitted over these networks, one must be able to reconstruct the missing data so that the impact of the errors is minimized. In this paper, we present an error-concealment technique for shape in MPEG-4 object-based video coding. This method, which is based on using global motion estimation and compensation techniques for boundary recovery, consists of three steps: (1) boundary extraction from shape; (2) boundary patching using global motion compensation; and (3) boundary filling to reconstruct the shape of the damaged video object planes. Global motion parameters are inserted as part of the USER_DATA field in the compressed stream and are utilized in reconstructing the damaged boundaries of compressed video object planes.     

Selective recovery of video packet loss using error concealment

An efficient recovery method using error concealment is proposed for video packet loss in fast packet switching networks. In this method, the receiver detects the damaged picture area caused by packet loss from the structured picture data received, makes error concealments, notifies the transmitter, and continues decoding. The transmitter, having received the notice, calculates the affected picture area in the local decoded picture and continues encoding without using this affected area. In this selective recovery method, video signals are not stopped even if a long propagation delay exists, no additional information is transmitted to error recovery and conventional coding algorithms can be used. The proposed method is suitable for multipoint communication. Simulation results show the affected picture area is localized for a considerable time attesting to the method's effectiveness     

Error concealment for video transmission with dual multiscale Markov random field modeling

A novel error concealment algorithm based on a stochastic modeling approach is proposed as a post-processing tool at the decoder side for recovering the lost information incurred during the transmission of encoded digital video bitstreams. In our proposed scheme, both the spatial and the temporal contextual features in video signals are separately modeled using the multiscale Markov random field (MMRF). The lost information is then estimated using maximum a posteriori (MAP) probabilistic approach based on the spatial and temporal MMRF models; hence, a unified MMRF-MAP framework. To preserve the high frequency information (in particular, the edges) of the damaged video frames through iterative optimization, a new adaptive potential function is also introduced in this paper. Comparing to the existing MRF-based schemes and other traditional concealment algorithms, the proposed dual MMRF (DMMRF) modeling method offers significant improvement on both objective peak signal-to-noise ratio (PSNR) measurement and subjective visual quality of restored video sequence.     

多视点视频中基于乘加误差模型的亮度和色差校正

由于相机性能的差异,多视点视频序列之间总是存在亮度和颜色的差异,这种差异会影响多视点视频压缩和显示等后续处理过程的性能.本文由此提出了一种适用于多视点视频的亮度和色差(亮色度)校正算法.根据相机成像原理,不同相机的亮色度差异可以归类为乘性误差和加性误差.灰度级偏移,相机电子元件的放大和量化因子差异等都可以由乘加误差模型表示.为降低计算复杂度,本文采用直方图形状匹配的方法计算乘加误差模型的参数.该算法可作为多视点视频系统中的预处理过程,实验结果表明,该算法能够有效地提高后续压缩过程的性能.     

Error concealment using affine transform for H.263 coded videotransmissions

A new error concealment method is proposed that uses motion estimation to consider actual motions, such as rotation, magnification, reduction, and parallel motion, in moving pictures. Since many videos include a variety of complex three-dimensional motions, the proposed method uses an affine transform to estimate the motion of lost data more accurately, thereby producing a higher peak signal-to-noise ratio value and better subjective video quality     

Real-time generation of multi-view video plus depth content using mixed narrow and wide baseline

Content production for stereoscopic 3D-TV displays has become mature in the past years while huge progress has also been achieved in the improvement of the image quality of glasses-free auto-stereoscopic displays and light-field displays. Concerning the latter two display families, the content production workflow is less elaborated and more complex, as the number of required views not only differs considerably but is also likely to increase in the near future. As a co-existence of all 3D display families can be expected for the next years, one aims to establish an efficient content production workflow which yields to high quality content for all 3D-TV displays.Against this background we present a real-time capable multi-view video plus depth (MVD) content production workflow based on a four-camera rig with mixed narrow and wide baseline. Results show the suitability of the approach to simultaneously produce high quality MVD4 and native stereoscopic 3D content.     

Error concealment techniques for digital TV

Compressed video bitstreams are intended for real-time transmission over communication networks. Most of the video coding standards employ the temporal and spatial prediction structure to reduce the transmitted video data. Therefore, the coded video bitstreams are highly sensitive to information loss and channel errors. Even a single bit error can lead to disastrous quality degradation in both time and space. This quality deterioration is exacerbated when no error resilient coding mechanism is employed to protect coded video data against the error prone environments. Error concealment is a data recovery technique that enables the decoder to conceal effects of transmission errors by predicting the lost or corrupted video data from the previously reconstructed error-free information. Motion vector recovery and motion compensation with the estimated motion vector is a good approach to conceal the corrupted macroblock data. In this paper, we develop various error concealment algorithms based on motion vector recovery, and compare their performances to those of conventional error concealment methods.     

基于下抽样多描述视频编码及错误隐藏

采用基于空间下抽样方法实现多描述视频编码,避免了常用多描述视频编码过程的误匹配问题;系统实现了2种不同的多描述分解方案,适用于不同纹理特点的视频图像.为了提高多描述解码器的性能,解码过程中使用基于自适应的边缘保持插值方法进行后处理.仿真实验结果表明,本文提出的编码方法能有效地应用于视频在不同的网络环境下传输;经过解码的后处理过程后,视频图像的信噪比有了明显提高.     

HEVC中基于前后景区域的错误隐藏

新一代视频编码标准HEVC采用四叉树编码结构,能够高效地进行视频编码,尤其适用于高清视频。然而,网络信道的不可靠性会导致视频数据的受损或丢失,HEVC的编码特性决定了视频数据的受损或丢失可能会发生在不同尺寸编码块中。因此,在分析各种不同尺寸编码块自身特性的基础上,提出首先对视频进行前景、后景区分,其中利用OBMA算法区分出边界处与运动物体,然后对于不同区域的错误块分别进行隐藏。     

Error concealment technique based on optical flow

A new motion vector recovery algorithm based on optical flow is proposed. Optical flow fields are very similar to the true motion and can be used to recover three-dimensional motion information. Experimental results on test video sequences produced a higher peak signal-to-noise ratio value than that of conventional methods based on the block matching algorithm