Combined and iterative form of spatial and temporal error concealment for video signals

Video transmission over unreliable channels may suffer from the data loss or corruption. To facilitate the transmission, video content is compressed to save the bandwidth. The compression algorithms remove the redundancies in the video signal, meanwhile increasing the dependencies among symbols in the compressed bit-stream. Thus, when errors occur, they may propagate in both space and time. Among various error control techniques, error concealment (EC) is an effective method that is performed at the decoder to mitigate the influence of errors on the quality of reconstructed images. In this paper, a joint spatial and temporal EC algorithm (JSTEC) is presented. First, several existing temporal EC algorithms are combined in an appropriate order. Then, the spatial correlation in the video is exploited in an iterative form to improve the performance of the temporal EC. Experimental results show that JSTEC performs better both in peak signal-to-noise ratio and subjective quality of images than the existing algorithms.     

动态全局PCA稀疏描述的分布式视频压缩感知

Video reconstruction quality largely depends on the ability of employed sparse domain to adequately represent the underlying video in Distributed Compressed Video Sensing (DCVS). In this paper, we propose a novel dynamic global-Principal Component Analysis (PCA) sparse representation algorithm for video based on the sparse-land model and nonlocal similarity. First, grouping by matching is realized at the decoder from key frames that are previously recovered. Second, we apply PCA to each group (sub-dataset) to compute the principle compo-nents from which the sub-dictionary is con-structed. Finally, the non-key frames are reconstructed from random measurement data using a Compressed Sensing (CS) recon-struction algorithm with sparse regularization. Experimental results show that our algorithm has a better performance compared with the DCT and K-SVD dictionaries.     

基于Node-Cell结构的HEVC帧内编码

随着视频传输和广播的发展,高分辨率视频的应用也越来越广泛,为了更好地适应高清视频内容,JCT-VC(Joint Collaborative Team on Video Coding)工作组制定了具有更高压缩效率的新一代视频压缩标准HEVC(High Efficiency Video Coding)。HEVC中的帧内预测包括Angular预测模式、planar模式等。基于Node-Cell结构的帧内预测方法在Angular预测基础上实现了双向预测,提供了更多的模式选择和更高的预测精度。Node-Cell结构中所有像素在当前块被分成两个子集,node像素点和cell像素点,node像素的重建值被用于内插预测cell像素。新增的帧内模式信息被设计为表示下采样率,它由该编码单元的细节及复杂度决定。为了保证重建质量,node像素和cell像素的残差均被发送到解码器。实验结果表明Node-Cell结构会提高预测精度。     

Side information creation for efficient Wyner–Ziv video coding: Classifying and reviewing

Video coding technologies have played a major role in the explosion of large market digital video applications and services. In this context, the very popular MPEG-x and H-26x video coding standards adopted a predictive coding paradigm, where complex encoders exploit the data redundancy and irrelevancy to ‘control’ much simpler decoders. This codec paradigm fits well applications and services such as digital television and video storage where the decoder complexity is critical, but does not match well the requirements of emerging applications such as visual sensor networks where the encoder complexity is more critical. The Slepian–Wolf and Wyner–Ziv theorems brought the possibility to develop the so-called Wyner–Ziv video codecs, following a different coding paradigm where it is the task of the decoder, and not anymore of the encoder, to (fully or partly) exploit the video redundancy. Theoretically, Wyner–Ziv video coding does not incur in any compression performance penalty regarding the more traditional predictive coding paradigm (at least for certain conditions). In the context of Wyner–Ziv video codecs, the so-called side information, which is a decoder estimate of the original frame to code, plays a critical role in the overall compression performance. For this reason, much research effort has been invested in the past decade to develop increasingly more efficient side information creation methods. This paper has the main objective to review and evaluate the available side information methods after proposing a classification taxonomy to guide this review, allowing to achieve more solid conclusions and better identify the next relevant research challenges. After classifying the side information creation methods into four classes, notably guess, try, hint and learn, the review of the most important techniques in each class and the evaluation of some of them leads to the important conclusion that the side information creation methods provide better rate-distortion (RD) performance depending on the amount of temporal correlation in each video sequence. It became also clear that the best available Wyner–Ziv video coding solutions are almost systematically based on the learn approach. The best solutions are already able to systematically outperform the H.264/AVC Intra, and also the H.264/AVC zero-motion standard solutions for specific types of content.     

基于ARM的AVS-P2视频解码器

研究了AVS视频解码部分,介绍了AVS视频编码过程的关键技术.详细阐述了针对ARM平台的AVS-P2视频解码算法的优化,并在基于ARM-Linux的嵌入武平台上实现了AVS视频解码.     

Comparative study of frame-compatible stereo 3D services and a novel method for spatial interleaving using HEVC

There are a number of ways to realize frame-compatible stereo 3D services, which includes the spatial and temporal interleaving based methods. We first present the detailed analysis of these methods with respect to compression efficiency, backward compatibility and the ability to re-use existing infrastructure. Simulations have been set up to benchmark the Rate–Distortion (R–D) performance of these options with the state-of-the-art Multiview Video Coding (MVC) and results show that temporal interleaving based method performs better when compared to spatial interleaving in both H.264/AVC as well as in HEVC compression schemes, with MVC still outperforms both the methods. In the case of the spatial interleaving method using HEVC, we propose to encapsulate the two views in different tiles with filters turned off across the tile boundary. Simulation results show that the proposed method, when compared to “no-tiles” case, improves decoder performance by approximately 50% with no reduction in R–D performance.     

High‐Performance Spatial and Temporal Error‐Concealment Algorithms for Block‐Based Video Coding Techniques

A compressed video bitstream is sensitive to errors that may severely degrade the reconstructed images even when the bit error rate is small. One approach to combat the impact of such errors is the use of error concealment at the decoder without increasing the bit rate or changing the encoder. For spatial‐error concealment, we propose a method featuring edge continuity and texture preservation as well as low computation to reconstruct more visually acceptable images. Aiming at temporal error concealment, we propose a two‐step algorithm based on block matching principles in which the assumption of smooth and uniform motion for some adjacent blocks is adopted. As simulation results show, the proposed spatial and temporal methods provide better reconstruction quality for damaged images than other methods.     

基于超图排序算法的视频摘要

视频摘要技术作为一种快速感知视频内容的方式得到了广泛的关注.现有基于图模型的视频摘要方法将视频帧作为顶点,通过边表示两个顶点之间的关系,但并不能很好地捕获视频帧之间的复杂关系.为了克服该缺点,本文提出了一种基于超图排序算法的静态视频摘要方法（Hyper-Graph Ranking based Video Summarization,HGRVS）.HGRVS方法首先通过构建视频超图模型,将任意多个有内在关联的视频帧使用一条超边连接;然后提出一种基于超图排序的视频帧分类算法将视频帧按内容分类;最后通过求解提出的一种优化函数来生成静态视频摘要.在Open Video Project和YouTube两个数据集上的大量主观与客观实验验证了所提HGRVS算法的优良性能.     

Error-resilient scalable compression based on distributed video coding

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian–Wolf (SW) and Wyner–Ziv (WZ). In this work, a performance analysis of image and video coding schemes based on DVC is presented, addressing temporal, quality and spatial scalability. More specifically, conventional coding is used to obtain a base layer while WZ coding generates the enhancement layers. At the decoder, the base layer is used to construct Side Information (SI) for the DVC decoding process. Initially, we show that the scalable DVC approach is codec-independent, which means that it is independent from the method used to encode the base layer. Moreover, the influence of the base layer quality on the overall performance of the schemes is studied. Finally, evaluation of the proposed schemes is performed in both cases, with and without transmission errors. The simulation results show that scalable DVC has a lower compression efficiency than conventional scalable coding (i.e. scalable video coding and JPEG2000 for video and image, respectively) in error-free conditions. On the other hand, the DVC-based schemes show better error resilience as they outperform conventional scalable coding in error-prone conditions. More specifically, the Rate Distortion (RD) performance of the proposed schemes for image coding is compared with respect to Reed Solomon (RS) protected JPEG2000. While the latter exhibits a cliff effect as its performance dramatically decreases after a certain error rate, the performance of the DVC-based schemes decreases in a steady way with error rate increase.     

Compressive Sensing Based Soft Video Broadcast Using Spatial and Temporal Sparsity

Video broadcasting over wireless network has become a very popular application. However, the conventional digital video broadcasting framework can hardly accommodate heterogeneous users with diverse channel conditions, which is called the cliff effects. To overcome this cliff effects and provide a graceful degradation to multi-receivers, in this paper, we use the nonlocal sparsity and hierarchical GOP structure to propose a novel CS based soft video broadcast scheme. CS has properties of minimizing bandwidth consumption and generating measurements with equal importance which are exactly needed by video soft broadcast. In the proposed scheme, the measurement data are generated by block-wise compressive sensing (BCS), and then the measurement data packets are sent over a highly dense constellation though OFDM channel to achieve a simple encoder. Ideally, with the GOP structure, inter frame has lower sampling rate than intra frame to achieve better compression efficiency. At the decoder side, due to equally-important packets and property of soft broadcast, each user can receive the noise-corrupted measurements matching its channel condition and reconstruct video. The hierarchical GOP structure is presented to explode the correlation and non-local sparsity among video frames during the recover process. Additionally, using non-local sparsity, group based CS reconstruction with adaptive dictionaries is proposed to improve decoding quality. The experimental results show that the proposed scheme provides better performance compared with the traditional SoftCast with up to 8 dB coding gain for some channel conditions.     

Tennis Video 2.0: A new presentation of sports videos with content separation and rendering

This paper proposes a new method for presenting sports videos. Tennis videos are used as an example for the implementation of a viewing program called as Tennis Video 2.0. For the methods in video analysis, background generation by considering the pixels in temporal and spatial distribution is proposed; foreground segmentation combining automatic trimap generation and matting model is proposed. To provide more functions in watching videos, the rendering flow of video contents and the semantic Scalability are proposed. With the new analysis and rendering tools, the presentation of sports videos has three properties—Structure, Interactivity, and Scalability. The experiments show that several broadcasting game videos are employed to evaluate the robustness and performance of the proposed system. For user study, 20 evaluators highly identify that Tennis Video 2.0 is a new presentation of sports videos and give people better viewing experience.     

Cross-band noise model refinement for transform domain Wyner-Ziv video coding

Distributed Video Coding (DVC) is a new video coding paradigm, which mainly exploits the source statistics at the decoder based on the availability of decoder side information. One approach to DVC is feedback channel based Transform Domain Wyner-Ziv (TDWZ) video coding. The efficiency of current TDWZ video coding trails that of conventional video coding solutions, mainly due to the quality of side information, inaccurate noise modeling and loss in the final coding step. The major goal of this paper is to enhance the accuracy of the noise modeling, which is one of the most important aspects influencing the coding performance of DVC. A TDWZ video decoder with a novel cross-band based adaptive noise model is proposed, and a noise residue refinement scheme is introduced to successively update the estimated noise residue for noise modeling after each bit-plane. Experimental results show that the proposed noise model and noise residue refinement scheme can improve the rate-distortion (RD) performance of TDWZ video coding significantly. The quality of the side information modeling is also evaluated by a measure of the ideal code length.     

On the methods and performances of rational downsizing video transcoding

Video transcoding is a popular technique for adapting the bit-rate or spatial/temporal resolution of a precoded video to suit better the constraints and requirements of different transmission networks and receiving devices. To minimize computational complexity, many fast methods have been proposed to obtain the motion vectors required for transcoding a precoded video through reducing its frame size by an integral factor. In this paper, we extend the existing work by developing and comparing several fast methods of downsizing precoded videos by a rational factor. Methods that outperform others under different conditions or with different computational requirements are identified, and an application scenario that can benefit from the proposed rational downsizing video transcoding is presented. An efficient scheme is also proposed to select the proper reduced frame size for sustaining the best possible video quality at a specified lower bit-rate. The superiority of the proposed transcoding approach in comparison with the existing integral downsizing video transcoding or cascaded video re-encoding methods is evident from the experimental results shown in this paper.     

Adaptive Block-size Transform based Just-Noticeable Difference model for images/videos

In this paper, we propose a novel Adaptive Block-size Transform (ABT) based Just-Noticeable Difference (JND) model for images/videos. Extension from 8×8 Discrete Cosine Transform (DCT) based JND model to 16×16 DCT based JND is firstly performed by considering both the spatial and temporal Human Visual System (HVS) properties. For still images or INTRA video frames, a new spatial selection strategy based on the Spatial Content Similarity (SCS) between a macroblock and its sub-blocks is proposed to determine the transform size to be employed to generate the JND map. For the INTER video frames, a temporal selection strategy based on the Motion Characteristic Similarity (MCS) between a macroblock and its sub-blocks is presented to decide the transform size for the JND. Compared with other JND models, our proposed scheme can tolerate more distortions while preserving better perceptual quality. In order to demonstrate the efficiency of the ABT-based JND in modeling the HVS properties, a simple visual quality metric is designed by considering the ABT-based JND masking properties. Evaluating on the image and video subjective databases, the proposed metric delivers a performance comparable to the state-of-the-art metrics. It confirms that the ABT-based JND consists well with the HVS. The proposed quality metric also is applied on ABT-based H.264/Advanced Video Coding (AVC) for the perceptual video coding. The experimental results demonstrate that the proposed method can deliver video sequences with higher visual quality at the same bit-rates.     

分布式多视点视频编码中的立体边信息生成

新兴的分布式多视点视频编码通过在解码端使用立体边信息可充分挖掘传统单视点的时间相关性和多视点特有的视间相关性.研究了一种多视点运动预测的立体边信息生成算法,提出了近似视差估计和编码端掩模融合算法.实验证明,解码端采用近似的视差矢量,编码端传递某些先验信息,都能有效提高立体边信息精度,降低计算复杂度.     

基于朴素贝叶斯分类的DVC-HEVC快速转码

分布式视频编码(DVC)与传统视频编码之间的转码为移动终端设备之间的低功耗视频通信提供了一种有效的实现思路。以DVC与HEVC转码为研究对象,利用DVC解码端信息,针对高效视频编码(HEVC)中复杂度极高的编码单元(CU)划分过程进行复杂度优化研究。在DVC解码端提取与CU划分相关的纹理复杂度、运动矢量及预测残差3种特征信息;在HEVC编码端基于朴素贝叶斯原理建立CU快速划分模型,模型生成后便可以通过输入特征信息对当前CU划分进行快速决策,避免大量率失真(RD)代价计算过程。实验结果表明,本方案在编码比特率略有上升的情况下大幅缩短了HEVC编码时间,平均下降幅度达到58.26%,且几乎不影响视频质量。     

基于CEVA平台的WMV视频解码器优化

针对VC-1标准下的WMV视频解码,提出了一种基于CEVA平台下的优化方案。该方案对运动补偿、去方块滤波、反量化、反变换的算法结构进行优化后,再利用CEVA的SIMD（单指令多数据处理）和VLIW（超长指令字）技术实现DSP平台上的优化。实验仿真表明：经过优化处理后,各个模块解码时间对比之前有了大幅度降低．解码速度最高...     

A joint encoder–decoder error control framework for stereoscopic video coding

Stereoscopic video coding (SSVC) plays an important role in various 3D video applications. In SSVC, robust stereoscopic video transmission over error-prone networks is still a challenge problem to be solved. In this paper, we propose a joint encoder–decoder error control framework for SSVC, where error-resilient source coding, transmission network conditions, and error concealment scheme are jointly considered to achieve better error robustness performance. The proposed joint encoder–decoder error control framework includes two parts: an error concealment algorithm at the decoder side and a rate–distortion optimized error resilience algorithm at the encoder side. For error concealment at the decoder side, an overlapped block motion and disparity compensation based error concealment scheme is proposed to adaptively utilize inter-view correlations and temporal correlations. For error resilience at the encoder side, first, the inter-view refreshment is proposed for SSVC to suppress error propagations. Then, an end-to-end distortion model for SSVC is derived, which jointly considers the transmission network conditions, inter-view refreshment, and error concealment tools at the decoder side. Finally, based on the derived end-to-end distortion model, the rate–distortion optimized error resilience algorithm is presented to adaptively select inter-view, inter- or intra-coding for SSVC. The experimental results show that the proposed joint encoder–decoder error control framework has superior error robustness performance for stereoscopic video transmission over error-prone networks.     

基于预测式错误恢复机制的多描述视频编码研究

该文提出一种基于预测式错误恢复机制的多描述视频编码方法。在编码端通过预测单路解码可能产生的错误影响,为每个描述分配必要的冗余信息。考虑到视频编码压缩的效率问题,设计了不同的编码模式来处理冗余信息。解码端可以充分利用冗余信息,从而实现丢失视频帧的高质量恢复。实验表明,与传统时域采样方法相比,该方法具有更好的率失真性能。