A fully scalable motion model for scalable video coding.

Motion information scalability is an important requirement for a fully scalable video codec, especially for decoding scenarios of low bit rate or small image size. So far, several scalable coding techniques on motion information have been proposed, including progressive motion vector precision coding and motion vector field layered coding. However, it is still vague on the required functionalities of motion scalability and how it collaborates flawlessly with other scalabilities, such as spatial, temporal, and quality, in a scalable video codec. In this paper, we first define the functionalities required for motion scalability. Based on these requirements, a fully scalable motion model is proposed along with tailored encoding techniques to minimize the coding overhead of scalability. Moreover, the associated rate distortion optimized motion estimation algorithm will be provided to achieve better efficiency throughout various decoding scenarios. Simulation results will be presented to verify the superiorities of proposed scalable motion model over nonscalable ones.     

Error-resilient scalable compression based on distributed video coding

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian–Wolf (SW) and Wyner–Ziv (WZ). In this work, a performance analysis of image and video coding schemes based on DVC is presented, addressing temporal, quality and spatial scalability. More specifically, conventional coding is used to obtain a base layer while WZ coding generates the enhancement layers. At the decoder, the base layer is used to construct Side Information (SI) for the DVC decoding process. Initially, we show that the scalable DVC approach is codec-independent, which means that it is independent from the method used to encode the base layer. Moreover, the influence of the base layer quality on the overall performance of the schemes is studied. Finally, evaluation of the proposed schemes is performed in both cases, with and without transmission errors. The simulation results show that scalable DVC has a lower compression efficiency than conventional scalable coding (i.e. scalable video coding and JPEG2000 for video and image, respectively) in error-free conditions. On the other hand, the DVC-based schemes show better error resilience as they outperform conventional scalable coding in error-prone conditions. More specifically, the Rate Distortion (RD) performance of the proposed schemes for image coding is compared with respect to Reed Solomon (RS) protected JPEG2000. While the latter exhibits a cliff effect as its performance dramatically decreases after a certain error rate, the performance of the DVC-based schemes decreases in a steady way with error rate increase.     

Fast multiresolution motion estimation algorithms for wavelet-based scalable video coding

Motion estimation and compensation in wavelet domain have received much attention recently. To overcome the inefficiency of motion estimation in critically sampled wavelet domain, the low-band-shift (LBS) method and the complete-to-overcomplete discrete wavelet transform (CODWT) method are proposed for motion estimation in shift-invariant wavelet domain. However, a major disadvantage of these methods is the computational complexity. Although the CODWT method has reduced the computational complexity by skipping the inverse wavelet transform and making the direct link between the critically sampled subbands and the shift-invariant subbands, the full search algorithm (FSA) increases it. In this paper, we proposed two fast multiresolution motion estimation algorithms in shift-invariant wavelet domain: one is the wavelet matching error characteristic based partial distortion search (WMEC-PDS) algorithm, which improves computational efficiency of conventional partial distortion search algorithms while keeping the same estimate accuracy as the FSA; another is the anisotropic double cross search (ADCS) algorithm using multiresolution-spatio-temporal context, which provides a significantly computational load reduction while only introducing negligible distortion compared with the FSA. Due to the multiresolution nature, both the proposed approaches can be applied to wavelet-based scalable video coding. Experimental results show the superiority of the proposed fast motion estimation algorithms against other fast algorithms in terms of speed-up and quality.     

Effective computation-aware algorithm by inter-layer motion analysis for scalable video coding

Scalable video coding incorporated with computation-aware ability achieves quality as well as being computation scalable. This paper presents a computation-aware algorithm for scalable video coding with spatial/quality scalability aiming for the best trade-off between rate distortion performance and computational consumption. We first observe and analyze and then establish a model for the motion vector difference relationship between the scalable base and enhancement layers. By using the modeling results, a linear algorithm for computation distribution is thus proposed to allocate the computation for each macroblock in the enhancement layer. In addition, the rate distortion costs of the base layer are also taken into account for the computation allocation process in order to further improve the coding performance. The simulation results demonstrate that our proposed computation-aware algorithm not only accomplishes better rate distortion performance than other works under the same computational constraints, but also achieves less computation necessities.     

Scalable video compression via overcomplete motion compensated wavelet coding

Recently, there have been a flurry of works on overcomplete motion compensated wavelet coding (OMCWC). In this paper, we address the importance of phase and focus on the design of scalable video coding algorithms within the OMCWC framework. Specifically, our new contributions consist of the following three components: (1) efficient block motion estimation techniques in the wavelet domain including hierarchical and fractional-pel block matching, (2) extend overcomplete motion compensated prediction (MCP) into overcomplete motion compensated temporal filtering (MCTF) to achieve temporal scalability (3) context modeling strategies for embedded quantization and entropy coding of 3D wavelet coefficients. Experiment results are used to demonstrate that the class of overomplete MCP/MCTF coders are capable of achieving comparable performance to other competing interframe wavelet coders.     

Bottom-up motion compensated prediction in wavelet domain for spatially scalable video coding

A new in-band motion compensation algorithm for wavelet-based video coding is proposed: the bottom-up prediction algorithm (BUP). This algorithm overcomes the periodic shift-invariance of the discrete wavelet transform (DWT) and is formalised into prediction rules using filtering operations. The combination of all prediction rules of the BUP algorithm defines a new transform: the bottom-up overcomplete DWT or BUP ODWT, which is shift-invariant. The envisaged application for the BUP algorithm is spatially scalable wavelet video coding.     

Foveation scalable video coding with automatic fixation selection

Image and video coding is an optimization problem. A successful image and video coding algorithm delivers a good tradeoff between visual quality and other coding performance measures, such as compression, complexity, scalability, robustness, and security. In this paper, we follow two recent trends in image and video coding research. One is to incorporate human visual system (HVS) models to improve the current state-of-the-art of image and video coding algorithms by better exploiting the properties of the intended receiver. The other is to design rate scalable image and video codecs, which allow the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. Specifically, we propose a foveation scalable video coding (FSVC) algorithm which supplies good quality-compression performance as well as effective rate scalability. The key idea is to organize the encoded bitstream to provide the best decoded video at an arbitrary bit rate in terms of foveated visual quality measurement. A foveation-based HVS model plays an important role in the algorithm. The algorithm is adaptable to different applications, such as knowledge-based video coding and video communications over time-varying, multiuser and interactive networks.     

Drift-resistant SNR scalable video coding.

We address the problem of enhancement layer drift estimation for fine granular scalable video. An optimal per-pixel drift estimation algorithm is introduced. The encoder assumes that there is some truncation of the enhancement layer, which does not allow the enhancement layer reference to be properly reconstructed, and the encoder recursively estimates the associated drift and chooses coding modes accordingly. The approach yields performance gains of about 1 dB across low to medium rates. In addition, we investigate dual frame prediction, for both base and enhancement layer, with pulsed-quality allocation in the base     

Multidimensional quantizers for scalable video compression

Scalable video compression requires the creation of an encoded bit stream that may be decoded in part if channel bandwidth drops, decoder resources are limited, or a smaller image than the source is desired. A separable spatiotemporal subband decomposition is combined with vector and lattice quantizers modified so that the individual subbands may be decoded scalably. This results in finer bandwidth control and more flexibility than simply discarding entire subbands     

Absolute value coding for robust and scalable video coding

Absolute value coding is introduced as a method for significantly reducing temporal drift within a motion compensated predictive video codec in the presence of loss. Drift reduction both improves error resilience and enables scalability by omission of parts of the bit-stream. In conjunction with matching pursuits, the system can be used to provide a displaced frame difference codec using fixed length codewords, which further improves error resilience and facilitates simple bit-stream editing.     

一种低编码复杂度可分级的视频编码系统

在可分级视频编码（SVC,scalable video coding）的框架下,利用分布式视频编码（DVC,distributed video coding）技术,设计了一种低编码复杂度的SVC方案。该系统具有空间可分级的特性,各分层中仅用到了传统的帧内编码技术和DVC技术,最大限度的减小了SVC系统的编码复杂度。在该...     

Error concealment for SNR scalable video coding

This paper proposes an efficient error concealment method for SNR scalable coded video. The algorithm adaptively selects a proper concealment candidate from the base or the enhanced pictures to conceal the artifact of a lost enhancement block. To determine the best concealment candidate, we propose a trial process in which the concealment candidates are examined based on two criteria: (1) picture continuity at the border of concealed macroblocks, and (2) to satisfy the coding distortion bound of the base layer coefficients when they are available. For the latter, requantization of the concealed picture with the base layer quantizer step size and its dequantized pixels should result in zero distortion. We have implemented the method on a proposed SNR scalable H.264 video codec and compared the decoded video quality against just copying the base layer pixels into the enhanced picture. Simulation results show that the proposed method can achieve a considerable improvement by up to 3 dB especially in situations where the enhancement layer contains a large portion of the picture information. This will make scalable video transmission more successful over unreliable channels.     

Interlayer bit allocation for scalable video coding

In this paper, we present a theoretical analysis of the distortion in multilayer coding structures. Specifically, we analyze the prediction structure used to achieve temporal, spatial, and quality scalability of scalable video coding (SVC) and show that the average peak signal-to-noise ratio (PSNR) of SVC is a weighted combination of the bit rates assigned to all the streams. Our analysis utilizes the end user's preference for certain resolutions. We also propose a rate-distortion (R-D) optimization algorithm and compare its performance with that of a state-of-the-art scalable bit allocation algorithm. The reported experiment results demonstrate that the R-D algorithm significantly outperforms the compared approach in terms of the average PSNR.     

Spatially scalable video compression employing resolution pyramids

A spatially scalable video coding scheme for low bit rates is proposed. The codec is especially well suited for communications applications because it is based on motion-compensated predictive coding which provides the necessary low-delay property. The frames to be coded are decomposed into a Gaussian pyramid. Motion estimation and compensation are performed between corresponding pyramid levels of successive frames. We show that, to fulfill specific needs of spatial scalability, the motion compensation on each level must result in compatible prediction errors (displaced frame differences, DFD). Compatibility of the prediction errors means that the pyramid formed by independently obtained DFD's (the DFD pyramid) is close to a Gaussian pyramid decomposition of the DFD of the highest resolution level. From the DFD pyramid, a least squares Laplacian pyramid is derived, which is quantized and coded. The DFD encoder outputs an embedded bit stream. Thus, the coder control may truncate the bit stream at any point, and can keep a fixed rate. The motion vector fields obtained at the different resolution levels are also encoded by employing a pyramid approach. Simulation results show that the proposed coder achieves a coding gain compared to simulcast coding     

Adaptive packet scheduling for scalable video streaming with network coding

Over the last decade, the emergence of new multimedia devices has motivated the research on efficient media streaming mechanisms that adapt to dynamic network conditions and heterogeneous devices’ capabilities. Network coding as a rateless code has been applied to collaborative media streaming applications and brings substantial improvements regarding throughput and delay. However, little attention has been given to the recoverability of encoded data, especially for the streaming with a strict deadline. This in turn leads to severe quality of experience. In this paper, we solve the unrecoverable transmission by proposing a multi-generation packet scheduling problem, which is treated as a video quality maximization problem and solved using dynamic programming algorithm. Experimental results confirm that the proposed algorithm brings better data recoverability and better quality of service in terms of video quality, delivery ratio, lower redundancy rate under different network sizes.     

Lifting-based invertible motion adaptive transform (LIMAT) framework for highly scalable video compression

We propose a new framework for highly scalable video compression, using a lifting-based invertible motion adaptive transform (LIMAT). We use motion-compensated lifting steps to implement the temporal wavelet transform, which preserves invertibility, regardless of the motion model. By contrast, the invertibility requirement has restricted previous approaches to either block-based or global motion compensation. We show that the proposed framework effectively applies the temporal wavelet transform along a set of motion trajectories. An implementation demonstrates high coding gain from a finely embedded, scalable compressed bit-stream. Results also demonstrate the effectiveness of temporal wavelet kernels other than the simple Haar, and the benefits of complex motion modeling, using a deformable triangular mesh. These advances are either incompatible or difficult to achieve with previously proposed strategies for scalable video compression. Video sequences reconstructed at reduced frame-rates, from subsets of the compressed bit-stream, demonstrate the visually pleasing properties expected from low-pass filtering along the motion trajectories. The paper also describes a compact representation for the motion parameters, having motion overhead comparable to that of motion-compensated predictive coders. Our experimental results compare favorably to others reported in the literature, however, our principal objective is to motivate a new framework for highly scalable video compression.     

可伸缩视频流的安全网络编码方案

为了使可伸缩视频流在异构网络中达到分层安全等级的目的,运用随机函数来随机化视频流各层中的部分数据流,并结合网络编码来抵御已知的明文攻击。此外,对网络编码器进行了研究,设计有序随机线性网络编码器用于可伸缩视频的传输,可以用很少的随机化操作来达到可扩展的安全等级,并降低通信开销。分析表明,所提方案可有效增加网络的吞吐率。     

Model design for scalable two-dimensional model-based video coding

Scalable and very low bit rate video coding is vital for audio-visual conversational services over narrow bandwidth channels. A novel model design scheme is proposed in order to make the points of an object model represent the motion more accurately, which will in turn enable better compression. Experimental results demonstrate the performance of the proposed scheme.     

A rate-distortion analysis on motion prediction efficiency and mode decision for scalable wavelet video coding

A rate-distortion model for describing the motion prediction efficiency in interframe wavelet video coding is proposed in this paper. Different from the non-scalable video coding, the scalable wavelet video coding needs to operate under multiple bitrate conditions and it has an open-loop structure. The conventional Lagrangian multiplier, which is widely used to solve the rate-distortion optimization problems in video coding, does not fit well into the scalable wavelet structure. In order to find the rate-distortion trade-off due to different bits allocated to motion and textual information, we suggest a motion information gain (MIG) metric to measure the motion prediction efficiency. Based on this metric, a new cost function for mode decision is proposed. Compared with the conventional Lagrangian method, our experiments show that the proposed method is less extraction-bitrate dependent and generally improves both the PSNR performance and the visual quality for the scalability cases.     

适用于实时交通监控的时空可伸缩视频编码方法

提出了一种适用于视频监控的时空可伸缩编码方法。首先提出一种适用于可伸缩视频编码特点的监控算法,利用该算法将运动区域从背景中提取出来,并用模式和方向快速判别算法去掉对编码增益很小的冗余模式,基本层利用该算法进行编码,增强层利用基本层的信息预测可能的模式集合,然后进行零块预先判决,最后对最高时间级的高速运动块通过视觉门限来降低其空间冗余度。实验证明,该方法能够较大地提高编码速度和一定程度上提高压缩率,并且视觉质量损失很小。