结合零树的分形运动补偿编码算法的研究

当前的视频编码算法无论是使用小波变换、分形理论还是运动补偿技术，使用单一的编码算法总是存在着压缩效率与实时视觉的矛盾。至今为止，已提出的小波分形混合编码方法也没能很好地利用小波变换后系数的统计特性，压缩的效果不论是在压缩比PSNR或是可伸缩性上并不理想，为此，本文提出了一种小波域内基于零树特点的分形运动补偿混合编码算法。实验证明，该算法具有良好的压缩性能，并且在重建图像和拥有逼真的视觉特件，获得了可伸缩的能够渐进的传输的编码码流。     

Region-based video coding using mathematical morphology

This paper presents a region-based coding algorithm for video sequences. The coding approach involves a time-recursive segmentation relying on the pixels homogeneity, a region-based motion estimation, and motion compensated contour and texture coding. This algorithm is mainly devoted to very low bit rate video coding applications. One of the important features of the approach is that no assumption is made about the sequence content. Moreover the algorithm structure leads to a scalable coding process giving various levels of quality and bit rates. The coding as well as the segmentation are controlled to regulate the bit stream. Finally, the interest of morphological tools in the content of region-based coding is extensively reviewed     

一种基于运动补偿三维小波的 多描述视频编码方法

 本文将多描述编码与运动补偿三维小波可扩展视频编码相结合,提出了一种基于运动补偿三维小波的多描述视频编码方法.该方法首先根据编码序列的运动特性,自适应地进行每个描述的码率分配,以控制各个描述中的冗余,然后将编码序列的关键信息-运动矢量和低频帧码流复制到两个描述中,并将高频帧码流分配到不同的描述中.在解码端根据正确接收信息的不同,采用不同的方法进行视频重建.实验结果表明,与单描述编码方法相比,在信道丢包率较高的情况下,本文方法可以提供更好的传输鲁棒性.     

计算能力可伸缩的运动估计率失真优化

不同硬件设备具有不同的计算能力,能否在任意给定计算能力约束下达到最好的编码效率,是当前视频编码研究领域的一个极具挑战性问题.同时,随着分块结构越来越灵活的编码标准不断出现（如：HEVC,H.264等）,运动估计不得不反复地应用在大小不同的各种分块上,导致其对编码总体计算复杂度的影响愈加重要.在此背景下,本文提出了一种针对运动估计的计算能力可伸缩（Complexity scalable）优化算法.我们通过对运动估计过程中预测失真度和计算复杂度的变化规律建模,发现根据各宏块的特性设置不同的预测失真度阈值可以优化地分配计算资源.而该阈值的大小则恰恰是各宏块的最小预测失真度加上一个由复杂度约束统一决定的偏移量.有鉴于此,我们进一步构造了计算能力可伸缩的优化运动估计算法,在不增加额外计算量的前提下,快速地得到各个宏块所对应的优化阈值,并完成运动估计.通过实验分析,该算法不仅具备自动适应不同计算复杂度约束的能力,而且在任意给定的复杂度约束下,都能提供优化的编码性能.     

A rate-distortion analysis on motion prediction efficiency and mode decision for scalable wavelet video coding

A rate-distortion model for describing the motion prediction efficiency in interframe wavelet video coding is proposed in this paper. Different from the non-scalable video coding, the scalable wavelet video coding needs to operate under multiple bitrate conditions and it has an open-loop structure. The conventional Lagrangian multiplier, which is widely used to solve the rate-distortion optimization problems in video coding, does not fit well into the scalable wavelet structure. In order to find the rate-distortion trade-off due to different bits allocated to motion and textual information, we suggest a motion information gain (MIG) metric to measure the motion prediction efficiency. Based on this metric, a new cost function for mode decision is proposed. Compared with the conventional Lagrangian method, our experiments show that the proposed method is less extraction-bitrate dependent and generally improves both the PSNR performance and the visual quality for the scalability cases.     

A fully scalable motion model for scalable video coding.

Motion information scalability is an important requirement for a fully scalable video codec, especially for decoding scenarios of low bit rate or small image size. So far, several scalable coding techniques on motion information have been proposed, including progressive motion vector precision coding and motion vector field layered coding. However, it is still vague on the required functionalities of motion scalability and how it collaborates flawlessly with other scalabilities, such as spatial, temporal, and quality, in a scalable video codec. In this paper, we first define the functionalities required for motion scalability. Based on these requirements, a fully scalable motion model is proposed along with tailored encoding techniques to minimize the coding overhead of scalability. Moreover, the associated rate distortion optimized motion estimation algorithm will be provided to achieve better efficiency throughout various decoding scenarios. Simulation results will be presented to verify the superiorities of proposed scalable motion model over nonscalable ones.     

一种低编码复杂度可分级的视频编码系统

在可分级视频编码（SVC,scalable video coding）的框架下,利用分布式视频编码（DVC,distributed video coding）技术,设计了一种低编码复杂度的SVC方案。该系统具有空间可分级的特性,各分层中仅用到了传统的帧内编码技术和DVC技术,最大限度的减小了SVC系统的编码复杂度。在该...     

Reliable tracking of facial features in semantic-based video coding

A new method of tracking the position of important facial features for semantic-based moving image coding is presented. Reliable and fast tracking of the facial features in head-and-shoulders scenes is of paramount importance for reconstruction of the speakers motion in videophone systems. The proposed method is based on eigenvalue decomposition of the sub-images extracted from subsequent frames of the video sequence. The motion of each facial feature (the left eye, the right eye, the nose and the lips) is tracked separately; this means that the algorithm can be easily adapted for a parallel machine. No restrictions, other than the presence of the speaker's face, were imposed on the actual contents of the scene. The algorithm was tested on numerous widely used head-and-shoulders video sequences containing moderate head pan, rotation and zoom, with remarkably good results. Tracking was maintained even when the facial features were occluded. The algorithm can also be used in other semantic-based systems     

Automatic face location detection for model-assisted rate control in H.261-compatible coding of video

We present a novel and practical way to integrate techniques from computer vision to low bit-rate coding systems for video teleconferencing applications. Our focus is to locate and track the faces and selected facial features of persons in typical head-and-shoulders video sequences, and to exploit the location information in a ‘classical’ video coding/decoding system. The motivation is to enable the system to encode selectively various image areas and to produce perceptually pleasing coded images where faces are sharper. We refer to this approach—a mix of classical waveform coding and model—based coding-as model-assisted coding. We propose two totally automatic algorithms which, respectively, perform the detection of a head outline, and identify an ‘eyes-nose-mouth’ region, both from downsampled binary thresholded edge images. The algorithms operate accurately and robustly, even in cases of significant head rotation or partial occlusion by moving objects. We show how the information about face and facial feature location can be advantageously exploited by low bit-rate waveform-based video coders. In particular, we describe a method of object-selective quantizer control in a standard coding system based on motion-compensated discrete cosine transform—CCITT's recommendation H.261. The approach is based on two novel algorithms, namely buffer rate modulation and buffer size modulation. By forcing the rate control algorithm to transfer a fraction of the total available bit-rate from the coding of the non-facial to that of the facial area, the coder produces images with better-rendered facial features, i.e. coding artefacts in the facial area are less pronounced and eye contact is preserved. The improvement was found to be perceptually significant on video sequences coded at the ISDN rate of 64 kbps, with 48 kbps for the input (color) video signal in QCIF format.     

Motion and texture rate-allocation for prediction-based scalable motion-vector coding

Modern video coding applications require data transmission over variable-bandwidth wired and wireless network channels to a variety of terminals, possibly having different screen resolutions and available computing power. Scalable video coding technology is needed to optimally support these applications. Recently proposed wavelet-based video codecs employing spatial-domain motion-compensated temporal filtering (SDMCTF) provide quality, resolution and frame-rate scalability while delivering compression performance comparable to that of H.264, the state-of-the-art in single-layer video coding. These codecs require quality-scalable coding of the motion vectors to support a large range of bit-rates with optimal compression efficiency. In this paper, the practical use of prediction-based scalable motion-vector coding in the context of scalable SDMCTF-based video coding is investigated. Extensive experimental results demonstrate that, irrespective of the employed motion model, our prediction-based scalable motion-vector codec (MVC) systematically outperforms state-of-the-art wavelet-based solutions for both lossy and lossless compression. A new rate-distortion optimized rate-allocation strategy is proposed, capable of optimally distributing the available bit-budget between the different frames and between the texture and motion information, making the integration of the scalable MVC into a scalable video codec possible. This rate-allocation scheme systematically outperforms heuristic approaches previously employed in the literature. Experiments confirm that by using a scalable MVC, lower bit-rates can be attained without sacrificing motion-estimation efficiency and that the overall coding performance at low rates is significantly improved by a better distribution of the available rate between texture and motion information. The only downside of scalable motion-vector coding is a slight performance loss incurred at high bit-rates.     

极低码率下的2D-3D人脸视频编解码

针对动态变化背景下的人脸视频编解码问题,该文提出了一种2D-3D混合编解码系统;具体包括:(1)基于多种观测信息,在线外观模型和粒子滤波的人脸三维运动跟踪;(2)结合参数模型与肌肉模型的3D人脸动画合成;(3)基于头发检测和3D头发模型的头发合成;(4)无缝地拼接前景的三维编解码结果和背景的二维编解码结果.在极低码率下,客观实验表明,该系统在编码效率和解码质量上有较好的综合优势.主观实验表明,该系统的解码结果在脸部具有较高的辨识度.     

Highly scalable video compression with scalable motion coding

A scalable video coder cannot be equally efficient over a wide range of bit rates unless both the video data and the motion information are scalable. We propose a wavelet-based, highly scalable video compression scheme with rate-scalable motion coding. The proposed method involves the construction of quality layers for the coded sample data and a separate set of quality layers for the coded motion parameters. When the motion layers are truncated, the decoder receives a quantized version of the motion parameters used to code the sample data. The effect of motion parameter quantization on the reconstructed video distortion is described by a linear model. The optimal tradeoff between the motion and subband bit rates is determined after compression. We propose two methods to determine the optimal tradeoff, one of which explicitly utilizes the linear model. This method performs comparably to a brute force search method, reinforcing the validity of the linear model itself. Experimental results indicate that the cost of scalability is small. In addition, considerable performance improvements are observed at low bit rates, relative to lossless coding of the motion information.     

基于双向运动补偿的三维小波变换方法

提出了基于双向运动补偿的三维小波变换方法,与以往的三维小波变换方法相比较,该方法在三维小波视频编码方法中应用了双向的运动估计/补偿,进而实现了两种时域可扩展方式的组合,提高了视频编码的可扩展性。     

Unconstrained motion compensated temporal filtering (UMCTF) for efficient and flexible interframe wavelet video coding

We introduce an efficient and flexible framework for temporal filtering in wavelet-based scalable video codecs called unconstrained motion compensated temporal filtering (UMCTF). UMCTF allows for the use of different filters and temporal decomposition structures through a set of controlling parameters that may be easily modified during the coding process, at different granularities and levels. The proposed framework enables the adaptation of the coding process to the video content, network and end-device characteristics, allows for enhanced scalability, content-adaptivity and reduced delay, while improving the coding efficiency as compared to state-of-the-art motion-compensated wavelet video coders. Additionally, a mechanism for the control of the distortion variation in video coding based on UMCTF employing only the predict step is proposed. The control mechanism is formulated by expressing the distortion in an arbitrary decoded frame, at any temporal level in the pyramid, as a function of the distortions in the reference frames at the same temporal level. All the different scenarios proposed in the paper are experimentally validated through a coding scheme that incorporates advanced features (such as rate-distortion optimized variable block-size multihypothesis prediction and overlapped block motion compensation). Experiments are carried out to determine the relative efficiency of different UMCTF instantiations, as well as to compare against the current state-of-the-art in video coding.     

A Probabilistic Network for Facial Feature Verification

In this paper, we present a probabilistic approach to determining whether extracted facial features from a video sequence are appropriate for creating a 3D face model. In our approach, the distance between two feature points selected from the MPEG‐4 facial object is defined as a random variable for each node of a probability network. To avoid generating an unnatural or non‐realistic 3D face model, automatically extracted 2D facial features from a video sequence are fed into the proposed probabilistic network before a corresponding 3D face model is built. Simulation results show that the proposed probabilistic network can be used as a quality control agent to verify the correctness of extracted facial features.     

Modeling and Evaluating Non-shared Memory CELL/BE Type Multi-core Architectures for Local Image and Video Processing

Local processing, which is a dominant type of processing in image and video applications, requires a huge computational power to be performed in real-time. However, processing locality, in space and/or in time, allows to exploit data parallelism and data reusing. Although it is possible to exploit these properties to achieve high performance image and video processing in multi-core processors, it is necessary to develop suitable models and parallel algorithms, in particular for non-shared memory architectures. This paper proposes an efficient and simple model for local image and video processing on non-shared memory multi-core architectures. This model adopts a single program multiple data approach, where data is distributed, processed and reused in an optimal way, regarding the data size, the number of cores and the local memory capacity. The model was experimentally evaluated by developing video local processing algorithms and programming the Cell Broadband Engine multi-core processor, namely for advanced video motion estimation and in-loop deblocking filtering. Furthermore, based on these experiences it is also addressed the main challenges of vectorization, and the reduction of branch mispredictions and computational load imbalances. The limits and advantages of the regular and adaptive algorithms are also discussed. Experimental results show the adequacy of the proposed model to perform local video processing, and that real-time is achieved even to process the most demanding parts of advanced video coding. Full-pixel motion estimation is performed over high resolution video (720×576 pixels) at a rate of 30 frames per second, by considering large search areas and five reference frames.     

Coding of 12-bit video from thermal imaging systems

A number of advanced applications of digital video technology exist which require pixels to be represented with a resolution higher than the 8 bits that is used in most existing video coding standards. These applications include the video generated by many modern thermal imaging systems used in surveillance applications, which has a dynamic range of 12 bits. In this paper, we describe the modifications required to extend existing video coding standards to support surveillance applications, concentrating on applications with pixels whose dynamic range is up to 12 bits. We examine the tools necessary to implement directly a 12-bit coding algorithm, as well as a scalable coding approach using SNR scalability.     

Error concealment for SNR scalable video coding

This paper proposes an efficient error concealment method for SNR scalable coded video. The algorithm adaptively selects a proper concealment candidate from the base or the enhanced pictures to conceal the artifact of a lost enhancement block. To determine the best concealment candidate, we propose a trial process in which the concealment candidates are examined based on two criteria: (1) picture continuity at the border of concealed macroblocks, and (2) to satisfy the coding distortion bound of the base layer coefficients when they are available. For the latter, requantization of the concealed picture with the base layer quantizer step size and its dequantized pixels should result in zero distortion. We have implemented the method on a proposed SNR scalable H.264 video codec and compared the decoded video quality against just copying the base layer pixels into the enhanced picture. Simulation results show that the proposed method can achieve a considerable improvement by up to 3 dB especially in situations where the enhancement layer contains a large portion of the picture information. This will make scalable video transmission more successful over unreliable channels.     

Effective computation-aware algorithm by inter-layer motion analysis for scalable video coding

Scalable video coding incorporated with computation-aware ability achieves quality as well as being computation scalable. This paper presents a computation-aware algorithm for scalable video coding with spatial/quality scalability aiming for the best trade-off between rate distortion performance and computational consumption. We first observe and analyze and then establish a model for the motion vector difference relationship between the scalable base and enhancement layers. By using the modeling results, a linear algorithm for computation distribution is thus proposed to allocate the computation for each macroblock in the enhancement layer. In addition, the rate distortion costs of the base layer are also taken into account for the computation allocation process in order to further improve the coding performance. The simulation results demonstrate that our proposed computation-aware algorithm not only accomplishes better rate distortion performance than other works under the same computational constraints, but also achieves less computation necessities.     

Wavelet-based coding of time-varying vector fields of ocean-surface winds

Geoscience applications often produce sizable datasets that are vector-valued and increasingly in need of compression algorithms to reduce storage and transmission burdens, particularly when the data are time-varying. In this paper, several advanced interframe-compression techniques are extended from the traditional realm of natural video to the coding of time-varying vector fields. Although similar to natural video in some respects, time-varying vector-field sequences often possess complex temporal evolution of vector-valued features that are important to the analytic quality of the data yet defy the simple motion models widely employed for natural video. To improve coding performance, motion compensation with reduced resolution is proposed such that motion compensation is applied only at low spatial resolution, while high-resolution information, for which the motion model fails, is intraframe coded with no temporal decorrelation. In empirical results on datasets of ocean-surface winds, this reduced-resolution motion-compensation technique results in significant performance improvement and greater feature preservation.