Overlapped block motion compensation: an estimation-theoreticapproach

We present an estimation-theoretic analysis of motion compensation that, when used with fields of block-based motion vectors, leads to the development of overlapped block algorithms with improved compensation accuracy. Overlapped block motion compensation (OBMC) is formulated as a probabilistic linear estimator of pixel intensities given the limited block motion information available to the decoder. Although overlapped techniques have been observed to reduce blocking artifacts in video coding, this analysis establishes for the first time how (and why) OBMC can offer substantial reductions in prediction error as well, even with no change in the encoder's search and no extra side information. Performance can be further enhanced with the use of state variable conditioning in the compensation process. We describe the design of optimized windows for OBMC. We also demonstrate how, with additional encoder complexity, a motion estimation algorithm optimized for OBMC offers further significant gains in compensation accuracy. Overall mean-square prediction improvements in the range of 16 to 40% (0.8 to 2.2 dB) are demonstrated     

Motion estimation methods for overlapped block motion compensation

An extension of conventional block motion compensation (BMC), overlapped block motion compensation (OBMC) has been shown to reduce residual errors and blocking effects in motion-compensated video. However, the overlap creates a noncausal spatial dependence between blocks and complicates motion estimation (ME) for OBMC. Iterative methods have traditionally been employed for overlapped block motion estimation (OBME). For compression, the rate for the motion vector field (MVF) may also be constrained. This work considers several rate-constrained OBME algorithms, both iterative and noniterative. Experiments demonstrate that a simple raster-scan algorithm is effective as a suboptimal, noniterative solution, with comparable or better rate-distortion performance and computational complexity than iterative OBME algorithms. Depending on the application, either this method or a simple block-matching algorithm plus iteration are the most attractive of the tested OBME schemes.     

Implementation of Multiple Macroblock Mode Overlapped Block Motion Compensation for Wavelet Video Coding

This paper proposes a hybrid video coding framework based on discrete wavelet transform and multiple macroblock mode overlapped block motion compensation (OBMC). Studying the neighborhood in multiple macroblock mode situations, we set rules for the selection of neighboring blocks in OBMC. Based on experimental results, the raised cosine window and the best neighboring blocks are chosen for their outstanding performance. The multiple macroblock mode OBMC has a good performance in reducing the blocking effect, and the coding efficiency is improved. The proposed scheme can provide a 1 dB coding gain compared to MPEG-4, and its performance is close to the latest H.264 standard. From a subjective point of view, the new framework is also better than the standards.     

Analysis of space-dependent characteristics of motion-compensatedframe differences based on a statistical motion distribution model

Although it has been observed that motion-compensated frame differences increase toward block boundaries and overlapped block motion compensation (OBMC) has been shown to provide reduced blocking artifacts as well as improved prediction accuracy, there is almost no satisfactory theoretical basis that clearly interprets the space-dependent characteristics of motion-compensated frame differences, nor have the theoretical aspects of OBMC been investigated thoroughly. We first interpret the space-dependent characteristics of motion-compensated frame differences based on a novel statistical motion distribution model. We then apply the statistical motion distribution model to the analysis of prediction efficiency of OBMC. Through the analysis, we prove theoretically that OBMC can reduce and equalize the motion-compensated frame differences across a block. The analytical results are justified by empirical experiments with typical image sequences.     

Motion-vector optimization of control grid interpolation andoverlapped block motion compensation using iterated dynamic programming

The application of advanced motion compensation techniques-control grid interpolation (CGI) and overlapped block motion compensation (OBMC)-to video coding systems provides significant performance advantages, terms of compression ratio and visual quality, over traditional block-matching motion compensation. However, the two-dimensional (2-D) interdependence among motion vectors introduced by these compensation frameworks makes the problem of finding rate-distortion optimal motion vectors, computationally prohibitive. Thus, iterative optimization techniques are often used to achieve good compensation performance. While most reported optimization algorithms adopt an approach that uses a block-matching algorithm to obtain an initial estimate and then successively optimize each motion vector, the over-relaxed motion-vector dependency relations often result in considerable performance degradation. In view of this problem, we present a new optimization scheme for dependent motion-vector optimization problems, one based on dynamic programming. Our approach efficiently decomposes 2-D dependency problems into a series of one-dimensional (1-D) dependency problems. We show that a reliable initial estimate of motion vectors can be obtained efficiently by only considering the dependency in the rate term. We also show that at the iterative optimization stage an effective logarithmic search strategy can be used with dynamic programming to reduce the necessary complexity involved in distortion computation. Compared to conventional iterative approaches, our experimental results demonstrate that our algorithm provides superior rate and distortion performance while maintaining reasonable complexity.     

Analysis of spatial transform motion estimation with overlappedcompensation and fractional-pixel accuracy

The authors investigate how the performance of spatial transform motion estimation can be significantly improved by incorporating overlapped compensation and fractional-pixel accuracy. An overlapped spatial transformation (OST) motion model is developed, which successfully addresses the inability of the conventional block matching technique to compensate for complex motion and inside-block motion field discontinuities. Simulation results show that the motion compensated prediction error of this method is reduced by 1.1 dB, compared with the conventional overlapped block matching motion estimation, for the same generated motion vectors overhead. To improve the performance, the overlap must be used in both the motion estimation and compensation processes. Further improvement can be obtained using half-pixel precision motion vectors. However, this improvement is comparatively less than the gain of conventional block matching from a half-pixel search     

New windowing techniques for variable-size block motioncompensation

The conventional overlapped block motion compensation (OBMC) technique, although effective in reducing the blocking effect for fixed-size partitioned frames, cannot be readily used for a frame partitioned region-wise or using variable-size blocks. The generalised OBMC (GOBMC) method, even though generally applicable, is not very effective in reducing the blocking effect and prediction error. Two windowing techniques are presented to reduce the blocking effect for a frame partitioned region-wise or using variable-size blocks. In the first technique, a virtual re-partitioning operation is employed, which maps a partitioned frame into its corresponding fully partitioned frame at the bottom level of the quadtree so that each resulting block has eight neighbouring blocks. In the second technique, the virtual re-partitioning operation for a given block (region) is carried out adaptively and performed locally; however, blocks (regions) need not always be virtually partitioned to the bottom level. Compared to the GOBMC method, the proposed techniques make use of more pixels in the close vicinity of boundaries of the regions in the windowing operation. Simulation results are included of applying the proposed techniques on a number of MPEG video sequences. These results indicate that the proposed techniques are superior to the GOBMC method in terms of reducing the prediction error as well as the blocking effect     

Efficiency analysis of multihypothesis motion-compensatedprediction for video coding

Overlapped block motion compensation or B-frames are examples of multihypothesis motion compensation where several motion-compensated signals are superimposed to reduce the bit-rate of a video codec. This paper extends the wide-sense stationary theory of motion-compensated prediction (MCP) for hybrid video codecs to multihypothesis motion compensation. The power spectrum of the prediction error is related to the displacement error probability density functions (pdfs) of an arbitrary number of hypotheses in a closed-form expression. We then study the influence of motion compensation accuracy on the efficiency of multihypothesis motion compensation as well as the influence of the residual noise level and the gain from optimal combination of N hypotheses. For the noise-free limiting case, doubling the number of (equally good) hypotheses can yield a gain of up to 1/2 bits/sample, while doubling the accuracy of motion compensation can additionally reduce the bit-rate by up to 1 bit/sample independent of N. For realistic noise levels, it is shown that the introduction of B-frames or overlapped block motion compensation can provide larger gains than doubling motion compensation accuracy. Spatial filtering of the motion-compensated candidate signals becomes less important if more hypotheses are combined. The critical accuracy beyond which the gain due to more accurate motion compensation is small moves to larger displacement error variances with increasing noise and increasing number of hypotheses N. Hence, sub-pel accurate motion compensation becomes less important with multihypothesis MCP. The theoretical insights are confirmed by experimental results for overlapped block motion compensation, B-frames, and multiframe motion-compensated prediction with up to eight hypotheses from ten previous frames.     

Frequency-domain motion estimation using a complex lapped transform

A frequency-domain algorithm for motion estimation based on overlapped transforms of the image data is developed as an alternative to block matching methods. The complex lapped transform (CLT) is first defined by extending the lapped orthogonal transform (LOT) to have complex basis functions. The CLT basis functions decay smoothly to zero at their end points, and overlap by 2:1 when a data sequence is transformed. A method for estimating cross-correlation functions in the CLT domain is developed. This forms the basis of a motion estimation algorithm that calculates vectors for overlapping, windowed regions of data. The overlapping data window used has no block edge discontinuities and results in smoother motion fields. Furthermore, when motion compensation is performed using similar overlapping regions, the algorithm gives comparable or smaller prediction errors than standard models using exhaustive search block matching, and computational load is lower for larger displacement ranges and block sizes.     

立体图像分层交叠块视差估计与自适应补偿

视差估计与补偿是立体图像编码的关键问题,本文提出了一种基于交叠块匹配的视差估计与补偿算法。基于交叠块匹配的分层视差估计,将块分割和平滑约束融入视差估计,可得到光滑准确的视差矢量场。自适应交叠块视差补偿可克服通常的交叠块视差补偿的过平滑效应,在相同的编码比特率下,与块补偿及OBDC相比,AOBDC可获得2.4 dB及0.9 dB的峰值信噪比增益。     

Neural fuzzy motion estimation and compensation

Neural fuzzy systems can improve motion estimation and compensation for video compression. Motion estimation and compensation are key parts of video compression. They help remove temporal redundancies in images. But most motion estimation algorithms neglect the strong temporal correlations of the motion field. The search windows stay the same through the image sequences and the estimation needs heavy computation. A neural vector quantizer system can use the temporal correlation of the motion field to estimate the motion vectors. First- and second-order statistics of the motion vectors give ellipsoidal search windows. This algorithm reduced the search area and entropy and gave clustered motion fields. Motion-compensated video coding further assumes that each block of pixels moves with uniform translational motion. This often does not hold and can produce block artifacts. We use a neural fuzzy system to compensate for the overlapped block motion. This fuzzy system uses the motion vectors of neighboring blocks to map the prior frame's pixel values to the current pixel value. The neural fuzzy system used 196 rules that came from the prior decoded frame. The fuzzy system learns and updates its rules as it decodes the image. The fuzzy system also improved the compensation accuracy. The paper derives both the fuzzy system and the neural learning laws that tune its parameters     

基于仿射模型的重叠块运动估计补偿方法

本文对 Ｈ．２６３中基于平移模型的重叠块运动估计补偿方法进行改进,提出了一种基于仿射模型的重叠块运动补偿方法,该方法对复杂运动图像质量有明显改善。     

可变尺寸块运动估计的R—D优化

本文应用率失真理论在固定码率的前提下对可变尺寸块运动估计过程进行R－D优化,获得了最佳预测块划分和最小运动补偿误差,并提出了基于树形结构的运动矢量编码方案。R－D优化问题的求解通过引入失真度－码率梯度,将R－D优化与分层运动估计结合在一起,克服了GP－BFOS算法需要对各个分块层次进行运动估计及所得局部最优解严格位于R－D曲线凸包络线上的弊病。软件模拟结果显示本算法大大降低了G－BFOS优化算法的计算量,且运动补偿RSNR值高于G－BFOS算法结果。     

一种新的小波可视电话编码算法

针对极低比特率应用提出一种新的结合H.263与SLCCA的混合小波视频编码算法。在提出的算法中,首先,用基于H.263的微调运动估计减小时间冗余,用无遗漏覆盖块运动补偿保证运动补偿误差帧的连续性;第二,对运动补偿误差帧进行小波变换得到全局能量压缩;第三,用SLCCA组织和表示小波变换后的数据,最后,运动向量的水平和垂直分量分别用自适应算法编码,算法在A级测试序列Akiyo和B级测试序列Foreman（QFIF）上测试取得了良好效果。     

A spatiotemporal super-resolution algorithm for a hybrid stereo video system

This paper presents a spatiotemporal super-resolution method to enhance both the spatial resolution and the frame rate in a hybrid stereo video system. In this system, a scene is captured by two cameras to form two videos, including a low spatial resolution with high-frame-rate video and a high spatial resolution with low-frame-rate video. For the low-spatial-resolution video, the low-resolution frames are spatially super-resolved by the high-resolution video via the stereo matching, the bilateral overlapped block motion estimation, and the adaptive overlapped block motion compensation algorithms, while for the low-frame-rate video, those missed frames are interpolated using the high-resolution frames obtained by fusing the disparity compensation and the motion compensation frame rate up-conversion. Experimental results demonstrate that the proposed mixed spatiotemporal super-resolution method has a more significant contribution to both the subjective and objective qualities than the pure spatial super-resolution or the frame rate up-conversion.     

基于运动相似性的帧率提升算法

为提升低比特率通信接收端的视频帧率,解决快速运动中的图像模糊问题,提出了一种基于运动相似性的帧率提升算法,其三维递归搜索来减少运动估计的计算复杂度,利用自适应的判决窗口减少非相似运动对运动矢量预测的影响,提高运动估计的精度;同时构建运动相似因子,对不同的运动相似块进行不同的运动矢量细化操作,减小块效应;利用对应的相似运动窗口来进行重叠块补偿插值,提高插值的精度.实验结果表明,该算法在主客观方面有较好的性能提升.     

Encoding of images based on a lapped orthogonal transform

Traditional block transform image coding systems generate artifacts near block boundaries which degrade low bit rate coded images. To reduce these artifacts, a class of unitary transformations, called lapped orthogonal transforms (LOT), is investigated. The basis function on which the signal is projected are overlapped for adjacent blocks. An example of an LOT optimized in terms of energy compaction is numerically derived, using an augmented Lagrangian optimization algorithm. Using this LOT, intraframe coding experiments for 256×240 pixel images were performed at bit rates between 0.1 and 0.35 bits/pixel. The LOT improved the coded image subjective quality over other transforms. The LOT was also used in interframe full-motion video coding experiments for head and shoulder sequences at 28 and 56 kb/s. Significant improvement resulted at low data rates and if no motion compensation were used. However, the improvement was no longer significant at 56 kb/s with full motion compensation     

Wavelet-based very low bit-rate video coding using image warpingand overlapped block motion compensation

The authors present an algorithm for very low bit-rate video coding that combines new ideas in motion estimation, wavelet filter design, and wavelet-based coding techniques. A new motion compensation technique using image warping and overlapped block motion compensation is proposed to reduce temporal redundancies in a given image sequence. This combined motion model has the advantage of representing more complex motion than simple block matching schemes. To further improve the quality of the temporal prediction, an adaptive grid with variable density according to the varying motion activity of a given scene is generated. An adaptively switched high-quality texture interpolation is employed to cope with the problem of fractional displacements in such a way that both objective and subjective reconstruction quality is improved. Spatial decorrelation of the motion compensated residual images is performed using an one-parametric family of biorthogonal infinite impulse response (IIR) wavelet filters coupled with the highly efficient pre-coding scheme of `partitioning, aggregation and conditional coding' (PACC). Experimental results demonstrate significant improvements in objective quality of 1.0-2.3 dB PSNR in comparison to the H.263+ test model TMN10 using advanced coding options. In addition, the authors' intracoding method provides a performance gain of 0.5 dB PSNR on the average for a test suite of various still images when compared to the emerging still image coding standard JPEG-2000     

一种新型基于运动补偿的自适应去隔行算法

去隔行是电视图像增强处理和视频格式转换的一项关键技术，传统的去隔行算法存在着难以克服的缺陷。文章提出了一种基于运动补偿的自适应去隔行算法，其中包括双向块匹配运动估计，消除块效应和自适应插值三个处理模块，实验证明本算法能够高质量且鲁棒地将场图像重建为帧。     

基于纹理图像的MPEG-4运动补偿电路的VLSI设计

在采用外部存储和内部缓存的两级存储方案的基础上,提出了一种基于纹理图像的MPEG-4ASP@L5运动补偿电路的硬件结构,并完成了VLSI设计。针对运动向量的预测算法,在满足实时译码的前提下对电路的内部缓存LM2进行了优化。对于重叠块运动补偿算法,提出了一种有效的双循环替换缓存结构。采用TSMC0.25μm1P5MCMOS工艺,完成了运动补偿电路的VLSI实现,芯片内核面积为1.31mm×1.31mm,最高工作频率150MHz。系统仿真结果表明该电路可在120MHz的频率下对符合ASProfile标准的ITU-R601格式的纹理视频流进行实时运动补偿。