Block-based motion field segmentation for video coding

In the past few years, motion compensation has been widely used in the coding of image sequences. Most of motion estimation and compensation schemes belong to block-based framework. The framework simplifies the complexity of motion estimation, but gives over constraints to the motion field, which results in worse accuracy on the boundary of moving objects. This paper presents a novel technique for raising motion field accuracy. It uses several pre-defined pattern types to segment the motion fields of the previous frame of a sequence. The segmentation is based on the MAP framework that uses iterative method to obtain the solution. In addition, we develop a predictive scheme to predict the location of motion field discontinuities in the current frame, which further reduces the side information for the representation of segmentation.     

Spatio-temporal segmentation of image sequences for object-oriented low bit-rate image coding

This paper describes a new method of segmentation of time-varying image sequences whose goal is object-oriented image coding. The segmentation represents a partition of each frame of the sequence into a set of regions which are homogeneous with regard to motion criterion. The region borders correspond to spatial contours of objects in the frame. Each spatio-temporal region is characterized by its temporal component, which is a model-dependent vector of motion parameters, and a structural component representing the polygonal approximation of the spatial contour of the region.

The construction of spatio-temporal segmentation includes two phases: the initialization step and temporal tracking. The initialization step is based on the spatial segmentation of the first frame of the sequence. Then homogeneous spatial regions are merged through motion estimation in accordance with a motion-based criterion. The temporal tracking consists of the projection of the segmentation along the time axis, and its adjustment. Special attention is paid to the processing of occlusions.

A predictive coding scheme is proposed which is based on the temporal coherence of the segmentation. This scheme is promising for a low bit-rate image compression.

The results for teleconference and TV sequences show the high visual quality of reconstructed only by prediction images. Moreover, the bit-rates for motion coding are very low: from 0.002 to 0.007 bit/pixel for teleconference sequence and from 0.004 to 0.021 bit/pixel for complex TV sequence. A scheme for encoding of the structural information is proposed which requires 0.083 – 0.17 bit per pixel depending on the content of the sequence.     

基于运动矢量多级分析的视频全局运动估计

基于运动矢量场的视频全局运动估计相较于基于像素的估计方法具有较低的计算复杂度,因而广泛应用于视频分割及视频压缩等领域中。然而噪声和前景目标等外点区域的存在,降低了全局运动估计的准确性。为了提高全局运动估计的准确度,该文提出一种基于运动矢量多级分析的全局运动估计算法,该算法根据局部运动与全局运动的运动特性差异自适应地滤除前景目标区域,由邻域矢量间相似性度量检测出纹理平滑周期区域,最后滤除孤立的噪声区域,由滤波得到的内点区域求解全局运动参数。实验结果表明,该方法能有效地滤除外点区域,提高全局运动估计的准确性。     

Improvement of region-based motion estimation by considering uncovered regions

In natural video sequences, object movement causes regions to be covered or uncovered. Conventional algorithms for region-based motion estimation do not take uncovered regions into full account. Uncovered regions seriously decrease the accuracy of motion estimation. This paper presents an algorithm for increasing the motion estimation accuracy. This algorithm detects uncovered regions and uses them to improve image segmentation and motion estimation. Experimental results show that the presented algorithm is effective in reducing the displaced frame difference, without introducing any extra information for coding applications.     

一种适用于遮挡问题的运动补偿帧插值方法

针对二维运动估计中常出现的遮挡问题,提出一种基于网格模型的运动补偿自适应帧插值技术,将图像帧中分为3种不同区域,根据各自的特点分别进行运动估计和运动补偿插值。在准确检测遮挡区域前提下,单独对遮挡区域处理有效地减小了运动估计误差。提出采用特征窗口运动补偿匹配的方法进行网格节点运动矢量估计,消除了传统块匹配方法导致的块效应,得到了亚像素的匹配精度。实验证明,该方法简单易行,有效地解决了遮挡区域的运动估计问题,插值图像具有较好的视觉效果。     

Motion-based region growing segmentation of image sequences

This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching algorithm to estimate motion vectors and uses morphological tools such as open-close by reconstruction and the region-growing version of the watershed algorithm for spatial segmentation to improve the temporal segmentation. In order to determine the reliable motion vectors, this paper also proposes a change detection algorithm and a multi-candidate pro- screening motion estimation method. Preliminary simulation results demonstrate that the proposed scheme is feasible. The main advantage of the scheme is its low computational load.     

Adaptive Least Squares Interpolation of Infrared Images

This paper presents an adaptive block-by-block least squares (LS) algorithm for the interpolation of infrared (IR) images. The suggested algorithm is based on the segmentation of the low resolution (LR) image into overlapping blocks and the interpolation of each block, separately. The purpose of the overlapping is to avoid edge effects between blocks. An iterative implementation of the proposed algorithm, which considers the image acquisition model, is used for the minimization of the estimation error in each block. A weight matrix of moderate dimensions is estimated in a small number of iterations to interpolate each block. This proposed algorithm avoids the large computational complexity resulting from the matrices of large dimensions required to interpolate the image as a whole. The performance of the proposed algorithm is compared with the standard as well as the warped distance optimal interpolation of maximal order with minimal support (O-MOMS) algorithm from the peak signal-to-noise ratio (PSNR) point of view. Numerical results reveal the superiority of the proposed LS algorithm to the cubic O-MOMS algorithm.     

基于网格模型的运动估计技术

网格模型的运动估计和跟踪在基于模型的编码方法中占有重要的地位,在现有的网格运动估计技术中,六边形匹配(Hexagonal Matching)是一种有效的网格模型运动估计算法,它采用局部最优和迭代策略相结合的方法得到较好的全局运动估计结果.六边形匹配算法由块搜索和迭代六边形匹配两步组成,同所有的基于模型的运动估计技术一样,六边形匹配算法也存在运算量大的问题难于满足实时编码需要.本文对六边形匹配算法从三个方面进行改进:用四步搜索替代全搜索、快速的纹理映射技术和有效的节点排序减少迭代次数.改进后的算法在基本上保证原算法的运动估计效率的情况下,算法的复杂度有明显的降低,使得基于模型的编码技术向实用化前进了一步.     

Multiresolution Vector Quantization for Video Coding

In this work, we propose a coding technique that is based on the generalized block prediction of the multiresolution subband decomposition of motion compensated difference image frames. A segmentation mask is used to distinguish between the regions where motion compensation was effective and those regions where the motion model did not succeed. The difference image is decomposed into a multiresolution pyramid of subbands where the highest resolution subbands are divided into two regions, based on the information given by the segmentation mask. Only the coefficients of the regions corresponding to the motion model failure are considered in the highest resolution subbands. The remaining coefficients are coded using a multiresolution vector quantization scheme that exploits inter-band non-linear redundancy. In particular, blocks in one subimage are predicted from blocks of the adjacent lower resolution subimage with the same orientation. This set of blocks plays the role of a codebook built from coefficients inside the subband decomposition itself. Whenever the inter-band prediction does not give satisfactory results with respect to a target quality, the block coefficients are quantized using a lattice vector quantizer for a Laplacian source.     

一种基于区域搜索的全局运动估计与补偿算法

开展全局运动估计与补偿研究是进行动态目标检测中的基础和前提。在总结现有运动估计与补偿方法的基础上,提出一种基于图像分割区域的运动性和大小的全局运动估计与补偿算法。首先,通过建立区域搜索的全局运动模型,同时进行区域定性分割和区域大小排序;然后,根据误差最小化准则在指定的分割区域中进行线性递归搜索,利用门限准则寻找出最佳的运动估计参数;最后,根据双线性内插法获得运动位移量。实验结果比较可知,所提算法较三步搜索算法(TSS)和全局搜索算法(FSA)等传统算法具有更高的准确性(图像平移帧差)和实时性(算法运行时间),能够很好地实现运动背景的全局校正。     

基于特征点匹配技术的运动估计及补偿方法

提出一种用于电子稳像技术的全局运动估计和补偿算法。通过特征点匹配,求其局部运动矢量;将其代入变换模型所得的线性力程组,求其全局运动矢量。用均值滤波的方法确定各帧的补偿量,并代入给定的变换模型对当前图像进行变换,实现对视频图像序列的稳定处理。为了减小特征点匹配计算的复杂性,加快匹配速度,还采用了多分辨率图像金字塔匹配策略。实验表明．使用该算法对于提高动态图像的稳定性有较好的效果。     

一种用于视频分割的快速运动估计方法

本文在考虑视频分割特点的基础上,提出了一种分层块匹配的运动估计方法,这处估计方法具有较快的估计速度和较高的峰值信噪比。用这种方法对一帧图像进行一次运行估计约需0．1秒,可以用于实时的视频分割系统中。实验证明这是一种性能良好的运动估计方法。     

基于运动矢量对消和差分原理的快速全局运动估计

该文针对全局运动估计计算复杂度过高的问题,提出了一种基于运动矢量对消和差分原理的快速全局运动估计方法。基于不同象限运动矢量对之间存在的对称抵消特性,首先估计出平移运动参数分量;再使用运动矢量对的差分原理,并结合一种置信判断的策略,估计出变换运动参数分量。在5组几何全局运动模型和真实视频序列上的实验结果表明,与基于迭代最小二乘(ILS)和基于偏导数(PD)的估计算法相比,该方法在保证估计精度的情况下,所需的平均计算时间只要它们的50％左右。     

一种适用于H.263的运动估计搜索算法

本文提出了一种用于视频编码快速运动估计的基于中心三步法（ＣＴＳＳ）的块匹配搜索算法。结果表明,这种算法与Ｈ．２６３的可选项结合使用时,大大减少了块匹配的计算量,并可获得与全搜索算法相当的图像质量、信噪比和编码比特数。     

Three-dimensional parameter estimation from stereo image sequences for model-based image coding

It is shown that the analysis of moving image sequences for 3D modelling can be performed in a relatively straightforward manner if the scene is captured in stereo. Output from a stereo disparity estimation process using calibrated cameras gives absolute 3D surface coordinates from a single stereo pair. When combined with monocular motion cues, the true 3D motion parameters of moving objects can be accurately calculated. Further analysis enables segmentation of body elements according to motion while the 3D surface feature structure, although available from the start, can be integrated and checked for anomalies over the sequence. These results are expected to alleviate the known problems of ambiguity suffered by monocular-source model-based coders.     

Motion‐Compensated Frame Interpolation Using a Parabolic Motion Model and Adaptive Motion Vector Selection

We propose a motion‐compensated frame interpolation method in which an accurate backward/forward motion vector pair (MVP) is estimated based on a parabolic motion model. A reliability measure for an MVP is also proposed to select the most reliable MVP for each interpolated block. The possibility of deformation of bidirectional corresponding blocks is estimated from the selected MVP. Then, each interpolated block is produced by combining corresponding blocks with the weights based on the possibility of deformation. Experimental results show that the proposed method improves PSNR performance by up to 2.8 dB as compared to conventional methods and achieves higher visual quality without annoying blockiness artifacts.     

一种新型基于运动补偿的自适应去隔行算法

去隔行是电视图像增强处理和视频格式转换的一项关键技术，传统的去隔行算法存在着难以克服的缺陷。文章提出了一种基于运动补偿的自适应去隔行算法，其中包括双向块匹配运动估计，消除块效应和自适应插值三个处理模块，实验证明本算法能够高质量且鲁棒地将场图像重建为帧。     

Resolution enhancement of monochrome and color video using motioncompensation

We propose an iterative algorithm for enhancing the resolution of monochrome and color image sequences. Various approaches toward motion estimation are investigated and compared. Improving the spatial resolution of an image sequence critically depends upon the accuracy of the motion estimator. The problem is complicated by the fact that the motion field is prone to significant errors since the original high-resolution images are not available. Improved motion estimates may be obtained by using a more robust and accurate motion estimator, such as a pel-recursive scheme instead of block matching, in processing color image sequences, there is the added advantage of having more flexibility in how the final motion estimates are obtained, and further improvement in the accuracy of the motion field is therefore possible. This is because there are three different intensity fields (channels) conveying the same motion information. In this paper, the choice of which motion estimator to use versus how the final estimates are obtained is weighed to see which issue is more critical in improving the estimated high-resolution sequences. Toward this end, an iterative algorithm is proposed, and two sets of experiments are presented. First, several different experiments using the same motion estimator but three different data fusion approaches to merge the individual motion fields were performed. Second, estimated high-resolution images using the block matching estimator were compared to those obtained by employing a pel-recursive scheme. Experiments were performed on a real color image sequence, and performance was measured by the peak signal to noise ratio (PSNR).