Level-set-based motion estimation algorithm for multiple reference frame motion estimation

Motion estimation (ME) has a variety of applications in image processing, pattern recognition, target tracking, and video compression. In modern video compression standards such as H.264/AVC and HEVC, multiple reference frame ME (MRFME) is adopted to reduce the temporal redundancy between successive frames in a video sequence. In MRFME, the motion search process is conducted using additional reference frames, thereby obtaining better prediction signal as compared to single reference frame ME (SRFME). However, its high computational complexity makes it difficult to be utilized in real-world applications. In order to reduce the computational complexity of MRFME, this paper proposes a level-set-based ME algorithm (LSME) without any penalty in the rate-distortion (RD) performance. First, the proposed algorithm partitions the motion search space into multiple level sets based on a rate constraint. The proposed algorithm then controls the ME process on the basis of the predetermined level sets. Experimental results show that the proposed algorithm reduces the ME time by up to 83.46% as compared to the conventional full search (FS) algorithm.     

Data reuse algorithm for multiple reference frame motion estimation

A data reuse algorithm for multiple reference frame motion estimation is described. The proposed algorithm reduces memory access by modifying the reference frame search order and search centre such that the likelihood of data reuse is increased. Experimental results show that the algorithm reduces memory access by 15-30% compared to the conventional fast reference frame selection algorithm, while maintaining similar bit rate and PSNR     

Fast motion estimation and mode decision for H.264 reverse transcoding

A fast reverse motion estimation algorithm with smart mode decision for H.264 reverse transcoding is proposed. The best mode for the backward transcoded macroblock is obtained by analysing the motion vectors and modes decoded from the forward bitstream. A significant reduction in the transcoding complexity with negligible impact on rate-distortion performance has been achieved     

A dynamic search pattern motion estimation algorithm using prioritized motion vectors

Motion estimation is one of the critical parts in video compression standards with a high computational load. Many motion estimation algorithms have been developed to reduce the number of search points compared to a full-search algorithm without losing the quality considerably. Most of them use fixed search patterns in their first step which may suffer from trapping into local minima or searching unnecessary blocks due to inappropriate size and type of search patterns. In this paper, a new dynamic search pattern using motion vectors of spatial and temporal neighboring blocks is proposed. The motion vectors of neighboring blocks are prioritized, in order to efficiently use of halfway stop technique. The simulation results indicate that proposed algorithm is very close to the full-search algorithm in quality, compared to other rivals. Moreover, the average number of searches is often less than other algorithms.     

Power efficient SDS motion estimation architecture using dynamic iteration control and hierarchical adder compressors for real time HDTV video coding

This paper presents a high performance, power efficient and low hardware cost architecture for motion estimation (ME) targeting portable consumer applications. This hardware uses the Sub-sampled Diamond Search algorithm (SDS) with a Dynamic Iteration Control (DIC). The SDS–DIC algorithm can significantly reduce the number of SAD (Sum of Absolute Difference) calculations for block matching, thus enabling the development of an efficient hardware design for the ME. The DIC technique allows for the required throughput to be achieved with a restriction in the number of iterations, which contributes to the reduction in the overall number of clock cycles needed for the motion vector calculation. The processing units (PU) of the ME were developed by using efficient hierarchical adder-compressors, where simultaneous additions of more than two operands can be performed. The results we present show that, by using both the adder compressors in the PU and the DIC technique, it is possible to obtain an efficient ME architecture with higher performance and reduced power consumption. The architecture that implements this algorithm and the PUs was described in VHDL. Hardware synthesis results are presented for a 0.18 μm CMOS standard cell library. The architecture can reach real time for HDTV 1080p with less than 40 mW of power consumption.     

Frequency-domain motion estimation using a complex lapped transform

A frequency-domain algorithm for motion estimation based on overlapped transforms of the image data is developed as an alternative to block matching methods. The complex lapped transform (CLT) is first defined by extending the lapped orthogonal transform (LOT) to have complex basis functions. The CLT basis functions decay smoothly to zero at their end points, and overlap by 2:1 when a data sequence is transformed. A method for estimating cross-correlation functions in the CLT domain is developed. This forms the basis of a motion estimation algorithm that calculates vectors for overlapping, windowed regions of data. The overlapping data window used has no block edge discontinuities and results in smoother motion fields. Furthermore, when motion compensation is performed using similar overlapping regions, the algorithm gives comparable or smaller prediction errors than standard models using exhaustive search block matching, and computational load is lower for larger displacement ranges and block sizes.     

A fast full search motion estimation algorithm using sequentialrejection of candidates from hierarchical decision structure

We propose a new as well as fast full search (FS) motion estimation algorithm for video coding. The computational reduction comes from sequential rejection of impossible candidates with derived formula and subblock norms. Our algorithm reduces more the computations than the FS motion estimation algorithms previously developed     

MPEG—2运动估计的实时实现

运动估计是ＭＰＥＧ图像编码中提高压缩率最有效的方法,也是运算量最大的部分,因而也是较难实时实现的。针对“高清晰度电视功能样机”信源编码的要求,就全局运动做主方法提出了相应的硬件设计,并且完成了其硬件实现。     

Nonuniform image motion estimation using the maximum a posterioriprinciple

An iterative scheme for frame-to-frame motion estimation from a pair of noisy images is established. The algorithm is developed by assuming that the Karhunen-Loeve coefficients of the motion vector waveform are zero mean and Gaussian random variables. Following the derivation of the generalized maximum likelihood (GML) algorithm, and invoking the maximum a posteriori (MAP) criterion, an iterative motion estimator is developed. A linear analysis of the algorithm is presented, and the convergence of the algorithm is discussed. Simulation experiments are performed and comparisons are made with the GML algorithm the algorithm reported by A.N. Netravali and J.D. Robbins (1979), and the scheme developed by K.P.G. Horn and G.G. Schunck (1981).     

Improved interpolation, motion estimation, and compensation forinterlaced pictures

This paper introduces a new method for the interpolation that has to be performed when motion estimation and compensation are applied to interlaced sequences with subpel accuracy. It is based on the assumption that a uniform motion exists between two successive frames. The exact formulas for the estimation are derived. They show that in order to obtain a correct interpolation of each field of one frame, it is necessary to use the information of both fields of this frame. Because the ideal filters have infinite impulse responses, the filter design is discussed, and the efficiency is measured for typical sequences.     

Fast reference frame and inter-mode selection method for H.264/AVC

This paper presents an efficient method for selecting the best reference frame (RF) and the best inter-mode in H.264/AVC. The proposed method employs a different examination order compared to conventional methods and examines the most important RF for all inter-modes in the first step. Then, in the second step, it further examines the best mode from the first step for all available RFs. Finally, in the third step, it additionally examines the remaining modes for the best RF determined in the second step. Based on the intermediate results after the first and second steps, the proposed method skips many less important RFs in each inter-mode and significantly reduces the coding complexity. However, the proposed method also shows very good performance in terms of the peak signal-to-noise ratio and bit rate, as it does not skip important RFs in each inter-mode. Simulation results show that the proposed method greatly reduces the encoding time while significantly improving the encoding performance of conventional methods.     

基于相对运动和纹理走向的快速帧间选择算法

针对H.264基于率失真优化算法的模式选择的高复杂度,提出了一种快速帧间预测算法:利用下层模式相对上层模式的运动矢量误差和宏块纹理方向来确定帧间模式的块是否要细分以及如何划分。这种帧间快速算法降低了50%以上的时间复杂度,而视频质量没有明显的下降,码率的增长也微乎其微。     

Dynamic time warping in classification and selection of motion capture data

Multidimensional Systems and Signal Processing - The paper is a comprehensive study on classification of motion capture data on the basis of dynamic time warping (DTW) transform. It presents both...     

On-line spectral estimation of nonstationary time series based onAR model parameter estimation and order selection with a forgettingfactor

A new method for on-line spectral estimation of nonstationary time series via autoregressive (AR) model construction is proposed. The method consists of on-line parameter estimation based on the recursive least squares ladder estimation algorithm with a forgetting factor and on-line order determination based on AIC with some modifications. The effectiveness of the proposed method is demonstrated by computer simulation study and applying to the actual data of electroencephalogram (EEG)     

Rate-driven key frame selection using temporal variation of visualcontent

A simple and fast method to select the assigned number of key frames in a video shot is presented. The algorithm selects the key frames so that the temporal variation of visual content within a video shot is equally distributed to each key frame. Simulation results on a real video sequence are shown to be in agreement with the human visual perception     

Characterization of cell deformation and migration using a parametric estimation of image motion

This paper deals with the spatio-temporal analysis of two-dimensional deformation and motion of cells from time series of digitized video images. A parametric motion approach based on an affine model has been proposed for the quantitative characterization of cellular movements in different experimental areas of cellular biology including spontaneous cell deformation, cell mitosis, individual cell migration and collective migration of cell populations as cell monolayer. The accuracy and robustness of the affine model parameter estimation, which is based on a multiresolution algorithm, has been established from synthesized image sequences. A major interest of our approach is to follow with time the evolution of a few number of parameters characteristic of cellular motion and deformation. From the time-varying eigenvalues of the affine model square matrix, a precise quantification of the cell pseudopodial activity, as well as of cell division has been performed. For migrating cells, the motion quantification confirms that cell body deformation has a leading role in controlling nucleus displacement, the nucleus itself undergoing a larger rotational motion. At the cell population level, image motion analysis of in vitro wound healing experiments quantifies the heterogeneous cell populations dynamics.     

基于运动估计的模糊图像盲复原

针对由于物体在不同方向移动速度不同导致退化的CCD图像空间移变特性的运动模糊,提出一种基于隔行扫描空间移变特性的CCD图像恢复方案。方法主要依据隔行扫描图像分为上下两场的特性,借助插值算法将单幅图像按场拆分为两幅存在时间差的图像,然后引入视频处理中基于图像序列确定运动向量的块匹配算法,估计模糊参数,从而实现应用图像序列算法估计单幅图像点扩散函数,最后利用最小二成方算法进行复原,其中引入了图像定向信息测量操作保持边缘。仿真实验结果表明,本文算法比以往算法具有大的信噪比(SNR)和图像保真度(IMFID),以及小的平均绝对差(ABSDIFF)和均方误差(MSE)。     

Fast and efficient motion estimation using diamond zonal-based algorithms

Motion estimation has always been an important part of video encoding systems because it can reduce temporal redundancy effectively and thus has significant impact on the bit rate and the output visual quality of the encoded sequence. Unfortunately, when using the brute-force full search algorithm, motion estimation consumes a very large portion of the encoding time. Previously, several algorithms have been proposed which try to reduce complexity, usually, with a significant loss in visual quality. Based on the diamond zonal search framework we introduced recently, we propose in this paper a novel algorithm called advanced diamond zonal search (ADZS), which was submitted to and well received by the Moving Pictures Experts Group (MPEG) standard committee for possible inclusion as an encoder optimization tool. ADZS was criticized in MPEG for using fixed thresholds, which may not be suitable for all video sequences. To address this issue, we further propose a threshold-adaptive version called threshold-adaptive advanced diamond zonal search (TAADZS). Simulation results verify the superior performance of ADZS and TAADZS over other fast algorithms and the robustness of TAADZS over ADZS.This project has been supported by the Research Grants Council of the Hong Kong Special Administrative Region, China (project no. HKUST6057/99E) and the Hong Kong Telecom Institute of Information Technology. Parts of this paper have been previously published in conference proceedings.