Super-resolution reconstruction of image sequence using multiple motion estimation fusion

Super-resolution reconstruction algorithm produces a high-resolution image from a low-resolution image sequence. The accuracy and the stability of the motion estimation (ME) are essential for the whole restoration. In this paper, a new super-resolution reconstruction algorithm is developed using a robust ME method, which fuses multiple estimated motion vectors within the sequence. The new algorithm has two major improvements compared with the previous research. First, instead of only two frames, the whole sequence is used to obtain a more accurate and stable estimation of the motion vector of each frame; second, the reliability of the ME is quantitatively measured and introduced into the cost function of the reconstruction algorithm. The algorithm is applied to both synthetic and real sequences, and the results are presented in the paper.     

利用运动强度判据的高效自适应运动估计算法

为减少运动估计计算量,提高视频编码效率,提出了基于运动强度的自适应运动估计搜索算法。该算法通过定义运动强度概念来反映帧间图像运动的剧烈程度,依据当前帧的运动强度信息预测下一帧运动情况,并自适应选择算法进行运动搜索:当运动强度高于设定阈值时选用UMHexagonS算法,低于该阈值时选用改进的六边形算法。实验仿真结果表明,该算法能在保证图像质量和压缩效果的基础上,大幅提高编码效率,并可通过调节阈值大小满足不同编码要求。     

Fast and accurate global motion compensation

Video understanding has attracted significant research attention in recent years, motivated by interest in video surveillance, rich media retrieval and vision-based gesture interfaces. Typical methods focus on analyzing both the appearance and motion of objects in video. However, the apparent motion induced by a moving camera can dominate the observed motion, requiring sophisticated methods for compensating for camera motion without a priori knowledge of scene characteristics. This paper introduces two new methods for global motion compensation that are both significantly faster and more accurate than state of the art approaches. The first employs RANSAC to robustly estimate global scene motion even when the scene contains significant object motion. Unlike typical RANSAC-based motion estimation work, we apply RANSAC not to the motion of tracked features but rather to a number of segments of image projections. The key insight of the second method involves reliably classifying salient points into foreground and background, based upon the entropy of a motion inconsistency measure. Extensive experiments on established datasets demonstrate that the second approach is able to remove camera-based observed motion almost completely while still preserving foreground motion.     

Content adaptive fast motion estimation based on spatio-temporal homogeneity analysis and motion classification

In video coding, research is focused on the development of fast motion estimation (ME) algorithms while keeping the coding distortion as small as possible. It has been observed that the real world video sequences exhibit a wide range of motion content, from uniform to random, therefore if the motion characteristics of video sequences are taken into account before hand, it is possible to develop a robust motion estimation algorithm that is suitable for all kinds of video sequences. This is the basis of the proposed algorithm. The proposed algorithm involves a multistage approach that includes motion vector prediction and motion classification using the characteristics of video sequences. In the first step, spatio-temporal correlation has been used for initial search centre prediction. This strategy decreases the effect of unimodal error surface assumption and it also moves the search closer to the global minimum hence increasing the computation speed. Secondly, the homogeneity analysis helps to identify smooth and random motion. Thirdly, global minimum prediction based on unimodal error surface assumption helps to identify the proximity of global minimum. Fourthly, adaptive search pattern selection takes into account various types of motion content by dynamically switching between stationary, center biased and, uniform search patterns. Finally, the early termination of the search process is adaptive and is based on the homogeneity between the neighboring blocks.Extensive simulation results for several video sequences affirm the effectiveness of the proposed algorithm. The self-tuning property enables the algorithm to perform well for several types of benchmark sequences, yielding better video quality and less complexity as compared to other ME algorithms. Implementation of proposed algorithm in JM12.2 of H.264/AVC shows reduction in computational complexity measured in terms of encoding time while maintaining almost same bit rate and PSNR as compared to Full Search algorithm.     

基于运动特征的自适应运动估计算法

提出了一种新的基于运动特征的自适应运动估计算法。该算法主要基于两方面：（1）建立具有自适应特性的搜索起点预测模型,根据运动相关性的变化调整模型参数,使预测结果更加接近最佳运动矢量。（2）采用的搜索模板可以根据物体的运动特征调整大小和形状,从而提高搜索效率。实验结果表明,该算法在PSNR和搜索速度两方面均明显优于常用的快速算法。     

A robust dynamic programming algorithm for motion detection and estimation

A new robust algorithm for motion detection and precise evaluation of the motion vectors of moving objects in a sequence of images is presented. It is well known that the accuracy of estimating motion vectors estimation is limited by smoothness constraints and mutual occlusions of motion segments. The proposed method is a fusion of block-matching motion estimation and global optimization technique. It is robust to motion discontinuity and moving objects occlusions. To avoid some contradictions between global optimization techniques and piece-wise smooth values of sought motion vectors, a hidden segmentation model is utilized. Computer simulation and experimental results demonstrate an excellent performance of the method in terms of dynamic motion analysis. This article was translated by the authors. Mikhail Mozerov received his MS degree in Physics from the Moscow State University in 1982 and his PhD degree in Image Processing from the Institute of Information Transmission Problems, Russian Academy of Sciences, in 1995. He works at the Laboratory of Digital Optics of the Institute of Information Transmission Problems, Russian Academy of Sciences. His research interests include signal and image processing, pattern recognition, digital holography. Vitaly Kober obtained his MS degree in Applied Mathematics from the Air-Space University of Samara (Russia) in 1984, and his PhD degree in 1992 and Doctor of Sciences degree in 2004 in Image Processing from the Institute of Information Transmission Problems, Russian Academy of Sciences. Now he is a titular researcher at the Centro de Investigación Científica y de Educación Superior de Ensenada (Cicese), México. His research interests include signal and image processing, pattern recognition. Iosif A. Ovseyevich graduated from the Moscow Electrotechnical Institute of Telecommunications. Received candidate’s degree in 1953 and doctoral degree in information theory in 1972. At present, he is Emeritus Professor at the Institute of Information Transmission Problems of the Russian Academy of Sciences. His research interests include information theory, signal processing, and expert systems. He is a Member of IEEE, Popov Radio Society.     

基于运动信息自适应的快速运动估计算法

UMHexagonS是H.264视频编码标准中所采用的快速整像素运动估计算法,但在许多实时场景的应用中,该算法还明显存在搜索点数过多、搜索速度较慢的缺憾,急需进一步的改进和优化。在UMHexagonS算法的基础上,提出一种基于运动信息自适应的快速运动估计算法。使用动态搜索窗为不同尺寸的块自适应地分配预测搜索窗;根据当前块的运动剧烈程度选择运动类型自适应的搜索方案;通过分析实际运动序列水平、垂直方向的偏向特性依次采用带方向的十字型搜索和自适应的矩形—菱形搜索;利用预测运动矢量的方向信息采用自适应的多层次八边形区域搜索;并依据块的尺寸大小采用自适应的六边形搜索。实验结果表明,本文算法相比于UMHexagonS算法而言,图像的峰值信噪比(PSNR)平均提高了0.0125 dB,同时运动估计时间减少了13%32%,其场景自适应能力和实时性能都得到了很大的增强。     

基于运动矢量空间相关性的H.264分像素运动估计

随着整像素运动估计快速算法的发展,分像素运动估计的计算量在运动估计中所占比重越发明显。为了减少分像素运动估计的计算量,提出了一种利用运动矢量空间相关性来预测整像素运动块,对整像素运动块进行分像素搜索过程跳过的分像素运动估计方法。实验结果表明,该算法与全分像素搜索算法结合使用,在基本保持搜索精度不变的情况下,比单纯的全分像素搜索算法减少60%左右的分像素搜索点。该算法可与其他快速分像素搜索算法结合使用,以获得更好的编码性能。     

一种运动估计的自适应菱形算法

在菱形搜索算法的基础上,依据图像序列的运动矢量的时空相关性和中心偏移特性,首先对宏块进行类型划分、设定阀值,进一步提出了初始搜索点的预测。实验证明,该算法在保证图像质量的同时,大大提高了搜索速度。     

结合运动矢量和像素递归的全局运动估计方法

全局运动估计是计算机视觉、视频处理等领域中一项重要研究课题。结合运动矢量和像素递归提出一种新的全局运动估计方法,该方法根据块运动矢量求出运动矢量直方图,找出主要块运动方向作为初始的全局运动方向,并初始化全局运动参数。利用运动矢量间距离及类间方差求出运动矢量分割阈值,自适应地去除外点块区域。根据背景块梯度和值的大小,在每个背景块中选择一到两个特征像素点进行运动参数估计。实验结果表明,该方法具有较快的计算速度,同时也具有较高的计算精度。     

基于序列图像的视觉定位与运动估计

研究了移动机器人的视觉定位和目标的运动估计。采用单目视觉系统,借助人工标识物,由小孔成像模型及空间几何关系,推导出视觉测距模型,并实现了移动机器人的自定位和目标的定位。通过序列图像,应用基于特征的运动分析方法估计球体的运动参数,推导出移动机器人对运动目标的跟踪模型。球体定位实验结果表明:该方法的定位精度较高。     

一种改进的运动估计新算法

提出一种改进的新三步搜索法（NITSS）。该方法充分利用视频序列运动矢量概率分布上的中心偏置特性,在三步搜索算法的基础上引入了六边型分布的6个点构成搜索点群,解决了三步法的小运动估计效果较差问题。实验结果表明,同TSS算法相比,NITSS算法降低了搜索运算量,提高了搜索精度。     

A pattern based PSO approach for block matching in motion estimation

Block matching motion estimation is a popular method in developing video coding applications. A new algorithm has been proposed for reducing the number of search points using a pattern based particle swarm optimization (PSO) for motion estimation. The conventional particle swarm optimization has been modified to provide accurate solutions in motion estimation problems. This leads to very low computational cost and good estimation accuracy. Due to the center biased nature of the videos, the proposed approach uses an initial pattern to speed up the convergence of the algorithm. Simulation results show that improvements over other fast block matching motion estimation algorithms could be achieved with 31%～63% of search point reduction, without degradation of image quality.     

Spatiotemporal video segmentation and motion estimation through irregular pyramids

This paper presents a new spatiotemporal segmentation technique for video sequences. It relies on building irregular pyramids based on its homogeneity over consecutive frames. Pyramids are interlinked to keep a relationship between the regions in the frames. Its performance is good in real-world conditions because it does not depend on image constrains.     

结合两步估算和运动补偿的子像素运动估计

针对图像超分辨中的运动估计问题,提出了一种两步估计方法,用于估计低分辨图像帧间的子像素相对运动。第一步,采用相关法或匹配法计算两幅低分辨率图像间的整像素相对位移,对其中的一幅按估计的参数进行运动补偿,第二步,对补偿后的两幅图像使用梯度法计算小数像素相对位移。通过两步计算,得到了比较精确的帧间相对运动参数。该方法不需要对低分辨率图像进行插值,以获得图像的高分辨近似,运算速度也较快。     

快速运动估计中一种改进的块匹配免疫算法

图像的运动估计中,基于块匹配的免疫算法由于算法的随机性,使得抗体群中不匹配块增多,导致了免疫算法运算量增大。改进算法搜索过程中,通过快速识别出搜索窗内不可能成为匹配块的候选块,并把这些块消除掉,不对其进行匹配误差运算,从而可以大幅度减少免疫算法运算量。同时,由于改进算法使得候选块的匹配可能性提高,使得相对于传统块匹配免疫算法,重建图像的均峰信噪比进一步提高,进而重建图像质量得以提高。仿真实验结果验证了改进算法的上述优点。     

一种十字形运动搜索算法

近几年,虽然运动估计算法有了很多种的快速算法,但是,运动搜索巨大的计算量依然是视频压缩速率的瓶颈,本文针对运动矢量的分布特点,提出了一种新的运动搜索算法,算法不仅结构简单,而且测试结果表明,该算法比原有DS9(dia-mondsearch)算法在搜索点数和图像质量方面有较大的提高,最好时的搜索点数只有DS算法的3/4。     

一种基于块匹配的鲁棒运动估计算法

详细论述了基于块匹配的鲁棒运动估计算法。跟已有的基于块匹配的运动估计算法比较，首先，我们引入颜色信息来提高运动估计的准确性；其次，在更广泛的意义上运用自适应策略来减少计算量并同时保证算法的鲁棒性；最后，提出的基于预测修正的复合查找方法充分利用了物体运动的全局信息，克服了三步查找算法以及全查找算法的缺点并充分发挥它们二者的优点从而提高查找的效率和匹配精度。实验结果表明基于块匹配的鲁棒运动估计算法具有抗干扰能力强、运动估计准确、计算效率高等优点。     

一种非插值子像素精度运动估计方法*

为了平衡运动估计中搜索算法的复杂度与搜索精度,基于双模式算法的思想,提出一种结合基于改进的粒子群算法（PSO）和十字搜索算法（ARPS）的双模式运动搜索算法。该算法对不同运动程度的图像采用不同的运动搜索算法（运动剧烈时采用PSO算法,运动平缓时采用ARPS算法）,有效地结合了PSO的全局性特点以及ARPS的局部性特点,同时保持了ARPS的快速性。实验表明,该算法的整体性能高于传统的单模式运动估计算法以及已有的多模式运动估计算法。