Edge oriented block motion estimation for video coding

Intensity-based block motion estimation and compensation algorithms are widely used to exploit temporal redundancies in video coding, although they suffer from several drawbacks. One of the problems is that blocks located on boundaries of moving objects are not estimated accurately. It causes poor motion-compensated prediction along the moving edges to which the human visual system is very sensitive. By considering the characteristics of block motions for typical image sequences, an intelligent classifier is proposed to separate blocks containing moving edges to improve on conventional intensity-based block matching approaches. The motion vectors of these blocks are computed using edge matching techniques, so that the motion-compensated frames are tied more closely to the physical features. The proposed method can then make use of this accurate motion information for edge blocks to compute the remaining non-edged blocks. Consequently, a fast and efficient block motion estimation algorithm is developed. Experimental results show that this approach gives a significant improvement in accuracy for motion-compensated frames and computational complexity, in comparison with the traditional intensity-based block motion estimation methods     

Improving the accuracy and reliability of motion tracking methods used for extracting temporal motor activity signals from video recordings of neonatal seizures

This paper presents an approach for improving the accuracy and reliability of motion tracking methods developed for video based on block motion models. This approach estimates the displacement of a block of pixels between two successive frames by minimizing an error function defined in terms of the pixel intensities at these frames. The minimization problem is made analytically tractable by approximating the error function using a second-order Taylor expansion. The improved reliability of the proposed method is illustrated by its application in the extraction of temporal motor activity signals from video recordings of neonatal seizures.     

Improved motion estimation time using a combination of dynamic reference frame selection and residue-based mode decision

Motion estimation using multiple reference frames is widely used as the basis for recent video coding standards (eg. H.264/AVC) to achieve increased coding efficiency. However, this increases the complexity of the encoding process. In this paper, a new technique for efficient motion estimation is proposed. A combination of multiple reference frame selection and image residue-based mode selection is used to improve motion estimation time. By dynamic selection of an initial reference frame in advance, the number of reference frames to be considered is reduced. In addition, from examination of the residue between the current block and reconstructed blocks in preceding frames, variable block size mode decisions are made. Modified initial motion vector estimation and early stop condition detection are also adopted to speed up the motion estimation procedure. Experimental results compare the performance of the proposed algorithm with a state of the art motion estimation algorithm and demonstrate significantly reduced motion estimation time while maintaining PSNR performance.     

Perceptual feedback in multigrid motion estimation using animproved DCT quantization

In this paper, a multigrid motion compensation video coder based on the current human visual system (HVS) contrast discrimination models is proposed. A novel procedure for the encoding of the prediction errors has been used. This procedure restricts the maximum perceptual distortion in each transform coefficient. This subjective redundancy removal procedure includes the amplitude nonlinearities and some temporal features of human perception. A perceptually weighted control of the adaptive motion estimation algorithm has also been derived from this model. Perceptual feedback in motion estimation ensures a perceptual balance between the motion estimation effort and the redundancy removal process. The results show that this feedback induces a scale-dependent refinement strategy that gives rise to more robust and meaningful motion estimation, which may facilitate higher level sequence interpretation. Perceptually meaningful distortion measures and the reconstructed frames show the subjective improvements of the proposed scheme versus an H.263 scheme with unweighted motion estimation and MPEG-like quantization.     

基于可变块运动矢量的边信息生成算法*

在分布式视频编码系统中,针对图像中细节丰富的区域易造成严重的块效应,提出了一种基于可变块运动矢量的边信息生成算法。根据前后相邻关键帧对应块的相关性,将像素块分为保留块和运动块。对保留块直接作保留处理,对运动块中的像素块继续进行分割并计算子块的初始运动矢量,最后将所有对应块的运动矢量进行加权自适应运动补偿得到改进的边信息。实验结果表明,对于运动较剧烈复杂的视频序列,该算法能够提高边信息生成质量, 并且使得改进后的边信息PSNR值提高了0.98~1.33 dB。     

基于可分离模糊核的复合运动模糊星图建模与仿真

在采用天文光学自主导航方式进行深空探测时,导航星的视星等高,为获取清晰的星图,星敏感器焦距长、曝光时间长,同时受探测器多种运动复合的影响,星图易产生模糊拖尾现象,很难用基于硬件的常规方法有效复原。研究复合运动模糊星图的建模方法,可揭示星敏感器与导航星的真实相对运动对星图像质的影响规律,从而用软件算法复原高像质星图,提高星敏感器的动态性能。由于缺乏确定的参数模型,复合运动模糊星图建模难度较大。提出了一种基于可分离模糊核的复合运动模糊星图建模的新方法,研究星敏感器角运动和角振动对成像的影响,并与常用的分步模糊方法比较。仿真表明:基于可分离模糊核方法的仿真星图更符合星敏感器实际运动对星图退化的影响;星敏感器的复合运动特别是旋转运动对质心提取效果具有显著影响,严重时会导致星敏感器无法定姿。     

Hierarchical optimal contour control of motion systems

Many motion control applications utilize multiple axes to traverse complex trajectories. The hierarchical contour control methodology proposed in this paper treats each axis as an individual subsystem and combines the Internal Model Principle with robust tracking and optimal hierarchical control techniques to track a desired trajectory. In this method the objectives are divided into two levels. Measurable goals of each subsystem are included in the bottom level and unmeasurable goals, which are estimated using the bottom level states, are considered in the top level where the subsystems are synchronized. The proposed methodology reduces system complexity while greatly improving tracking performance. The tracking error for each axis is considered in the bottom level where the Internal Model Principle is used to compensate for unmodeled nonlinear friction and slowly varying uncertainties. The top level goal (i.e., zero contour error) is propagated to the lower level by an aggregation relationship between contour error and physical linear axis variables. A controller is designed at the bottom level which simultaneously satisfies the bottom level goals (i.e., individual axis tracking) and the top level goal. Experimental results implemented on a table top CNC machine for diamond and freeform contours illustrate the performance of the proposed methodology. While this methodology was implemented for a two-axis motion system, it can be extended to any motion system containing more than two axes.     

Multi-frame de-raining algorithm using a motion-compensated non-local mean filter for rainy video sequences

This paper proposes a rain detection and removal algorithm that is robust against camera motion. The proposed algorithm initially detects possible rain streaks by using spatial properties, such as the luminance and structure of rain streaks. Then, the rain streak candidates are selected based on a Gaussian distribution model. Finally, these detected regions are improved with an advanced temporal property in a block-matching process. After the rain detection step, a non-rain block-matching algorithm for each block is performed between adjacent frames to find blocks similar to the block that has rain pixels. If similar blocks are obtained, the rain region of the block is reconstructed by spatio-temporal non-local mean filtering using similar neighboring regions. Finally, a specific post-processing is performed for visibility enhancement and flickering artifact removal. Experiment results show that the proposed algorithm uses only five temporally adjacent frames for rain removal but outperforms previous methods in terms of subjective visual quality.     

Dynamic swarm particle for fast motion vehicle tracking

Nowadays, the broad availability of cameras and embedded systems makes the application of computer vision very promising as a supporting technology for intelligent transportation systems, particularly in the field of vehicle tracking. Although there are several existing trackers, the limitation of using low‐cost cameras, besides the relatively low processing power in embedded systems, makes most of these trackers useless. For the tracker to work under those conditions, the video frame rate must be reduced to decrease the burden on computation. However, doing this will make the vehicle seem to move faster on the observer's side. This phenomenon is called the fast motion challenge. This paper proposes a tracker called dynamic swarm particle (DSP), which solves the challenge. The term particle refers to the particle filter, while the term swarm refers to particle swarm optimization (PSO). The fundamental concept of our method is to exploit the continuity of vehicle dynamic motions by creating dynamic models based on PSO. Based on the experiments, DSP achieves a precision of 0.896 and success rate of 0.755. These results are better than those obtained by several other benchmark trackers.     

Simplex minimisation for fast long-term memory motion estimation

Three algorithms for fast long-term memory block matching motion estimation are proposed. The algorithms are extensions of the fast single-reference simplex minimisation search (SMS) algorithm. The proposed algorithms represent different degrees of compromise between prediction quality and computational complexity. With a long-term memory of 50 frames the proposed algorithms have a computational complexity comparable to that of single-reference full-search while still maintaining the prediction gain of long-term memory motion estimation     

Fuzzy Logic Based Temporal Error Concealment for H.264 Video

In this paper, a new error concealment algorithm is proposed for the H.264 standard. The algorithm consists of two processes. The first process uses a fuzzy logic method to select the size type of lost blocks. The motion vector of a lost block is calculated from the current frame, if the motion vectors of the neighboring blocks surrounding the lost block are discontinuous. Otherwise, the size type of the lost block can be determined from the preceding frame. The second process is an error concealment algorithm via a proposed adapted multiple‐reference‐frames selection for finding the lost motion vector. The adapted multiple‐reference‐frames selection is based on the motion estimation analysis of H.264 coding so that the number of searched frames can be reduced. Therefore the most accurate mode of the lost block can be determined with much less computation time in the selection of the lost motion vector. Experimental results show that the proposed algorithm achieves from 0.5 to 4.52 dB improvement when compared to the method in VM 9.0.     

Real-time speaker tracking using particle filter sensor fusion

Sensor fusion for object tracking has become an active research direction during the past few years. But how to do it in a robust and principled way is still an open problem. In this paper, we propose a new fusion framework that combines both the bottom-up and top-down approaches to probabilistically fuse multiple sensing modalities. At the lower level, individual vision and audio trackers are designed to generate effective proposals for the fuser. At the higher level, the fuser performs reliable tracking by verifying hypotheses over multiple likelihood models from multiple cues. Unlike traditional fusion algorithms, the proposed framework is a closed-loop system where the fuser and trackers coordinate their tracking information. Furthermore, to handle nonstationary situations, the proposed framework evaluates the performance of the individual trackers and dynamically updates their object states. We present a real-time speaker tracking system based on the proposed framework by fusing object contour, color and sound source location. We report robust tracking results.     

Panoramic view construction

In this paper, the problem of constructing the whole view of a scene background from an image sequence is considered. First, point or block correspondence between each pair of successive frames is determined. Three parametric motion models are used: 2-D translation with scale change, affine, and projective. Motion parameters are estimated using either robust criteria and the Levenberg–Marquardt algorithm, or affine moment invariants. Then the parametric models are composed and all the frames are aligned, yielding a whole view of the scene background. A new technique is introduced for the correction of accumulated frame alignment errors.     

An efficient search strategy for block motion estimation usingimage features

Block motion estimation using the exhaustive full search is computationally intensive. Fast search algorithms offered in the past tend to reduce the amount of computation by limiting the number of locations to be searched. Nearly all of these algorithms rely on this assumption: the mean absolute difference (MAD) distortion function increases monotonically as the search location moves away from the global minimum. Essentially, this assumption requires that the MAD error surface be unimodal over the search window. Unfortunately, this is usually not true in real-world video signals. However, we can reasonably assume that it is monotonic in a small neighborhood around the global minimum. Consequently, one simple strategy, but perhaps the most efficient and reliable, is to place the checking point as close as possible to the global minimum. In this paper, some image features are suggested to locate the initial search points. Such a guided scheme is based on the location of certain feature points. After applying a feature detecting process to each frame to extract a set of feature points as matching primitives, we have extensively studied the statistical behavior of these matching primitives, and found that they are highly correlated with the MAD error surface of real-world motion vectors. These correlation characteristics are extremely useful for fast search algorithms. The results are robust and the implementation could be very efficient. A beautiful point of our approach is that the proposed search algorithm can work together with other block motion estimation algorithms. Results of our experiment on applying the present approach to the block-based gradient descent search algorithm (BBGDS), the diamond search algorithm (DS) and our previously proposed edge-oriented block motion estimation show that the proposed search strategy is able to strengthen these searching algorithms. As compared to the conventional approach, the new algorithm, through the extraction of image features, is more robust, produces smaller motion compensation errors, and has a simple computational complexity.     

基于稀疏稠密结构表示与在线鲁棒字典学习的视觉跟踪

L1跟踪对适度的遮挡具有鲁棒性,但是存在速度慢和易产生模型漂移的不足。为了解决上述两个问题,该文首先提出一种基于稀疏稠密结构的鲁棒表示模型。该模型对目标模板系数和小模板系数分别进行L2范数和L1范数正则化增强了对离群模板的鲁棒性。为了提高目标跟踪速度,基于块坐标优化原理,用岭回归和软阈值操作建立了该模型的快速算法。其次,为降低模型漂移的发生,该文提出一种在线鲁棒的字典学习算法用于模板更新。在粒子滤波框架下,用该表示模型和字典学习算法实现了鲁棒快速的跟踪方法。在多个具有挑战性的图像序列上的实验结果表明：与现有跟踪方法相比,所提跟踪方法具有较优的跟踪性能。     

Motion estimation in low-delay hierarchical p-frame coding using motion vector composition

Low-delay hierarchical prediction structure is currently adopted in various new video coding standards. The only hurdle of this structure is the need of motion estimation in distant reference frames. To maintain high coding efficiency, a large search range for motion estimation can improve the coding efficiency in distant reference pictures. Computational complexity will thus be increased dramatically. In this paper, a fast motion estimation scheme for a low-delay hierarchical P-frame structure is proposed. The proposed scheme adopts a motion vector composition strategy to expedite the motion estimation process for distant reference frames in the hierarchical P structure. In addition, a motion vector composition algorithm is tailor-made with the proposed hierarchical P coding scheme to further improve the coding efficiency. Simulation results show that the proposed scheme can deliver a remarkable complexity savings and coding efficiency improvement on coding a frame in low temporal layers of the hierarchical P structure.     

A multistage motion vector processing method for motion-compensated frame interpolation.

In this paper, a novel, low-complexity motion vector processing algorithm at the decoder is proposed for motion-compensated frame interpolation or frame rate up-conversion. We address the problems of having broken edges and deformed structures in an interpolated frame by hierarchically refining motion vectors on different block sizes. Our method explicitly considers the reliability of each received motion vector and has the capability of preserving the structure information. This is achieved by analyzing the distribution of residual energies and effectively merging blocks that have unreliable motion vectors. The motion vector reliability information is also used as a prior knowledge in motion vector refinement using a constrained vector median filter to avoid choosing identical unreliable one. We also propose using chrominance information in our method. Experimental results show that the proposed scheme has better visual quality and is also robust, even in video sequences with complex scenes and fast motion.     

A hierarchical design methodology for full-search block matching motion estimation

Many useful DSP algorithms have high dimensions and complex logic. Consequently, an efficient implementation of these algorithms on parallel processor arrays must involve a structured design methodology. Full-search block-matching motion estimation is one of those algorithms that can be developed using parallel processor arrays. In this paper, we present a hierarchical design methodology for the full-search block matching motion estimation. Our proposed methodology reduces the complexity of the algorithm into simpler steps and then explores the different possible design options at each step. Input data timing restrictions are taken into consideration as well as buffering requirements. A designer is able to modify system performance by selecting some of the algorithm variables for pipelining or broadcasting. Our proposed design strategy also allows the designer to study time and hardware complexities of computations at each level of the hierarchy. The resultant architecture allows easy modifications to the organization of data buffers and processing elements-their number, datapath pipelining, and complexity-to produce a system whose performance matches the video data sample rate requirements.     

Enhanced dynamic pattern search algorithm with weighted search points for fast motion estimation

For video encoding, a number of block-based motion estimation techniques have been proposed in the literature. Some of them are based on fixed search patterns, whereas others are utilizing spatial and temporal correlations of blocks within video frames. It has been found that the block correlations give an initial lead in identification of motion vector of the candidate block and the possibility of oversearch or undersearch is minimized. In this paper, the dynamic pattern search (DPS) algorithm for block-based motion estimation has been enhanced which reduces the search point computation by first identifying the stationary blocks. Further, in the proposed algorithm, search points within the search area are evaluated for minimum distortion point in prioritized fashion to reduce the possibility of being trapped in local minima. The proposed work has been compared experimentally with fixed search pattern techniques like full search, diamond search (DS) and hexagon search and adaptive search pattern techniques like adaptive rood pattern search and DPS in terms of various performance parameters. It has been found that the proposed technique is faster by 6.32 and 3.30 times than DS and DPS techniques in terms of avg. search point computation per block and produces better peak signal to noise ratio than DS and DPS by the value of 1.03 and 0.63 dB, respectively. In terms of structural similarity measurement index value, enhanced DPS gives 8% better results than DPS and performs better compression than any other technique except FS.