Motion estimation in the 3-D Gabor domain.

Motion estimation methods can be broadly classified as being spatiotemporal or frequency domain in nature. The Gabor representation is an analysis framework providing localized frequency information. When applied to image sequences, the 3-D Gabor representation displays spatiotemporal/spatiotemporal-frequency (st/stf) information, enabling the application of robust frequency domain methods with adjustable spatiotemporal resolution. In this work, the 3-D Gabor representation is applied to motion analysis. We demonstrate that piecewise uniform translational motion can be estimated by using a uniform translation motion model in the st/stf domain. The resulting motion estimation method exhibits both good spatiotemporal resolution and substantial noise resistance compared to existing spatiotemporal methods. To form the basis of this model, we derive the signature of the translational motion in the 3-D Gabor domain. Finally, to obtain higher spatiotemporal resolution for more complex motions, a dense motion field estimation method is developed to find a motion estimate for every pixel in the sequence.     

Accurate optical flow in noisy image sequences using flow adapted anisotropic diffusion

In this paper, we combine 3D anisotropic diffusion and motion estimation for image denoising and improvement of motion estimation. We compare different continuous isotropic nonlinear and anisotropic diffusion processes, which can be found in literature, with a process especially designed for image sequence denoising for motion estimation. All of these processes initially improve motion estimation due to reduction of noise and high frequencies. But while all the well known processes rapidly destroy or hallucinate motion information, the process brought forward here shows considerably less information loss or violation even at motion boundaries. We show the superior behavior of this process. Further we compare the performance of a standard finite difference diffusion scheme with several schemes using derivative filters optimized for rotation invariance. Using the discrete scheme with least smoothing artifacts we demonstrate the denoising capabilities of this approach. We exploit the motion estimation to derive an automatic stopping criterion.     

Two-dimensional matched filtering for motion estimation

In this work, we describe a frequency domain technique for the estimation of multiple superimposed motions in an image sequence. The least-squares optimum approach involves the computation of the three-dimensional (3-D) Fourier transform of the sequence, followed by the detection of one or more planes in this domain with high energy concentration. We present a more efficient algorithm, based on the properties of the Radon transform and the two-dimensional (2-D) fast Fourier transform, which can sacrifice little performance for significant computational savings. We accomplish the motion detection and estimation by designing appropriate matched filters. The performance is demonstrated on two image sequences.     

3-D Kalman filter for image motion estimation

This paper presents a new three-dimensional (3-D) Markov model for motion vector fields. The three dimensions consist of the two space dimensions plus a scale dimension. We use a compound signal model to handle motion discontinuity in this 3-D Markov random field (MRF). For motion estimation, we use an extended Kalman filter as a pel-recursive estimator. Since a single observation can be sensitive to local image characteristics, especially when the model is not accurate, we employ windowed multiple observations at each pixel to increase accuracy. These multiple observations employ different weighting values for each observation, since the uncertainty in each observation is different. Finally, we compare this 3-D model with earlier proposed one-dimensional (1-D) (coarse-to-fine scale) and two-dimensional (2D) spatial compound models, in terms of motion estimation performance on a synthetic and a real image sequence.     

Suppression of slice selection axis motion artifacts in MRI

Patient motion during data acquisition in magnetic resonance imaging causes artifacts in the reconstructed image, which for two-dimensional Fourier transform imaging techniques appear as blurring and ghost repetitions of the moving structures. T. Mitsa et al. (1990) proposed a technique for suppressing artifacts from periodic motion along the slice selection axis. A different approach to the same problem is presented which is not restricted to periodic motion. The algorithm is verified using a simulated phantom and motion. It is also shown to perform well in the presence of noise and motion within the imaging plane.     

Iterative maximum likelihood displacement field estimation inquantum-limited image sequences

We develop an algorithm for obtaining the maximum likelihood (ML) estimate of the displacement vector field (DVP) from two consecutive image frames of an image sequence acquired under quantum-limited conditions. The estimation of the DVF has applications in temporal filtering, object tracking, stereo matching, and frame registration in low-light level image sequences as well as low-dose clinical X-ray image sequences. In the latter case, a controlled X-ray dosage reduction may be utilized to lower the radiation exposure to the patient and the medical staff. The quantum-limited effect is modeled as an undesirable, Poisson-distributed, signal-dependent noise artifact. A Fisher-Bayesian formulation is used to estimate the DVF and a block component search algorithm is employed in obtaining the solution. Several experiments involving a phantom sequence and a teleconferencing image sequence with realistic motion demonstrate the effectiveness of this estimator in obtaining the DVF under severe quantum noise conditions (20-25 events/pixel).     

Maximum a Posteriori Strategy for the Simultaneous Motion and Material Property Estimation of the Heart

In addition to its technical merits as a challenging nonrigid motion and structural integrity analysis problem, quantitative estimation of cardiac regional functions and material characteristics has significant physiological and clinical value. We developed a stochastic finite-element framework for the simultaneous recovery of myocardial motion and material parameters from medical image sequences with an extended Kalman filter approach, and we have shown that this simultaneous estimation strategy achieves more accurate and robust results than separated motion and material estimation efforts. In this paper, we present a new computational strategy for the framework based upon the maximum a posteriori estimation principles, realized through the extended Kalman smoother, that produces a sequence of kinematics state and material parameter estimation of the entire myocardium, including the endocardial, epicardial, and midwall tissues. The system dynamics equations of the heart are constructed using a biomechanical model with stochastic parameters, and the tissue material and deformation parameters are jointly estimated from the periodic imaging data. Noise-corrupted synthetic image sequences with known kinematics and material parameters are used to assess the accuracy and robustness of the framework. Experiments with canine magnetic resonance tagging and phase-contrast image sequences have been conducted with very promising results, as validated through comparison to the histological staining of postmortem myocardium.     

Motion analysis and segmentation through spatio-temporal slices processing

This paper presents new approaches in characterizing and segmenting the content of video. These approaches are developed based upon the pattern analysis of spatio-temporal slices. While traditional approaches to motion sequence analysis tend to formulate computational methodologies on two or three adjacent frames, spatio-temporal slices provide rich visual patterns along a larger temporal scale. We first describe a motion computation method based on a structure tensor formulation. This method encodes visual patterns of spatio-temporal slices in a tensor histogram, on one hand, characterizing the temporal changes of motion over time, on the other hand, describing the motion trajectories of different moving objects. By analyzing the tensor histogram of an image sequence, we can temporally segment the sequence into several motion coherent subunits, in addition, spatially segment the sequence into various motion layers. The temporal segmentation of image sequences expeditiously facilitates the motion annotation and content representation of a video, while the spatial decomposition of image sequences leads to a prominent way of reconstructing background panoramic images and computing foreground objects.     

单目图像序列中基于正则化的局部三维非刚体运动估计

通过局部的三维非刚体运动估计进而达到全局估计结果是三维非刚体运动估计中的重要方法。本文提出了在单目图像序列中利用正则化的手段解决局部的三维非刚体运动估计。首先，在帧间特征点匹配已确立的前提下，利用仿射运动模型并结合中心投影方式提出了运动估计的最小二乘模型；然后针对三维运动估计的不适定性提出了正则化的运动估计方法，以正则化的形式融入运动的先验知识，使运动估计的结果更具鲁棒性；最后利用Levenberg-Marquart方法实现运动参数的求解。仿真图像序列的实验反映了本文方法的有效性。     

Block-based motion field segmentation for video coding

In the past few years, motion compensation has been widely used in the coding of image sequences. Most of motion estimation and compensation schemes belong to block-based framework. The framework simplifies the complexity of motion estimation, but gives over constraints to the motion field, which results in worse accuracy on the boundary of moving objects. This paper presents a novel technique for raising motion field accuracy. It uses several pre-defined pattern types to segment the motion fields of the previous frame of a sequence. The segmentation is based on the MAP framework that uses iterative method to obtain the solution. In addition, we develop a predictive scheme to predict the location of motion field discontinuities in the current frame, which further reduces the side information for the representation of segmentation.     

Estimation of Motions in Color Image Sequences Using Hypercomplex Fourier Transforms

Although the motion estimation problem has been extensively studied, most of the proposed estimation approaches deal mainly with monochrome videos. The most usual way to apply them also in color image sequences is to process each color channel separately. A different, more sophisticated approach is to process the color channels in a “holistic” manner using quaternions, as proposed by Ell and Sangwine. In this paper, we extend standard spatiotemporal Fourier-based approaches to handle color image sequences, using the hypercomplex Fourier transform. We show that translational motions are manifested as energy concentration along planes in the hypercomplex 3-D Fourier domain and we describe a methodology to estimate the motions, based on this property. Furthermore, we compare the three-channels-separately approach with our approach and we show that the computational effort can be reduced by a factor of 1/3, using the hypercomplex Fourier transform. Also, we propose a simple, accompanying method to extract the moving objects in the hypercomplex Fourier domain. Our experimental results on synthetic and natural images verify our arguments throughout the paper.      

Three-Dimensional Motion Estimation of Atmospheric Layers From Image Sequences

In this paper, we address the problem of estimating 3-D motions of a stratified atmosphere from satellite image sequences. The analysis of 3-D atmospheric fluid flows associated with incomplete observation of atmospheric layers due to the sparsity of cloud systems is very difficult. This makes the estimation of dense atmospheric motion field from satellite image sequences very difficult. The recovery of the vertical component of fluid motion from a monocular sequence of image observations is a very challenging problem for which no solution exists in the literature. Based on a physically sound vertical decomposition of the atmosphere into cloud layers of different altitudes, we propose here a dense motion estimator dedicated to the extraction of 3-D wind fields characterizing the dynamics of a layered atmosphere. Wind estimation is performed over the complete 3-D space, using a multilayer model describing a stack of dynamic horizontal layers of evolving thickness, interacting at their boundaries via vertical winds. The efficiency of our approach is demonstrated on synthetic and real sequences.      

Curvelet-Based Snake for Multiscale Detection and Tracking of Geophysical Fluids

Detection and target tracking have an application to many scientific problems. The approach developed in this paper is motivated by the applications of detection and tracking characteristic deformable structures in geophysical fluids. We develop an integrated detection and tracking method of geophysical fluids based on a discrete curvelet representation of the information characterizing the targets. Curvelets are in some sense geometric wavelets, allowing an optimal sparse representation of two-dimensional piecewise continuous objects with C ²-singularities. The proposed approach first identifies a consistent vortex by a curvelet-based gradient-vector-flow snake and then establishes the motion correspondence of the snaxels between successive time frames by a constructed so-called semi-T or comp-T multiscale motion-estimation method based on the geometric wavelets. Furthermore, a combination of total-variation regularization and cycle-spinning techniques effectively removes false matches and improves significantly the estimation. Numerical experiments at each stage demonstrate the performance of the proposed tracking methodology for temporal oceanographic satellite image sequences corrupted by noise, with weak edges and submitted to large deformations, in comparison to conventional methods     

Multiple motion segmentation with level sets

Segmentation of motion in an image sequence is one of the most challenging problems in image processing, while at the same time one that finds numerous applications. To date, a wealth of approaches to motion segmentation have been proposed. Many of them suffer from the local nature of the models used. Global models, such as those based on Markov random fields, perform, in general, better. In this paper, we propose a new approach to motion segmentation that is based on a global model. The novelty of the approach is twofold. First, inspired by recent work of other researchers we formulate the problem as that of region competition, but we solve it using the level set methodology. The key features of a level set representation, as compared to active contours, often used in this context, are its ability to handle variations in the topology of the segmentation and its numerical stability. The second novelty of the paper is the formulation in which, unlike in many other motion segmentation algorithms, we do not use intensity boundaries as an accessory; the segmentation is purely based on motion. This permits accurate estimation of motion boundaries of an object even when its intensity boundaries are hardly visible. Since occasionally intensity boundaries may prove beneficial, we extend the formulation to account for the coincidence of motion and intensity boundaries. In addition, we generalize the approach to multiple motions. We discuss possible discretizations of the evolution (PDE) equations and we give details of an initialization scheme so that the results could be duplicated. We show numerous experimental results for various formulations on natural images with either synthetic or natural motion.     

Double compression detection based on local motion vector field analysis in static-background videos

Videos captured by stationary cameras are widely used in video surveillance and video conference. This kind of video often has static or gradually changed background. By analyzing the properties of static-background videos, this work presents a novel approach to detect double MPEG-4 compression based on local motion vector field analysis in static-background videos. For a given suspicious video, the local motion vector field is used to segment background regions in each frame. According to the segmentation of backgrounds and the motion strength of foregrounds, the modified prediction residual sequence is calculated, which retains robust fingerprints of double compression. After post-processing, the detection and GOP estimation results are obtained by applying the temporal periodic analysis method to the final feature sequence. Experimental results have demonstrated better robustness and efficiency of the proposed method in comparison to several state-of-the-art methods. Besides, the proposed method is more robust to various rate control modes.     

Image sequence filtering in quantum-limited noise with applications to low-dose fluoroscopy

Clinical angiography requires hundreds of X-ray images, putting the patients and particularly the medical staff at risk. Dosage reduction involves an inevitable sacrifice in image quality. In this work, the latter problem is addressed by first modeling the signal-dependent, Poisson-distributed noise that arises as a result of this dosage reduction. The commonly utilized noise model for single images is shown to be obtainable from the new model. Stochastic temporal filtering techniques are proposed to enhance clinical fluoroscopy sequences corrupted by quantum mottle. The temporal versions of these filters as developed here are more suitable for filtering image sequences, as correlations along the time axis can be utilized. For these dynamic sequences, the problem of displacement field estimation is treated in conjunction with the filtering stage to ensure that the temporal correlations are taken along the direction of motion to prevent object blur.     

Resolution enhancement of monochrome and color video using motioncompensation

We propose an iterative algorithm for enhancing the resolution of monochrome and color image sequences. Various approaches toward motion estimation are investigated and compared. Improving the spatial resolution of an image sequence critically depends upon the accuracy of the motion estimator. The problem is complicated by the fact that the motion field is prone to significant errors since the original high-resolution images are not available. Improved motion estimates may be obtained by using a more robust and accurate motion estimator, such as a pel-recursive scheme instead of block matching, in processing color image sequences, there is the added advantage of having more flexibility in how the final motion estimates are obtained, and further improvement in the accuracy of the motion field is therefore possible. This is because there are three different intensity fields (channels) conveying the same motion information. In this paper, the choice of which motion estimator to use versus how the final estimates are obtained is weighed to see which issue is more critical in improving the estimated high-resolution sequences. Toward this end, an iterative algorithm is proposed, and two sets of experiments are presented. First, several different experiments using the same motion estimator but three different data fusion approaches to merge the individual motion fields were performed. Second, estimated high-resolution images using the block matching estimator were compared to those obtained by employing a pel-recursive scheme. Experiments were performed on a real color image sequence, and performance was measured by the peak signal to noise ratio (PSNR).     

High time-resolved cardiac functional imaging using temporal regularization for small animal on a clinical 3T scanner

Accurate assessment of mice cardiac function with magnetic resonance imaging is essential for longitudinal studies and for drug development related to cardiovascular diseases. Whereas dedicated small animal MR scanners are not readily available, it would be a great advantage to be able to perform cardiac assessment on clinical systems, in particular, in the context of translational research. However, mouse imaging remains challenging since it requires both high spatial and temporal resolutions, while gradient performances of clinical scanners often limit the reachable parameters. In this study, we propose a new cine sequence, named "interleaved cine," which combines two repetitions of a standard cine sequence shifted in time in order to reach resolution parameters compatible with mice imaging. More precisely, this sequence allows temporal resolution to be reduced to 6.8 ms instead of 13.5 ms initially imposed by the system's hardware. We also propose a two-step denoising algorithm to suppress some artifacts inherent to the new interleaved cine thus allowing an efficient enhancement of the image quality. In particular, we model and suppress the periodic intensity pattern and further denoise the sequence by soft thresholding of the temporal Fourier coefficients. This sequence was successfully validated with mass and function measurements on relevant mice models of cardiovascular diseases.