Piecewise-Smooth Dense Optical Flow via Level Sets

We propose a new algorithm for dense optical flow computation. Dense optical flow schemes are challenged by the presence of motion discontinuities. In state of the art optical flow methods, over-smoothing of flow discontinuities accounts for most of the error. A breakthrough in the performance of optical flow computation has recently been achieved by Brox et~al. Our algorithm embeds their functional within a two phase active contour segmentation framework. Piecewise-smooth flow fields are accommodated and flow boundaries are crisp. Experimental results show the superiority of our algorithm with respect to alternative techniques. We also study a special case of optical flow computation, in which the camera is static. In this case we utilize a known background image to separate the moving elements in the sequence from the static elements. Tests with challenging real world sequences demonstrate the performance gains made possible by incorporating the static camera assumption in our algorithm.     

On the Fourier Properties of Discontinuous Motion

Retinal image motion and optical flow as its approximation are fundamental concepts in the field of vision, perceptual and computational. However, the computation of optical flow remains a challenging problem as image motion includes discontinuities and multiple values mostly due to scene geometry, surface translucency and various photometric effects such as reflectance. In this contribution, we analyze image motion in the frequency space with respect to motion discontinuities and translucence. We derive the frequency structure of motion discontinuities due to occlusion and we demonstrate its various geometrical properties. The aperture problem is investigated and we show that the information content of an occlusion almost always disambiguates the velocity of an occluding signal suffering from the aperture problem. In addition, the theoretical framework can describe the exact frequency structure of Non-Fourier motion and bridges the gap between Non-Fourier visual phenomena and their understanding in the frequency domain.     

面向运动分割优化的变分光流计算方法研究

面向运动分割的需求,围绕变分光流计算中的运动边缘保留问题,对反应-扩散式光流计算模型中的扩散张量设计与改进方法进行系统深入研究。 在分析已有设计方法的基础上,提出了一种融合图像和流场信息驱动的扩散张量设计方法,并利用该扩散张量建立光流计算模型,然后在多尺度计算框架下给出了相应的偏微分方程数值计算方法。 理论分析与对比实验结果表明,这种设计方法能有效克服现有方法提取运动边缘不精确的缺点,更准确地刻画出运动边缘,提高光流的计算精度。     

Exploiting Discontinuities in Optical Flow

Most optical flow estimation techniques have substantial difficulties dealing with flow discontinuities. Methods which simultaneously detect flow boundaries and use the detected boundaries to aid in flow estimation can produce significantly improved results. Current approaches to implementing these methods still have important limitations, however. We demonstrate three such problems: errors due to the mixture of image properties across boundaries, an intrinsic ambiguity in boundary location when only short sequences are considered, and difficulties insuring that the motion of a boundary aids in flow estimation for the surface to which it is attached without corrupting the flow estimates for the occluded surface on the other side. The first problem can be fixed by basing flow estimation only on image changes at edges. The second requires an analysis of longer time intervals. The third can be aided by using a boundary detection mechanism which classifies the sides of boundaries as occluding and occluded at the same time as the boundaries are detected.     

Non Uniform Multiresolution Method for Optical Flow and Phase Portrait Models: Environmental Applications

In this paper we define a complete framework for processing large image sequences for a global monitoring of short range oceanographic and atmospheric processes. This framework is based on the use of a non quadratic regularization technique for optical flow computation that preserves flow discontinuities. We also show that using an appropriate tessellation of the image according to an estimate of the motion field can improve optical flow accuracy and yields more reliable flows. This method defines a non uniform multiresolution approach for coarse to fine grid generation. It allows to locally increase the resolution of the grid according to the studied problem. Each added node refines the grid in a region of interest and increases the numerical accuracy of the solution in this region. We make use of such a method for solving the optical flow equation with a non quadratic regularization scheme allowing the computation of optical flow field while preserving its discontinuities. The second part of the paper deals with the interpretation of the obtained displacement field. For this purpose a phase portrait model used along with a new formulation of the approximation of an oriented flow field allowing to consider arbitrary polynomial phase portrait models for characterizing salient flow features. This new framework is used for processing oceanographic and atmospheric image sequences and presents an alternative to complex physical modeling techniques.     

A Multigrid Platform for Real-Time Motion Computation with Discontinuity-Preserving Variational Methods

Variational methods are among the most accurate techniques for estimating the optic flow. They yield dense flow fields and can be designed such that they preserve discontinuities, estimate large displacements correctly and perform well under noise and varying illumination. However, such adaptations render the minimisation of the underlying energy functional very expensive in terms of computational costs: Typically one or more large linear or nonlinear equation systems have to be solved in order to obtain the desired solution. Consequently, variational methods are considered to be too slow for real-time performance. In our paper we address this problem in two ways: (i) We present a numerical framework based on bidirectional multigrid methods for accelerating a broad class of variational optic flow methods with different constancy and smoothness assumptions. Thereby, our work focuses particularly on regularisation strategies that preserve discontinuities. (ii) We show by the examples of five classical and two recent variational techniques that real-time performance is possible in all cases—even for very complex optic flow models that offer high accuracy. Experiments show that frame rates up to 63 dense flow fields per second for image sequences of size 160 × 120 can be achieved on a standard PC. Compared to classical iterative methods this constitutes a speedup of two to four orders of magnitude.     

Coarse to over-fine optical flow estimation

We present a readily applicable way to go beyond the accuracy limits of current optical flow estimators. Modern optical flow algorithms employ the coarse to fine approach. We suggest to upgrade this class of algorithms, by adding over-fine interpolated levels to the pyramid. Theoretical analysis of the coarse to over-fine approach explains its advantages in handling flow-field discontinuities and simulations show its benefit for sub-pixel motion. By applying the suggested technique to various multi-scale optical flow algorithms, we reduced the estimation error by 10-30% on the common test sequences. Using the coarse to over-fine technique, we obtain optical flow estimation results that are currently the best for benchmark sequences.     

Smoothing of optical flow using robustified diffusion kernels

This paper proposes a new optical flow smoothing methodology combining vector diffusion and robust statistics. Vector smoothing using diffusion preserves moving object boundaries and the main motion discontinuities. According to a study provided in the paper, diffusion does not remove the outliers but spreads them out, introducing a bias in the neighbourhood. In this paper robust statistics operators such as the median and alpha-trimmed mean are considered for robustifying the diffusion kernels. The robust diffusion smoothing process is extended to 3-D lattices as well. The proposed algorithms are applied for smoothing artificially generated vector fields as well as the optical flow estimated from image sequences.     

Image flow segmentation and estimation by constraint lineclustering

Image flow is the velocity field in the image plane caused by the motion of the observer, objects in the scene, or apparent motion, and can contain discontinuities due to object occlusion in the scene. An algorithm that can estimate the image flow velocity field when there are discontinuities due to occlusions is described. The constraint line clustering algorithm uses a statistical test to estimate the image flow velocity field in the presence of step discontinuities in the image irradiance or velocity field. Particular emphasis is placed on motion estimation and segmentation in situations such as random dot patterns where motion is the only cue to segmentation. Experimental results on a demanding synthetic test case and a real image are presented. A smoothing algorithm for improving the velocity field estimate is also described. The smoothing algorithm constructs a smooth estimate of the velocity field by approximating a surface between step discontinuities. It is noted that the velocity field estimate can be improved using surface reconstruction between velocity field boundaries     

Adaptive diffusion flow active contours for image segmentation

Gradient vector flow (GVF) active contour model shows good performance at concavity convergence and initialization insensitivity, yet it is susceptible to weak edges as well as deep and narrow concavity. This paper proposes a novel external force, called adaptive diffusion flow (ADF), with adaptive diffusion strategies according to the characteristics of an image region in the parametric active contour model framework for image segmentation. We exploit a harmonic hypersurface minimal functional to substitute smoothness energy term in GVF for alleviating the possible leakage. We make use of the p(x) harmonic maps, in which p(x) ranges from 1 to 2, such that the diffusion process of the flow field can be adjusted adaptively according to image characteristics. We also incorporate an infinity laplacian functional to ADF active contour model to drive the active contours onto deep and narrow concave regions of objects. The experimental results demonstrate that ADF active contour model possesses several good properties, including noise robustness, weak edge preserving and concavity convergence.     

Rigid body segmentation and shape description from dense opticalflow under weak perspective

We present an algorithm for identifying and tracking independently moving rigid objects from optical flow. Some previous attempts at segmentation via optical flow have focused on finding discontinuities in the flow field. While discontinuities do indicate a change in scene depth, they do not in general signal a boundary between two separate objects. The proposed method uses the fact that each independently moving object has a unique epipolar constraint associated with its motion. Thus motion discontinuities based on self-occlusion can be distinguished from those due to separate objects. The use of epipolar geometry allows for the determination of individual motion parameters for each object as well as the recovery of relative depth for each point on the object. The algorithm assumes an affine camera where perspective effects are limited to changes in overall scale. No camera calibration parameters are required. A Kalman filter based approach is used for tracking motion parameters with time     

一种具有自适应权值系数的全局平滑光流算法

分析了Horn-Schunk方法在运动边界处，光流场不能很好地保持不连续性的原因，并从3个方面对Horn-Schunk迭代模型作了改进：（1）在能量方程中用可变的权值系数代替原来的常权值系数；（2）采用一种新方法求解迭代方程中的速度均值，新方法体现了邻域的亮度差别对速度扩散的影响；（3）引入补偿迭代方法去求解相关Euler-Lagrange方程，实验证明这种迭代方法比Gauss-Seidel方法更加有效。试验结果验证了改进的光流求解模型提高了光流场的计算精度，并能更好地保持光流场在运动边界处的不连续性。     

Accurate optical flow computation under non-uniform brightness variations

In this paper, we present a very accurate algorithm for computing optical flow with non-uniform brightness variations. The proposed algorithm is based on a generalized dynamic image model (GDIM) in conjunction with a regularization framework to cope with the problem of non-uniform brightness variations. To alleviate flow constraint errors due to image aliasing and noise, we employ a reweighted least-squares method to suppress unreliable flow constraints, thus leading to robust estimation of optical flow. In addition, a dynamic smoothness adjustment scheme is proposed to efficiently suppress the smoothness constraint in the vicinity of the motion and brightness variation discontinuities, thereby preserving motion boundaries. We also employ a constraint refinement scheme, which aims at reducing the approximation errors in the first-order differential flow equation, to refine the optical flow estimation especially for large image motions. To efficiently minimize the resulting energy function for optical flow computation, we utilize an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the large linear system. Experimental results on some synthetic and real image sequences show that the proposed algorithm compares favorably to most existing techniques reported in literature in terms of accuracy in optical flow computation with 100% density.     

System Constraints and Location Hypersurfaces in Discrete-continuous Systems

This paper presents some of the underlying principles relevant to the implementation of system constraints using signal dependent sampling, as distinct from direct signal constraints. Furthermore, the concept of a location hypersurface is introduced and related to basic sampling interval criteria. Some theorems are presented.     

一种基于线性亮度变化模型的鲁棒的光流算法

提出了一种鲁棒光流算法,用于计算光照强度、帧间运动速度及运动速度变化较大情况下的光流场。在梯度约束方程中嵌入了线性亮度变化模型,以提高大的光照强度变化下算法稳健性;将各向异性扩散方程引入空间方向平滑约束,以改善运动不连续处的流速计算精度,并依此建立了多尺度空间微分光流算法。参数的均衡化得到了线性尺度变化下的恒定能量函数。迭代运算引入运动补偿的概念,使亮度误差减小。实验结果表明,在光照强度和运动速度及速度变化较大时,本文算法具有很好的计算精度,并产生密度100%的光流场。     

Robust Optical Flow Computation Based on Least-Median-of-Squares Regression

An optical flow estimation technique is presented which is based on the least-median-of-squares (LMedS) robust regression algorithm enabling more accurate flow estimates to be computed in the vicinity of motion discontinuities. The flow is computed in a blockwise fashion using an affine model. Through the use of overlapping blocks coupled with a block shifting strategy, redundancy is introduced into the computation of the flow. This eliminates blocking effects common in most other techniques based on blockwise processing and also allows flow to be accurately computed in regions containing three distinct motions.A multiresolution version of the technique is also presented, again based on LMedS regression, which enables image sequences containing large motions to be effectively handled.An extensive set of quantitative comparisons with a wide range of previously published methods are carried out using synthetic, realistic (computer generated images of natural scenes with known flow) and natural images. Both angular and absolute flow errors are calculated for those sequences with known optical flow. Displaced frame difference error, used extensively in video compression, is used for those natural scenes with unknown flow. In all of the sequences tested, a comparison with those methods that result in a dense flow field (greater than 80% spatial coverage), show that the LMedS technique produces the least error irrespective of the error measure used.     

Motion estimation and segmentation

In the general structure-from-motion (SFM) problem involving several moving objects in a scene, the essential first step is to segment moving objects independently. We attempt to deal with the problem of optical flow estimation and motion segmentation over a pair of images. We apply a mean field technique to determine optical flow and motion boundaries and present a deterministic algorithm. Since motion discontinuities represented by line process are embedded in the estimation of the optical flow, our algorithm provides accurate estimates of optical flow especially along motion boundaries and handles occlusion and multiple motions. We show that the proposed algorithm outperforms other well-known algorithms in terms of estimation accuracy and timing.     

An extended residual-based variational multiscale method for two-phase flow including surface tension

In this study, an extended residual-based variational multiscale method is proposed for two-phase flow including surface tension. The extended residual-based variational multiscale method combines a residual-based form of the variational multiscale method and the extended finite element method (XFEM). By extending the solution spaces, it is possible to reproduce discontinuities of the solution fields inside elements intersected by the interface. In particular, we propose a quasi-static enrichment to reproduce time-dependent discontinuities. Kink enrichments of both velocity and pressure as well as kink enrichments of velocity combined with jump enrichments of pressure are considered here. To capture the interface between the phases on a fixed grid, a level-set approach is used. A residual-based variational multiscale method is employed for computing both flow and interface motion. The presented method is tested for various two-phase flow examples exhibiting small and large density and viscosity ratios, with and without surface tension: a two-phase Couette flow, a Rayleigh–Taylor instability, a sloshing tank and a three-dimensional rising bubble. To the best of our knowledge, these are the first simulation results for representative time-dependent three-dimensional two-phase flow problems using an extended finite element method. Stable and accurate results are obtained for all test examples.     

Metric-discontinuous zonal grid calculations using the Osher scheme

Computations on zonal grids—in particular, grids with metric discontinuities resulting from the interspersion of highly clustered regions with coarse regions—are possible using a fully conservative form of the Osher upwind scheme. These zonal grids can result from an abrupt clustering of points near solution discontinuities or near other flow features that require improved resolution. The zonal approach is shown to capture shocks with almost “shock-fitting” quality but with minimal effort. Results for inviscid flow, including quasi-one-dimensional nozzle flow, supersonic flow over a cylinder, and blast-wave diffraction by a ramp, are presented. These calculations demonstrate the powerful capabilities of the Osher scheme used in conjunction with zonal grids in simulating flow fields with complex shock patterns.