Dense versus Sparse Approaches for Estimating the Fundamental Matrix

There are two main strategies for solving correspondence problems in computer vision: sparse local feature based approaches and dense global energy based methods. While sparse feature based methods are often used for estimating the fundamental matrix by matching a small set of sophistically optimised interest points, dense energy based methods mark the state of the art in optical flow computation. The goal of our paper is to show that this separation into different application domains is unnecessary and can be bridged in a natural way. As a first contribution we present a new application of dense optical flow for estimating the fundamental matrix. Comparing our results with those obtained by feature based techniques we identify cases in which dense methods have advantages over sparse approaches. Motivated by these promising results we propose, as a second contribution, a new variational model that recovers the fundamental matrix and the optical flow simultaneously as the minimisers of a single energy functional. In experiments we show that our coupled approach is able to further improve the estimates of both the fundamental matrix and the optical flow. Our results prove that dense variational methods can be a serious alternative even in classical application domains of sparse feature based approaches.     

Optical flow estimation: an error analysis of gradient-based methods with local optimization

Multiple views of a scene can provide important information about the structure and dynamic behavior of three-dimensional objects. Many of the methods that recover this information require the determination of optical flow-the velocity, on the image, of visible points on object surfaces. An important class of techniques for estimating optical flow depend on the relationship between the gradients of image brightness. While gradient-based methods have been widely studied, little attention has been paid to accuracy and reliability of the approach. Gradient-based methods are sensitive to conditions commonly encountered in real imagery. Highly textured surfaces, large areas of constant brightness, motion boundaries, and depth discontinuities can all be troublesome for gradient-based methods. Fortunately, these problematic areas are usually localized can be identified in the image. In this paper we examine the sources of errors for gradient-based techniques that locally solve for optical flow. These methods assume that optical flow is constant in a small neighborhood. The consequence of violating in this assumption is examined. The causes of measurement errors and the determinants of the conditioning of the solution system are also considered. By understanding how errors arise, we are able to define the inherent limitations of the technique, obtain estimates of the accuracy of computed values, enhance the performance of the technique, and demonstrate the informative value of some types of error.     

Piecewise-Smooth Dense Optical Flow via Level Sets

We propose a new algorithm for dense optical flow computation. Dense optical flow schemes are challenged by the presence of motion discontinuities. In state of the art optical flow methods, over-smoothing of flow discontinuities accounts for most of the error. A breakthrough in the performance of optical flow computation has recently been achieved by Brox et~al. Our algorithm embeds their functional within a two phase active contour segmentation framework. Piecewise-smooth flow fields are accommodated and flow boundaries are crisp. Experimental results show the superiority of our algorithm with respect to alternative techniques. We also study a special case of optical flow computation, in which the camera is static. In this case we utilize a known background image to separate the moving elements in the sequence from the static elements. Tests with challenging real world sequences demonstrate the performance gains made possible by incorporating the static camera assumption in our algorithm.     

Histogram-Based Optical Flow for Motion Estimation in Ultrasound Imaging

Motion estimation on ultrasound data is often referred to as ‘Speckle Tracking’ in clinical environments and plays an important role in diagnosis and monitoring of cardiovascular diseases and the identification of abnormal cardiac motion. The impact of physical effects in the process of data acquisition raises many problems for conventional image processing techniques. The most significant difference to other medical data is its high level of speckle noise, which has completely different characteristics from other noise models, e.g., additive Gaussian noise. In this paper we address the problem of multiplicative speckle noise for motion estimation techniques that are based on optical flow methods and prove that the influence of this noise leads to wrong correspondences between image regions if not taken into account. To overcome these problems we propose the use of local statistics and introduce an optical flow method which uses histograms as discrete representations of local statistics for motion analysis. We show that this approach is more robust under the presence of speckle noise than classical optical flow methods.     

Multi-objective optimization for parameter selection and characterization of optical flow methods

Optical flow methods are among the most accurate techniques for estimating displacement and velocity fields in a number of applications that range from neuroscience to robotics. The performance of any optical flow method will naturally depend on the configuration of its parameters, and for different applications there are different trade-offs between the corresponding evaluation criteria (e.g. the accuracy and the processing speed of the estimated optical flow). Beyond the standard practice of manual selection of parameters for a specific application, in this article we propose a framework for automatic parameter setting that allows searching for an approximated Pareto-optimal set of configurations in the whole parameter space. This final Pareto-front characterizes each specific method, enabling proper method comparison and proper parameter selection. Using the proposed methodology and two open benchmark databases, we study two recent variational optical flow methods. The obtained results clearly indicate that the method to be selected is application dependent, that in general method comparison and parameter selection should not be done using a single evaluation measure, and that the proposed approach allows to successfully perform the desired method comparison and parameter selection.     

一种局部和全局相结合的光流计算方法

光流场是计算机视觉的一个研究方向,微分法是计算光流场的一个常用方法,它分为全局方法和局部方法,全局方法能够得到100％的致密的光流场,而局部方法大多只能得到稀疏的光流场,但它在噪声情况下具有更好的鲁棒性。本文提出一种局部和全局相结合的方法．首先给出五点光流约束的局部方法,再结合全局方法,计算得到了既致密又 鲁棒的光流场。     

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

The accuracy of optical flow estimation algorithms has been improving steadily as evidenced by results on the Middlebury optical flow benchmark. The typical formulation, however, has changed little since the work of Horn and Schunck. We attempt to uncover what has made recent advances possible through a thorough analysis of how the objective function, the optimization method, and modern implementation practices influence accuracy. We discover that “classical” flow formulations perform surprisingly well when combined with modern optimization and implementation techniques. One key implementation detail is the median filtering of intermediate flow fields during optimization. While this improves the robustness of classical methods it actually leads to higher energy solutions, meaning that these methods are not optimizing the original objective function. To understand the principles behind this phenomenon, we derive a new objective function that formalizes the median filtering heuristic. This objective function includes a non-local smoothness term that robustly integrates flow estimates over large spatial neighborhoods. By modifying this new term to include information about flow and image boundaries we develop a method that can better preserve motion details. To take advantage of the trend towards video in wide-screen format, we further introduce an asymmetric pyramid downsampling scheme that enables the estimation of longer range horizontal motions. The methods are evaluated on the Middlebury, MPI Sintel, and KITTI datasets using the same parameter settings.     

一种基于PSO的改进光流算法

针对变分光流法无法有效检测由间断、遮挡等因素造成的错误光流分量的缺陷,提出一种基于PSO(Particle Swarm Optimization)的光流算法。该方法在Classic+NL算法模型的基础上计算出光流后,引入前向光流和后向光流的运动一致性理论来判断遮挡区域,并通过基于PSO的修补法来实现对遮挡区域错误光流的有效修补,同时,利用邻域光流修补法实现了再次修补。实验结果表明,该方法能有效克服由间断、遮挡等因素造成的错误光流分量的缺陷,更准确地刻画出光流,提高光流的计算精度。     

基于改进光流场模型的医学图像配准方法

基于光流场模型的图像配准方法计算简单快速,但采用原始光流场模型进行图像配准会使图像出现严重的模糊导致不能使用。提出了对原始光流场模型的正则项进行改进,同时引入运动模糊图像复原算法,改进的算法改善了原始光流场模型造成的图像模糊。实验结果表明,基于改进光流场模型的医学图像配准算法配准结果准确,具有较快的配准速度。     

Over-Parameterized Variational Optical Flow

A novel optical flow estimation process based on a spatio-temporal model with varying coefficients multiplying a set of basis functions at each pixel is introduced. Previous optical flow estimation methodologies did not use such an over parameterized representation of the flow field as the problem is ill-posed even without introducing any additional parameters: Neighborhood based methods of the Lucas–Kanade type determine the flow at each pixel by constraining the flow to be described by a few parameters in small neighborhoods. Modern variational methods represent the optic flow directly via the flow field components at each pixel. The benefit of over-parametrization becomes evident in the smoothness term, which instead of directly penalizing for changes in the optic flow, accumulates a cost of deviating from the assumed optic flow model. Our proposed method is very general and the classical variational optical flow techniques are special cases of it, when used in conjunction with constant basis functions. Experimental results with the novel flow estimation process yield significant improvements with respect to the best results published so far.