Recursive estimation of 3D motion and surface structure from local affine flow parameters

A recursive structure from motion algorithm based on optical flow measurements taken from an image sequence is described. It provides estimates of surface normal in addition to 3D motion and depth. The measurements are affine motion parameters which approximate the local flow fields associated with near-planar surface patches in the scene. These are integrated over time to give estimates of the 3D parameters using an extended Kalman filter. This also estimates the camera focal length and, so, the 3D estimates are metric. The use of parametric measurements means that the algorithm is computationally less demanding than previous optical flow approaches and the recursive filter builds in a degree of noise robustness. Results of experiments on synthetic and real image sequences demonstrate that the algorithm performs well.     

Reactive control of zoom while fixating using perspective and affine cameras

This paper describes reactive visual methods of controlling the zoom setting of the lens of an active camera while fixating upon an object. The first method assumes a perspective projection and adjusts zoom to preserve the ratio of focal length to scene depth. The active camera is constrained to rotate, permitting self-calibration from the image motion of points on the static background. A planar structure from motion algorithm is used to recover the depth of the foreground. The foreground-background segmentation exploits the properties of the two different interimage homographies which are observed. The fixation point is updated by transfer via the observed planar structure. The planar method is shown to work on real imagery, but results from simulated data suggest that its extension to general 3D structure is problematical under realistic viewing and noise regimes. The second method assumes an affine projection. It requires no self-calibration and the zooming camera may move generally. Fixation is again updated using transfer, but now via the affine structure recovered by factorization. Analysis of the projection matrices allows the relative scale of the affine bases in different views to be found in a number of ways and, hence, controlled to unity. The various ways are compared and the best used on real imagery captured from an active camera fitted with a controllable zoom lens in both look-move and continuous operation.     

Affine Approximation for Direct Batch Recovery of Euclidian Structure and Motion from Sparse Data

We present a batch method for recovering Euclidian camera motion from sparse image data. The main purpose of the algorithm is to recover the motion parameters using as much of the available information and as few computational steps as possible. The algorithm thus places itself in the gap between factorisation schemes, which make use of all available information in the initial recovery step, and sequential approaches which are able to handle sparseness in the image data. Euclidian camera matrices are approximated via the affine camera model, thus making the recovery direct in the sense that no intermediate projective reconstruction is made. Using a little known closure constraint, the F_A-closure, we are able to formulate the camera coefficients linearly in the entries of the affine fundamental matrices. The novelty of the presented work is twofold: Firstly the presented formulation allows for a particularly good conditioning of the estimation of the initial motion parameters but also for an unprecedented diversity in the choice of possible regularisation terms. Secondly, the new autocalibration scheme presented here is in practice guaranteed to yield a Least Squares Estimate of the calibration parameters. As a bi-product, the affine camera model is rehabilitated as a useful model for most cameras and scene configurations, e.g. wide angle lenses observing a scene at close range. Experiments on real and synthetic data demonstrate the ability to reconstruct scenes which are very problematic for previous structure from motion techniques due to local ambiguities and error accumulation.     

基于光流法的定点监控虚拟预置位图像提取方法

大范围定点监控存在摄像头预置位不足以覆盖全部监控区域的问题，而依靠摄像头本身的三维定位功能抓取监控点图像速度太慢，且可能存在定位误差。针对上述问题，提出了一种用于土地违法与违章建筑大范围定点监控的虚拟预置位图像提取及配准方法。对前后两帧图像进行灰度转化和中值滤波去除噪声点，使用基于金字塔的Lucas-Kanade光流算法计算前一帧图像强角点的光流，通过计算仿射变换得到帧间运动矢量；对相邻帧间运动矢量进行积分，得到每一帧图像总的运动矢量；通过提取出运动矢量与虚拟预置位运动矢量最接近的那一帧，得到所需的新时相虚拟预置位图像。实验结果表明，本文算法可以更快速地提取出同一监控点不同时相的监控对比图像，图像重叠率优于95%，不论是重叠率还是图像质量都可以很好地满足日常监控的需求。     

Quasi-perspective Projection Model: Theory and Application to Structure and Motion Factorization from Uncalibrated Image Sequences

This paper addresses the problem of factorization-based 3D reconstruction from uncalibrated image sequences. Previous studies on structure and motion factorization are either based on simplified affine assumption or general perspective projection. The affine approximation is widely adopted due to its simplicity, whereas the extension to perspective model suffers from recovering projective depths. To fill the gap between simplicity of affine and accuracy of perspective model, we propose a quasi-perspective projection model for structure and motion recovery of rigid and nonrigid objects based on factorization framework. The novelty and contribution of this paper are as follows. Firstly, under the assumption that the camera is far away from the object with small lateral rotations, we prove that the imaging process can be modeled by quasi-perspective projection, which is more accurate than affine model from both geometrical error analysis and experimental studies. Secondly, we apply the model to establish a framework of rigid and nonrigid factorization under quasi-perspective assumption. Finally, we propose an Extended Cholesky Decomposition to recover the rotation part of the Euclidean upgrading matrix. We also prove that the last column of the upgrading matrix corresponds to a global scale and translation of the camera thus may be set freely. The proposed method is validated and evaluated extensively on synthetic and real image sequences and improved results over existing schemes are observed.     

On 3-D scene flow and structure recovery from multiview image sequences

Two novel systems computing dense three-dimensional (3-D) scene flow and structure from multiview image sequences are described in this paper. We do not assume rigidity of the scene motion, thus allowing for nonrigid motion in the scene. The first system, integrated model-based system (IMS), assumes that each small local image region is undergoing 3-D affine motion. Non-linear motion model fitting based on both optical flow constraints and stereo constraints is then carried out on each local region in order to simultaneously estimate 3-D motion correspondences and structure. The second system is based on extended gradient-based system (EGS), a natural extension of two-dimensional (2-D) optical flow computation. In this method, a new hierarchical rule-based stereo matching algorithm is first developed to estimate the initial disparity map. Different available constraints under a multiview camera setup are further investigated and utilized in the proposed motion estimation. We use image segmentation information to adopt and maintain the motion and depth discontinuities. Within the framework for EGS, we present two different formulations for 3-D scene flow and structure computation. One formulation assumes that initial disparity map is accurate, while the other does not. Experimental results on both synthetic and real imagery demonstrate the effectiveness of our 3-D motion and structure recovery schemes. Empirical comparison between IMS and EGS is also reported.     

Depth recovery and affine reconstruction under camera pure translation

This paper addresses the problems of depth recovery and affine reconstruction from two perspective images, which are generated by an uncalibrated translating camera. Firstly, we develop a new constraint that the homography for the plane, which is orthogonal to the optical axis, is determined only by the epipole and the plane's relative distance to the origin under camera pure translation. The algorithm of depth recovery is based on this new constraint, and it can successfully avoid the step of camera calibration. With the recovered depth, we show that affine reconstruction can be obtained readily. The proposed affine reconstruction does not need any control points, which were used to expand the affine coordinate system in existing method. Therefore, it could avoid the step of non-planarity verification as well as the errors from the control points. Error analysis is also presented to evaluate the uncertainty for the recovered depth value. Finally, we have tested the proposed algorithm with both simulated data and real image data. And the results show that the proposed algorithm is accurate and practical.     

Trackability as a cue for potential obstacle identification and 3-D description

In many man-made environments, obstacles in the path of a mobile robot can be characterized asshallow, that is, they have relatively small extent in depth compared to the distance from the camera. We present a framework for segmenting shallow structures from their background over a sequence of images. Shallowness is first quantified asaffine describability. This is embedded in a tracking system within which hypothesized model structures undergo a cycle of prediction and model-matching. Structures emerge either as shallow or nonshallow based on theiraffine trackability. Two major contributions of this work are (i) aggregate object tracking based on 3-D motion and structure constraints in constrast with traditional primitive feature tracking based on image motion heuristics, and (ii) use of temporal behavior for object segmentation and 3-D reconstruction.     

Affine structure from line correspondences with uncalibrated affinecameras

This paper presents a linear algorithm for recovering 3D affine shape and motion from line correspondences with uncalibrated affine cameras. The algorithm requires a minimum of seven line correspondences over three views. The key idea is the introduction of a one-dimensional projective camera. This converts 3D affine reconstruction of “line directions” into 2D projective reconstruction of “points”. In addition, a line-based factorization method is also proposed to handle redundant views. Experimental results both on simulated and real image sequences validate the robustness and the accuracy of the algorithm