Object pose from 2-D to 3-D point and line correspondences

In this paper we present a method for optimally estimating the rotation and translation between a camera and a 3-D object from point and/or line correspondences. First we devise an error function and second we show how to minimize this error function. The quadratic nature of this function is made possible by representing rotation and translation with a dual number quaternion. We provide a detailed account of the computational aspects of a trust-region optimization method. This method compares favourably with Newton's method which has extensively been used to solve the problem at hand, with Faugeras-Toscani's linear method (Faugeras and Toscani 1986) for calibrating a camera, and with the Levenberg-Marquardt non-linear optimization method. Finally we present some experimental results which demonstrate the robustness of our method with respect to image noise and matching errors.This work has been supported by the Esprit programme through the SECOND project (Esprit-BRA No. 6769).     

一种高精度鲁棒的基于直线对应的位姿估计迭代算法

2D-3D特征对应位姿估计问题算法多基于点对应,而基于直线对应求解比基于点对应求解更具有优势。从欧氏空间这一新颖角度出发,提出了一种从直线对应求解位姿估计问题的迭代算法,算法思想是先迭代求解出最优的旋转矩阵,然后再得到平移向量。针对不同的直线组情形,给出了相应的迭代初始值计算方法。仿真实验数据表明,算法具有对初始值较不敏感、高精度、鲁棒性好等特点。     

Tracking 3-D motion from straight lines with trifocal tensors

We present a novel approach to track the position and orientation of a stereo camera using line features in the images. The method combines the strengths of trifocal tensors and Bayesian filtering. The trifocal tensor provides a geometric constraint to lock line features among every three frames. It eliminates the explicit reconstruction of the scene even if the 3-D scene structure is not known. Such a trifocal constraint thus makes the algorithm fast and robust. The twist motion model is applied to further improve its computation efficiency. Another major contribution is that our approach can obtain the 3-D camera motion using as little as 2 line correspondences instead of 13 in the traditional approaches. This makes the approach attractive for realistic applications. The performance of the proposed method has been evaluated using both synthetic and real data with encouraging results. Our algorithm is able to estimate 3-D camera motion in real scenarios accurately having little drifting from an image sequence longer than a 1,000 frames.     

基于ORB特征的单目视觉定位算法研究

针对大范围室外场景和具有重复、高频纹理特征(例如水泥地、草坪)的场景,提出了一种鲁棒性强、定位精度高、速度更快的视觉定位算法。采用8级图像金字塔的ORB (Oriented FAST and Rotated BRIEF)特征描述子提取图像特征点,通过K近邻(KNN)匹配相邻图像序列的特征点对,依次解算基础矩阵F和本质矩阵E,采用自适应法利用单应矩阵和本质矩阵进行位姿估计,最后解算两帧图像间相机刚体运动的旋转R和平移t,利用三角测量法则求解出匹配点的三维坐标,重建相机运动轨迹。为了提高算法性能,提出采用最小化基于点特征的非线性重投影误差优化三维点。通过调用OpenCV在C++中实现,对所采集的数据集进行测试,测试结果表明,该方法比传统的3D位姿估计更优,实时可行。由于其基于单目而实现,因此无法得到尺度信息。     

Iterative Pose Computation from Line Correspondences

This paper presents a method for estimating the position and orientation of a camera with respect to a known 3-D object from line correspondences. The main idea of the method is to estimate a pose with either a weak perspective or a paraperspective camera model and to improve this pose iteratively. At convergence the result is compatible with a perspective camera model. This iterative improvement of a linear (affine) camera model has already been used for points but has never been extended to lines. Known methods which compute pose from line correspondences deal with a set of nonlinear equations which are solved either in closed-form or using minimization techniques. These methods have to deal with multiple solutions. In contrast our method starts with a solution which is very close to the true solution and converges in very few iterations (typically three to five iterations). The rank analysis of the linear system to be solved at each iteration allows us to characterize geometric configurations which defeat the algorithm.     

一种基于4对图像对应点的欧氏重建方法

摄像机自标定算法通常是非线性的，为了得到线性的方法，提出了一种在RANSAC框架下由4对图像对应点线性标定摄像机并对场景进行鲁棒性欧氏重建的方法。当摄像机作两组平移运动时，若在两组平移运动之间摄像机具有不同的姿态，则从4对图像对应点可以线性地重建场景的欧氏几何。模拟实验和真实图像实验均证明了本文方法的可行性。     

Two-View Orthographic Epipolar Geometry: Minimal and Optimal Solvers

We will in this paper present methods and algorithms for estimating two-view geometry based on an orthographic camera model. We use a previously neglected nonlinear criterion on rigidity to estimate the calibrated essential matrix. We give efficient algorithms for estimating it minimally (using only three point correspondences), in a least squares sense (using four or more point correspondences), and optimally with respect to the number of inliers. The inlier-optimal algorithm is based on a three-point solver and gives a fourth-order polynomial time algorithm. These methods can be used as building blocks to robustly find inlier correspondences in the presence of high degrees of outliers. We show experimentally that our methods can be used in many instances, where the orthographic camera model isn’t generally used. A case of special interest is situations with repetitive structures, which give high amounts of outliers in the initial feature point matching.     

Some properties of the E matrix in two-view motionestimation

In the eight-point linear algorithm for determining 3D motion/structure from two perspective views using point correspondences, the E matrix plays a central role. The E matrix is defined as a skew-symmetrical matrix (containing the translation components) postmultiplied by a rotation matrix. The authors show that a necessary and sufficient condition for a 3×3 matrix to be so decomposable is that one of its singular values is zero and the other two are equal. Several other forms of this property are presented. Some applications are briefly described     

一种基于任意点对的相机标定方法

提出了一种基于任意点对的几何标定方法,用于标定相机外部参数．该方法考虑点对而不是单个点与相机的几何关系．首先根据三个标定点对估计旋转矩阵,然后根据相机和一个点对的几何关系直接计算平移向量．整个标定过程无需任何点对的绝对位置信息,而且对点对的位置关系也没有任何限制．试验结果表明了该方法的有效性．     

Appearance-Based Structure from Motion Using Linear Classes of 3-D Models

In this paper, we address the problem of recovering 3-D models from sequences of partly calibrated images with unknown correspondence. To that end, we integrate tracking, structure from motion with geometric constraints (specifically in the form of linear class models) in a single framework. The key to making the proposed approach work is the use of appearance-based model matching and refinement which updates the estimated correspondences on each iteration of the algorithm. Another key feature is the matching of a 3-D model directly with the input images without the conventional 2-step approach of stereo data recovery and 3-D model fitting. Initialization of the linear class model to one of the input images (the reference image) is currently partly manual.This synthesis and refine approach, or appearance-based constrained structure from motion (AbCSfm), is especially useful in recovering shapes of objects whose general structureis known but which may have little discernable texture in significant parts of their surfaces. We applied the proposed approach to 3-D face modeling from multiple images to create new 3-D faces for DECface, a synthetic talking head developed at Cambridge Research Laboratory, Digital Equipment Corporation. The DECface model comprises a collection of 3-D triangular and rectangular facets, with nodes as vertices. In recovering the DECface model, we assume that the sequence of images is taken with a camera with unknown focal length and pose. The geometric constraints used are of the form of linear combination of prototypes of 3-D faces of real people. Results of this approach show its good convergence properties and its robustness against cluttered backgrounds.     

Lines and Points in Three Views and the Trifocal Tensor

This paper discusses the basic role of the trifocal tensor in scene reconstruction from three views. This 3× 3× 3 tensor plays a role in the analysis of scenes from three views analogous to the role played by the fundamental matrix in the two-view case. In particular, the trifocal tensor may be computed by a linear algorithm from a set of 13 line correspondences in three views. It is further shown in this paper, that the trifocal tensor is essentially identical to a set of coefficients introduced by Shashua to effect point transfer in the three view case. This observation means that the 13-line algorithm may be extended to allow for the computation of the trifocal tensor given any mixture of sufficiently many line and point correspondences. From the trifocal tensor the camera matrices of the images may be computed, and the scene may be reconstructed. For unrelated uncalibrated cameras, this reconstruction will be unique up to projectivity. Thus, projective reconstruction of a set of lines and points may be carried out linearly from three views.     

Efficient and accurate image based camera registration

A technique for efficient and accurate camera registration based on stereo image analysis is presented. Initially, few correspondences are estimated with high accuracy using a probabilistic relaxation technique. Accuracy is achieved by considering the continuous approximations of selected image areas using second order polynomials and a relaxation rule defined according to the likelihood that estimates obey stereoscopic constraints. The extrinsic camera parameters are then obtained using a novel efficient and robust approach derived from the classic eight point algorithm. Efficiency is achieved by solving a parametric linear optimization problem rather than a nonlinear one as more conventional methods attempt to do. Robustness is obtained by applying two novel strategies: normalization of the initial data via a simple but efficient diagonal scaling approach, and regularization of the underlying linear parametric optimization problem using meaningful constraints. The performance of the presented methods is assessed in several computer experiments using natural video data.     

Object Pose: The Link between Weak Perspective,Paraperspective, and Full Perspective

Recently, DeMenthon and Davis (1992, 1995) proposed a method for determining the pose of a 3-D object with respect to a camera from 3-D to 2-D point correspondences. The method consists of iteratively improving the pose computed with a weak perspective camera model to converge, at the limit, to a pose estimation computed with a perspective camera model. In this paper we give an algebraic derivation of DeMenthon and Davis' method and we show that it belongs to a larger class of methods where the perspective camera model is approximated either at zero order (weak perspective) or first order (paraperspective). We describe in detail an iterative paraperspective pose computation method for both non coplanar and coplanar object points. We analyse the convergence of these methods and we conclude that the iterative paraperspective method (proposed in this paper) has better convergence properties than the iterative weak perspective method. We introduce a simple way of taking into account the orthogonality constraint associated with the rotation matrix. We analyse the sensitivity to camera calibration errors and we define the optimal experimental setup with respect to imprecise camera calibration. We compare the results obtained with this method and with a non-linear optimization method.     

3-D motion estimation using a sequence of noisy stereo images:models, estimation, and uniqueness results

A kinematic model-based approach for the estimation of 3-D motion parameters from a sequence of noisy stereo images is discussed. The approach is based on representing the constant acceleration translational motion and constant precession rotational motion in the form of a bilinear state-space model using standard rectilinear states for translation and quaternions for rotation. Closed-form solutions of the state transition equations are obtained to propagate the quaternions. The measurements are noisy perturbations of 3-D feature points represented in an inertial coordinate system. It is assumed that the 3-D feature points are extracted from the stereo images and matched over the frames. Owing to the nonlinearity in the state model, nonlinear filters are designed for the estimation of motion parameters. Simulation results are included. The Cramer-Rao performance bounds for motion parameter estimates are computed. A constructive proof for the uniqueness of motion parameters is given. It is shown that with uniform sampling in time, three noncollinear feature points in five consecutive binocular image pairs contain all the spatial and temporal information. Both nondegenerate and degenerate motions are analyzed. A deterministic algorithm to recover motion parameters from a stereo image sequence is summarized from the constructive proof     

Three-Dimensional Reconstruction of Points and Lines with Unknown Correspondence across Images

Three-dimensional reconstruction from a set of images is an important and difficult problem in computer vision. In this paper, we address the problem of determining image feature correspondences while simultaneously reconstructing the corresponding 3D features, given the camera poses of disparate monocular views. First, two new affinity measures are presented that capture the degree to which candidate features from different images consistently represent the projection of the same 3D point or 3D line. An affinity measure for point features in two different views is defined with respect to their distance from a hypothetical projected 3D pseudo-intersection point. Similarly, an affinity measure for 2D image line segments across three views is defined with respect to a 3D pseudo-intersection line. These affinity measures provide a foundation for determining unknown correspondences using weighted bipartite graphs representing candidate point and line matches across different images. As a result of this graph representation, a standard graph-theoretic algorithm can provide an optimal, simultaneous matching and triangulation of points across two views, and lines across three views. Experimental results on synthetic and real data demonstrate the effectiveness of the approach.An erratum to this article can be found at     

自由角度显示系统下的运动向量算法

为了解决三维运动矢量计算的问题,提出一种算法,这种算法是在静态三维重建的基础上用二维图片计算刚体运动的线性方法,利用QR分解求解线性方程的最小均方误差解,迭代来消除离群值,来得到运动矢量.这种算法不需要摄像机标定,实时性好,有利于视频的流畅显示,实验结果表明,此种算法具有一定的鲁棒性.     

Self-Calibration of a Moving Camera from Point Correspondences and Fundamental Matrices

We address the problem of estimating three-dimensional motion, and structure from motion with an uncalibrated moving camera. We show that point correspondences between three images, and the fundamental matrices computed from these point correspondences, are sufficient to recover the internal orientation of the camera (its calibration), the motion parameters, and to compute coherent perspective projection matrices which enable us to reconstruct 3-D structure up to a similarity. In contrast with other methods, no calibration object with a known 3-D shape is needed, and no limitations are put upon the unknown motions to be performed or the parameters to be recovered, as long as they define a projective camera.The theory of the method, which is based on the constraint that the observed points are part of a static scene, thus allowing us to link the intrinsic parameters and the fundamental matrix via the absolute conic, is first detailed. Several algorithms are then presented, and their performances compared by means of extensive simulations and illustrated by several experiments with real images.     

一种基于TOF相机与CCD相机的联合标定算法研究

针对基于Time-of-Flight(TOF)相机的彩色目标三维重建需标定CCD相机与TOF相机联合系统的几何参数,在研究现有的基于彩色图像和TOF深度图像标定算法的基础上,提出了一种基于平面棋盘模板的标定方法。拍摄了固定在平面标定模板上的彩色棋盘图案在不同角度下的彩色图像和振幅图像,改进了Harris角点提取,根据棋盘格上角点与虚拟像点的共轭关系,建立了相机标定系统模型,利用Levenberg-Marquardt算法求解,进行了标定实验。获取了TOF与CCD相机内参数,并利用像平面之间的位姿关系估计两相机坐标系的相对姿态,最后进行联合优化,获取了相机之间的旋转矩阵与平移向量。实验结果表明,提出的算法优化了求解过程,提高了标定效率,能够获得较高的精度。     

Affine structure from line correspondences with uncalibrated affinecameras

This paper presents a linear algorithm for recovering 3D affine shape and motion from line correspondences with uncalibrated affine cameras. The algorithm requires a minimum of seven line correspondences over three views. The key idea is the introduction of a one-dimensional projective camera. This converts 3D affine reconstruction of “line directions” into 2D projective reconstruction of “points”. In addition, a line-based factorization method is also proposed to handle redundant views. Experimental results both on simulated and real image sequences validate the robustness and the accuracy of the algorithm