多视点图像中线特征的三维重建

将多视点图像中的离散线段(称为线特征)用于场景的三维重建过程.首先将平均重投影几何误差作为线特征三角形法和光束平差法的目标函数,保证了重建结果的最优性;其次提出一种对含噪声的空间直线Plucker坐标进行双线性约束矫正的解析方法,并给出理论证明;最后推导出针对具有4自由度空间直线的最少参数化迭代方法,消除了优化过程中的过参数化问题,以避免由于内约束存在而导致迭代难于收敛的情况,提高了重建结果的精度.仿真数据以及真实图像的实验结果验证了该方法的有效性和精确性.     

Plücker直线描述的空间后方交会

目的 现有的空间后方交会方法没有充分利用成像光线的几何拓扑关系,共线条件的几何约束性弱,提出基于Plücker坐标的空间后方交会方法.方法 将成像光线用Plücker直线表示,通过Plücker成像光线在空间中的螺旋位移,使得物点在像点和投影中心所确定的Plücker直线上,建立基于Plücker成像光线的共线条件方程.结果 针对小倾角航空与近景影像,本文方法与传统的欧拉角法相比求解得到的外方位元素最大线元素误差分别从0.88 m降到0.53 m,从0.06 m降到0.01 m;最大角元素误差分别从0.016 2弧度降到0.002 9弧度,从0.087弧度降到0.066弧度.针对大倾角影像,欧拉角法不收敛而本文方法能正确解算.结论 本文方法由于增强了模型的几何强度,获得了更高的定位精度,且对于大倾角影像也同样能够获得正确而稳定的外方位元素解算结果.     

The 3D Line Motion Matrix and Alignment of Line Reconstructions

We study the problem of aligning two 3D line reconstructions in projective, affine, metric or Euclidean space.We introduce the 6 × 6 3D line motion matrix that acts on Plücker coordinates. We characterize its algebraic properties and its relation to the usual 4 × 4 point motion matrix, and propose various methods for estimating 3D motion from line correspondences, based on cost functions defined in images or 3D space. We assess the quality of the different estimation methods using simulated data and real images.     

Weak-perspective structure from motion by fast alternation

This paper addresses the problem of moving object reconstruction. Several methods have been published in the past 20 years including stereo reconstruction as well as multi-view factorization methods. In general, reconstruction algorithms compute the 3D structure of the object and the camera parameters in a non-optimal way, and then a nonlinear and numerical optimization algorithm refines the reconstructed camera parameters and 3D coordinates. In this paper, we propose an adjustment method which is the improved version of the well-known Tomasi–Kanade factorization method. The novelty, which yields the high speed of the algorithm, is that the core of the proposed method is an alternation and we give optimal solutions to the subproblems in the alternation. The improved method is discussed here and it is compared to the widely used bundle adjustment algorithm.     

Suboptimal Solutions to the Algebraic-Error Line Triangulation

Line triangulation, a foundational problem in computer vision, is to estimate the 3D line position from a set of measured image lines with known camera projection matrices. Aiming to improve the triangulation’s efficiency, in this work, two algorithms are proposed to find suboptimal solutions under the algebraic-error optimality criterion of the Plücker line coordinates. In these proposed algorithms, the algebraic-error optimality criterion is reformulated by the transformation of the Klein constraint. By relaxing the quadratic unit norm constraint to six linear constraints, six new single-quadric-constraint optimality criteria are constructed in the new formulation, whose optimal solutions can be obtained by solving polynomial equations. Moreover, we prove that the minimum algebraic error of either the first three or the last three of the six new criteria is not more than \(\sqrt{3}\) times of that of the original algebraic-error optimality criterion. Thus, with three new criteria and all the six criteria, suboptimal solutions under the algebraic error minimization and the geometric error minimization are obtained. Experimental results show the effectiveness of our proposed algorithms.     

Three-Dimensional Reconstruction of Points and Lines with Unknown Correspondence across Images

Three-dimensional reconstruction from a set of images is an important and difficult problem in computer vision. In this paper, we address the problem of determining image feature correspondences while simultaneously reconstructing the corresponding 3D features, given the camera poses of disparate monocular views. First, two new affinity measures are presented that capture the degree to which candidate features from different images consistently represent the projection of the same 3D point or 3D line. An affinity measure for point features in two different views is defined with respect to their distance from a hypothetical projected 3D pseudo-intersection point. Similarly, an affinity measure for 2D image line segments across three views is defined with respect to a 3D pseudo-intersection line. These affinity measures provide a foundation for determining unknown correspondences using weighted bipartite graphs representing candidate point and line matches across different images. As a result of this graph representation, a standard graph-theoretic algorithm can provide an optimal, simultaneous matching and triangulation of points across two views, and lines across three views. Experimental results on synthetic and real data demonstrate the effectiveness of the approach.An erratum to this article can be found at     

Grassmann—Cayley algebra and robotics

Plücker coordinates are a suitable way to represent lines in space for purposes of studying instantaneous motions of robot arms. Similarly, we can represent any affine subspace of Euclidean space of any dimension. This can be useful for studying motion spaces of robot arms. In this paper, after we introduce Plücker coordinates and develop a few of their basic properties, we present them in a somewhat more abstract setting, called the Grassmann—Cayley algebra. Several other applications of this algebra are also described.     

Trajectory triangulation: 3D reconstruction of moving points from amonocular image sequence

We consider the problem of reconstructing the 3D coordinates of a moving point seen from a monocular moving camera, i.e., to reconstruct moving objects from line-of-sight measurements only. The task is feasible only when some constraints are placed on the shape of the trajectory of the moving point. We coin the family of such tasks as “trajectory triangulation.” We investigate the solutions for points moving along a straight-line and along conic-section trajectories, We show that if the point is moving along a straight line, then the parameters of the line (and, hence, the 3D position of the point at each time instant) can be uniquely recovered, and by linear methods, from at least five views. For the case of conic-shaped trajectory, we show that generally nine views are sufficient for a unique reconstruction of the moving point and fewer views when the conic is of a known type (like a circle in 3D Euclidean space for which seven views are sufficient). The paradigm of trajectory triangulation, in general, pushes the envelope of processing dynamic scenes forward. Thus static scenes become a particular case of a more general task of reconstructing scenes rich with moving objects (where an object could be a single point)     

单轴运动下未标定图像序列的度量重构

提出了一种由单轴运动下的未标定运动图像序列进行三维重构的新方法。利用不变量计算出第一幅图像对应的摄像机矩阵,根据转台的旋转角度计算出其余图像对应的摄像机矩阵,再使用SIFT算法查找相邻两幅图像可见的匹配点,并利用线性三角形法对图像进行重构,最后用捆集调整优化重构的结果。真实图像序列的重构实验验证了算法的正确性和有效性。     

任意视角的多视图立体匹配系统

为了得到高精度深度图,通过特征点提取与匹配、计算基础矩阵F、引导互匹配、捆集调整等一系列技术,求出每幅视图在同一空间坐标系下的高精度摄像机矩阵P.为任意角度的多个视图建立可扩展的主框架,不需要图像矫正.基于图像坐标和自动计算出的视差范围建立三维网络,为每条边分配适当的权值,把多视图立体匹配问题转化为网络最大流问题.算法保证了高准确率和平滑性,实验结果表明此方法适用于三维模型重建.     

Deformation visual inspection of industrial parts with image sequence

A new approach to reconstructing and inspecting the deformation of industrial parts, especially sheetmetal parts based on CAD-designed data and hybrid point-line bundle adjustment with image sequence, is proposed. Nonmetric image sequence and CAD-designed data of parts are used as the source of information. The strategy of our approach is to reconstruct and inspect deformations of parts automatically with image points and line segments extracted from the images. Basic error equation of line photogrammetry and its modified form are addressed in detail. It is shown that when a certain proper weight is selected, adjustment by condition equations and adjustment by observation equations are equivalent for line photogrammetry. A novel hybrid point-line bundle adjustment algorithm is used to reconstruct industrial parts. The proposed hybrid adjustment model can be used in various 3D reconstruction applications of objects mainly composed of points and lines. The developed inspection system is tested with true image data acquired by a CCD camera, and the results are very satisfying.Received: 27 May 2002, Accepted: 18 September 2003, Published online: 8 June 2004     

Large scale multiple robot visual mapping with heterogeneous landmarks in semi-structured terrain

This paper addresses the cooperative localization and visual mapping problem with multiple heterogeneous robots. The approach is designed to deal with the challenging large semi-structured outdoors environments in which aerial/ground ensembles are to evolve. We propose the use of heterogeneous visual landmarks, points and line segments, to achieve effective cooperation in such environments. A large-scale SLAM algorithm is generalized to handle multiple robots, in which a global graph maintains the relative relationships between a series of local sub-maps built by the different robots. The key issue when dealing with multiple robots is to find the link between them, and to integrate these relations to maintain the overall geometric consistency; the events that introduce these links on the global graph are described in detail. Monocular cameras are considered as the primary extereoceptive sensor. In order to achieve the undelayed initialization required by the bearing-only observations, the well-known inverse-depth parametrization is adopted to estimate 3D points. Similarly, to estimate 3D line segments, we present a novel parametrization based on anchored Plücker coordinates, to which extensible endpoints are added. Extensive simulations show the proposed developments, and the overall approach is illustrated using real-data taken with a helicopter and a ground rover.     

Image mosaicing using sequential bundle adjustment

We describe the construction of accurate panoramic mosaics from multiple images taken with a rotating camera, or alternatively of a planar scene. The novelty of the approach lies in (i) the transfer of photogrammetric bundle adjustment techniques to mosaicing; (ii) a new representation of image line measurements enabling the use of lines in camera self-calibration, including computation of the radial and other non-linear distortion; and (iii) the application of the variable state dimension filter to obtain efficient sequential updates of the mosaic as each image is added.We demonstrate that our method achieves better results than the alternative approach of optimising over pairs of images.     

Optimal camera placement for accurate reconstruction

Three-dimensional (3D) measurements can be recovered from several views by triangulation. This paper deals with the problem of where to place the cameras in order to obtain a minimal error in the 3D measurements, also called camera network design in photogrammetry. We pose the problem in terms of an optimization design, dividing it into two main components: (1) an analytical part dedicated to the analysis of error propagation from which a criterion is derived, and (2) a global optimization process to minimize this criterion. In this way, the approach consists of an uncertainty analysis applied to the reconstruction process from which a covariance matrix is computed. This matrix represents the uncertainty of the detection from which the criterion is derived. Moreover, the optimization has discontinuities due to the presence of occluding surfaces between the viewpoint and the object point group, which leads to a combinatorial optimization process. These aspects are solved using a multi-cellular genetic algorithm. Experimental results are provided to illustrate the effectiveness and efficiency of the solution.     

针对大规模点集三维重建问题的分布式捆绑调整方法

捆绑调整(Bundle adjustment, BA)是三维重建中的关键步骤,它需要消耗大量的计算时间和内存存储空间.本 文旨在处理三维点数比相机模型数多很多的捆绑调整问题,我们称之为针对大规模三维点集的捆 绑调整(Massive-points bundle adjustment, MPBA)问题.此类问题在对高分辨率图像进行三 维重建时会经常出现.为了高效地解决MPBA问题,本文提出一种分布式的捆绑调整算法.通过基 于三维点集划分的分解方法,原MPBA问题被分成若干子问题.该分解方法不依赖于输入参数的内在 联系,因而分解结果与具体BA问题无关.算法被映射于两个集群上,一个集群有5台计算机,另一个集 群有3台计算机,其中每台机器都配置一块图形处理器(Graphic processing unit, GPU).通过对若 干MPBA问题的实验,与经典捆绑调整算法SBA (Sparse bundle adjustment)相比,本文算法获得了 最高达75倍的加速比,并保持了算法的高精确度.而且,本文算法的两个实现所消耗的单机内存存储 空间,仅为SBA实现的1/7和1/4.     

基于序列图像的鲁棒三维重建方法

提出了基于序列图像的鲁棒三维重建方法。首先利用两幅图像的最优参数估计,然后添加新图像并采用稀疏调整,减少图像坐标测量值的最小几何误差。通过对三维结构和摄像机参数进行全局优化处理,以提高重建的鲁棒性。实验结果表明,该方法提高了重建的精度和鲁棒性,并真实地再现了物体的三维模型。     

Keyframe-based recognition and localization during video-rate parallel tracking and mapping

Generating situational awareness by augmenting live imagery with collocated scene information has applications from game-playing to military command and control. We propose a method of object recognition, reconstruction, and localization using triangulation of SIFT features from keyframe camera poses in a 3D map. The map and keyframe poses themselves are recovered at video-rate by bundle adjustment of FAST image features in the parallel tracking and mapping algorithm. Detected objects are automatically labeled on the user's display using predefined annotations. Experimental results are given for laboratory scenes, and in more realistic applications.     

基于二维凸多边形内散乱点的三角划分新算法

基于给定的平面散点数据,提出了逐层提取轮廓线,并将轮廓线之间的区域进行三角划分的新算法。实现这一算法的关键是在给定阈值的条件下逐层提取内部离散点的轮廓线,再在所提取的轮廓线间进行等比例三角划分。最后,利用Lawson提出的局部优化过程来优化分割好的三角形网。通过实验证明了算法的有效性。     

一种非定标图像高精度三维重建算法

由非定标图像重建三维场景有着广泛的应用。给出了一种非定标多视图像三维重建算法。该算法主要基于因子分解和光束法平差技术。首先用因子分解方法得到射影空间下相机投影矩阵和物点坐标,以旋转矩阵的正交性以及对偶绝对二次曲面秩为3为约束,将射影空间升级到欧式空间,最后用光束法平差进行优化。该方法可同时获得相机的内外参数、畸变系数和场景的三维坐标。仿真实验表明,在1000 mm×1000 mm×400mm的范围内,当像点检测误差在0-1pixel和0-2pixel内,所重建三维点的误差分别为0.1530 mm和0.6712 mm。在500 mm×500 m×200 mm下,真实实验重构三维点的误差在0.3 mm以内。所提出的算法稳定可靠,可对实际工程进行指导。     

Non-Rigid Stereo Factorization

In this paper we address the problem of recovering 3D non-rigid structure from a sequence of images taken with a stereo pair. We have extended existing non-rigid factorization algorithms to the stereo camera case and presented an algorithm to decompose the measurement matrix into the motion of the left and right cameras and the 3D shape, represented as a linear combination of basis-shapes. The added constraints in the stereo camera case are that both cameras are viewing the same structure and that the relative orientation between both cameras is fixed. Our focus in this paper is on the recovery of flexible 3D shape rather than on the correspondence problem. We propose a method to compute reliable 3D models of deformable structure from stereo images. Our experiments with real data show that improved reconstructions can be achieved using this method. The algorithm includes a non-linear optimization step that minimizes image reprojection error and imposes the correct structure to the motion matrix by choosing an appropriate parameterization. We show that 3D shape and motion estimates can be successfully disambiguated after bundle adjustment and demonstrate this on synthetic and real image sequences. While this optimization step is proposed for the stereo camera case, it can be readily applied to the case of non-rigid structure recovery using a monocular video sequence. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.