Integrating Automated Range Registration with Multiview Geometry for the Photorealistic Modeling of Large-Scale Scenes

The photorealistic modeling of large-scale scenes, such as urban structures, requires a fusion of range sensing technology and traditional digital photography. This paper presents a system that integrates automated 3D-to-3D and 2D-to-3D registration techniques, with multiview geometry for the photorealistic modeling of urban scenes. The 3D range scans are registered using our automated 3D-to-3D registration method that matches 3D features (linear or circular) in the range images. A subset of the 2D photographs are then aligned with the 3D model using our automated 2D-to-3D registration algorithm that matches linear features between the range scans and the photographs. Finally, the 2D photographs are used to generate a second 3D model of the scene that consists of a sparse 3D point cloud, produced by applying a multiview geometry (structure-from-motion) algorithm directly on a sequence of 2D photographs. The last part of this paper introduces a novel algorithm for automatically recovering the rotation, scale, and translation that best aligns the dense and sparse models. This alignment is necessary to enable the photographs to be optimally texture mapped onto the dense model. The contribution of this work is that it merges the benefits of multiview geometry with automated registration of 3D range scans to produce photorealistic models with minimal human interaction. We present results from experiments in large-scale urban scenes.     

Plane-based registration of construction laser scans with 3D/4D building models

With the development of building information modelling (BIM) and terrestrial laser scanning (TLS) in the architecture, engineering, construction and facility management (AEC/FM) industry, the registration of site laser scans and project 3D (BIM) models in a common coordinate system is becoming critical to effective project control. The co-registration of 3D datasets is normally performed in two steps: coarse registration followed by fine registration. Focusing on the coarse registration, model-scan registration has been well investigated in the past, but it is shown in this article that the context of the AEC/FM industry presents specific (1) constraints that make fully-automated registration very complex and often ill-posed, and (2) advantages that can be leveraged to develop simpler yet effective registration methods.This paper thus presents a novel semi-automated plane-based registration system for coarse registration of laser scanned 3D point clouds with project 3D models in the context of the AEC/FM industry. The system is based on the extraction of planes from the laser scanned point cloud and project 3D/4D model. Planes are automatically extracted from the 3D/4D model. For the point cloud data, two methods are investigated. The first one is fully automated, and the second is a semi-automated but effective one-click RANSAC-supported extraction method. In both cases, planes are then manually but intuitively matched by the user. Experiments, which compare the proposed system to software packages commonly used in the AEC/FM industry, demonstrate that at least as good registration quality can be achieved by the proposed system, in a simpler and faster way. It is concluded that, in the AEC/FM context, the proposed plane-based registration system is a compelling alternative to standard point-based registration techniques.     

互信息医学图像配准的实时性研究

基于互信息的医学图像配准是一种高精稳健的自动配准算法,可以达到亚像素级精度且无需提取解剖特征而倍受重视,但其最大问题是速度慢,致使其不能满足临床的实时需求。在分析影响其速度因素的基础上提出一套加速方案,即采用快速粗配准来缩小互信息的搜索范围、利用非等间隔的灰度压缩来加快互信息的计算、通过混合遗传算法和单纯形算法来加快互信息的搜索。实验表明,改进后的算法在保证配准精度的前提下能显著提高配准速度。     

Registration of multiple overlapping range images: scenes withoutdistinctive features

A scheme is developed to register range images in an environment where distinctive features are scarce. Another issue addressed here is conflicting situations that may arise from pair-wise registration of multiple overlapping range images whether or not they contain distinctive features. There can be two causes for this: (a) error in individual registrations or (b) compression or bending in range images. The authors develop a scheme for resolving such conflicts for the case where range images share a common reference surface, i.e. when the transformation matrix between two overlapping images involves only three components: two translations and one rotation. The authors implemented this scheme to map the floor of the ocean, where the range data is obtained by a multibeam echo-sounder system installed aboard a sailing ship producing multiple overlapping range images. The system developed is the first automated system for correctly registered mapping of the ocean floor; it is efficient and robust     

A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization

Registration of 3D data is a key problem in many applications in computer vision, computer graphics and robotics. This paper provides a family of minimal solutions for the 3D-to-3D registration problem in which the 3D data are represented as points and planes. Such scenarios occur frequently when a 3D sensor provides 3D points and our goal is to register them to a 3D object represented by a set of planes. In order to compute the 6 degrees-of-freedom transformation between the sensor and the object, we need at least six points on three or more planes. We systematically investigate and develop pose estimation algorithms for several configurations, including all minimal configurations, that arise from the distribution of points on planes. We also identify the degenerate configurations in such registrations. The underlying algebraic equations used in many registration problems are the same and we show that many 2D-to-3D and 3D-to-3D pose estimation/registration algorithms involving points, lines, and planes can be mapped to the proposed framework. We validate our theory in simulations as well as in three real-world applications: registration of a robotic arm with an object using a contact sensor, registration of planar city models with 3D point clouds obtained using multi-view reconstruction, and registration between depth maps generated by a Kinect sensor.     

Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans

We present a registration algorithm for pairs of deforming and partial range scans that addresses the challenges of non‐rigid registration within a single non‐linear optimization. Our algorithm simultaneously solves for correspondences between points on source and target scans, confidence weights that measure the reliability of each correspondence and identify non‐overlapping areas, and a warping field that brings the source scan into alignment with the target geometry. The optimization maximizes the region of overlap and the spatial coherence of the deformation while minimizing registration error. All optimization parameters are chosen automatically; hand‐tuning is not necessary. Our method is not restricted to part‐in‐whole matching, but addresses the general problem of partial matching, and requires no explicit prior correspondences or feature points. We evaluate the performance and robustness of our method using scan data acquired by a structured light scanner and compare our method with existing non‐rigid registration algorithms.     

基于位平面的超光谱图像无损压缩算法的实现

分析了超光谱遥感图像的特征,根据它对压缩算法的特殊要求,提出了基于位平面的无损压缩算法.对于相关性较高的高位位平面采用计算位平面的差值矩阵直接去相关,而相关性较差的低位位平面则采用四叉树划分的方法重组各个像块的大小、位置、灰度信息,从而得到图像的混合编码.实验结果表明,该算法与其它常用无损压缩算法压缩比相当,但压缩时间提高了50%左右.该算法简单实用,适合有实时性要求的超光谱遥感图像压缩.     

医学图像配准中的一种互信息快速计算方法

基于互信息的医学图像配准,可以达到亚像素级精度且无需提取图像的解剖特征,是一种高精稳健的配准方法。但其中频繁的互信息计算使配准速度很慢,不能满足临床的实时要求。提出一种基于统计直方图灰度压缩的互信息加速计算方法,能显著减少灰度级,加快互信息计算,从而加快配准速度。实验表明,该方法能在不影响配准精度的前提下显著缩短配准时间。     

基于法向量投票的点云配准方法

针对多平面结构的物体,传统的点特征点云配准方法存在鲁棒性差、易收敛到局部最优解等问题,提出了一种基于法向量投票的点云配准方法。用平面特征代替点特征作为配准基元,建立基于平面的坐标转换模型。首先构建kd-tree,计算各点的法向量,并将法向量转换到霍夫空间进行投票,提取平面特征;然后将单位四元数作为特征描述算子,以同名平面特征作为约束条件,根据最小二乘平差原则,求解点云之间的位姿变换关系。实验结果表明：相较于其他两种方法,提出方法对初始位置没有依赖性,在配准过程中可以有效避免局部最小陷阱,并且配准精度得到了提高。     

Supine and prone colon registration using quasi-conformal mapping

In virtual colonoscopy, CT scans are typically acquired with the patient in both supine (facing up) and prone (facing down) positions. The registration of these two scans is desirable so that the user can clarify situations or confirm polyp findings at a location in one scan with the same location in the other, thereby improving polyp detection rates and reducing false positives. However, this supine-prone registration is challenging because of the substantial distortions in the colon shape due to the patient's change in position. We present an efficient algorithm and framework for performing this registration through the use of conformal geometry to guarantee that the registration is a diffeomorphism (a one-to-one and onto mapping). The taeniae coli and colon flexures are automatically extracted for each supine and prone surface, employing the colon geometry. The two colon surfaces are then divided into several segments using the flexures, and each segment is cut along a taenia coli and conformally flattened to the rectangular domain using holomorphic differentials. The mean curvature is color encoded as texture images, from which feature points are automatically detected using graph cut segmentation, mathematic morphological operations, and principal component analysis. Corresponding feature points are found between supine and prone and are used to adjust the conformal flattening to be quasi-conformal, such that the features become aligned. We present multiple methods of visualizing our results, including 2D flattened rendering, corresponding 3D endoluminal views, and rendering of distortion measurements. We demonstrate the efficiency and efficacy of our registration method by illustrating matched views on both the 2D flattened colon images and in the 3D volume rendered colon endoluminal view. We analytically evaluate the correctness of the results by measuring the distance between features on the registered colons.     

Weighted motion averaging for the registration of multi-view range scans

Multi-view registration is a fundamental but challenging task in 3D reconstruction and robot vision. Although the original motion averaging algorithm has been introduced as an effective means to solve the multi-view registration problem, it does not consider the reliability and accuracy of each relative motion. Accordingly, this paper proposes a novel motion averaging algorithm for multi-view registration. Firstly, it utilizes the pair-wise registration algorithm to estimate the relative motion and overlapping percentage of each scan pair with a certain degree of overlap. With the overlapping percentage available, it views the overlapping percentage as the corresponding weight of each scan pair and proposes the weighted motion averaging algorithm, which can pay more attention to reliable and accurate relative motions. By treating each relative motion distinctively, more accurate registration can be achieved by applying the weighted motion averaging to multi-view range scans. Experimental results demonstrate the superiority of our proposed approach compared with the state-of-the-art methods in terms of accuracy, robustness and efficiency.     

6D scan registration using depth-interpolated local image features

This paper describes a novel registration approach that is based on a combination of visual and 3D range information. To identify correspondences, local visual features obtained from images of a standard color camera are compared and the depth of matching features (and their position covariance) is determined from the range measurements of a 3D laser scanner. The matched depth-interpolated image features allow one to apply registration with known correspondences. We compare several ICP variants in this paper and suggest an extension that considers the spatial distance between matching features to eliminate false correspondences. Experimental results are presented in both outdoor and indoor environments. In addition to pair-wise registration, we also propose a global registration method that registers all scan poses simultaneously.     

Polar-Cartesian Hybrid Transforms: A novel 2D range scan registration algorithm

Registration of range scans is commonly required in many localization and mapping algorithms. In this paper, we introduce a novel approach called Polar-Cartesian Hybrid Transforms for pair-wise registration of range scans. The proposed algorithm iteratively establishes correspondences by searching the points with closest polar angles in the polar coordinate frame. An angular look-up table is constructed based on the properties of the laser range finder to accelerate the searching procedure. In order to speed up the convergence, we compute the difference of polar range of every matched point pair to select the most contributing correspondences. After the correspondences are determined, the transformation is computed in Cartesian coordinate frame using a point-to-line metric. Combining the advantages of the polar and Cartesian coordinate frames, both robustness and efficiency are greatly improved compared with an up-to-date ICP algorithm.     

Scan-based volume animation driven by locally adaptive articulated registrations

This paper describes a complete system to create anatomically accurate example-based volume deformation and animation of articulated body regions, starting from multiple in vivo volume scans of a specific individual. In order to solve the correspondence problem across volume scans, a template volume is registered to each sample. The wide range of pose variations is first approximated by volume blend deformation (VBD), providing proper initialization of the articulated subject in different poses. A novel registration method is presented to efficiently reduce the computation cost while avoiding strong local minima inherent in complex articulated body volume registration. The algorithm highly constrains the degrees of freedom and search space involved in the nonlinear optimization, using hierarchical volume structures and locally constrained deformation based on the biharmonic clamped spline. Our registration step establishes a correspondence across scans, allowing a data-driven deformation approach in the volume domain. The results provide an occlusion-free person-specific 3D human body model, asymptotically accurate inner tissue deformations, and realistic volume animation of articulated movements driven by standard joint control estimated from the actual skeleton. Our approach also addresses the practical issues arising in using scans from living subjects. The robustness of our algorithms is tested by their applications on the hand, probably the most complex articulated region in the body, and the knee, a frequent subject area for medical imaging due to injuries.     

Robust point correspondence applied to two- and three-dimensionalimage registration

Accurate and robust correspondence calculations are very important in many medical and biological applications. Often, the correspondence calculation forms part of a rigid registration algorithm, but accurate correspondences are especially important for elastic registration algorithms and for quantifying changes over time. In this paper, a new correspondence calculation algorithm, CSM (correspondence by sensitivity to movement), is described. A robust corresponding point is calculated by determining the sensitivity of a correspondence to movement of the point being matched. If the correspondence is reliable, a perturbation in the position of this point should not result in a large movement of the correspondence. A measure of reliability is also calculated. This correspondence calculation method is independent of the registration transformation and has been incorporated into both a 2D elastic registration algorithm for warping serial sections and a 3D rigid registration algorithm for registering pre and postoperative facial range scans. These applications use different methods for calculating the registration transformation and accurate rigid and elastic alignment of images has been achieved with the CSM method. It is expected that this method will be applicable to many different applications and that good results would be achieved if it were to be inserted into other methods for calculating a registration transformation from correspondences     

改进ICP算法实现多视点云精确配准研究

复杂面形的三维整体测量能否顺利完成取决于不同视下测得的三维点云的配准精度。研究表明:采用点到点,点到三角面配准方法易受噪声干扰,采用面形比较计算量大,且在平面和标准球面情况下容易失效。以粗配准标记点所在的立方体区域为重合区域,使用点到点的多邻接三角面距离最近的点对作为初始匹配点,并根据几何结构最大相似原则对所求得的多个粗匹配点对进行筛选,再对筛选后的点对应用最近点迭代(ICP)算法。改进后的ICP算法实现了重合区域的快速自动定位,实现了不同视下点云的快速精确配准,在多个实例下获得了配准精度优于0.01 mm的实验结果。     

Perception and Grasping of Object Parts from Active Robot Exploration

Like humans, robots that need semantic perception and accurate estimation of the environment can increase their knowledge through active interaction with objects. This paper proposes a novel method for 3D object modeling for a robot manipulator with an eye-in-hand laser range sensor. Since the robot can only perceive the environment from a limited viewpoint, it actively manipulates a target object and generates a complete model by accumulation and registration of partial views. Three registration algorithms are investigated and compared in experiments performed in cluttered environments with complex rigid objects made of multiple parts. A data structure based on proximity graph, that encodes neighborhood relations in range scans, is also introduced to perform efficient range queries. The proposed method for 3D object modeling is applied to perform task-level manipulation. Indeed, once a complete model is available the object is segmented into its constituent parts and categorized. Object sub-parts that are relevant for the task and that afford a grasping action are identified and selected as candidate regions for grasp planning.     

利用信息融合方法的三维特征提取

本文提出了一种基于信息融合的物体三维特征的提取方法，该方法利用两幅互相配准的三维测距图像和灰度图像，来提取多面体的三维特征。首先，通过分析灰度图像中的灰度变化及测距图像中的测距值变化，分别求取各自图像中物体的特征点及特征边；然后，利用两配准图像之间的对应关系，求得所有特征点、面与多边形在三维测距图像中的三维表示；接着，通过分析三维测距图像中所测得的各候选平面上特定点与边处的曲率及法向，验证候选平面