移动机器人基于三维激光测距的室内场景认知

研究了移动机器人在室内三维环境中的场景认知问题.室内场景框架具有结构化特性,而室 内多样化的物体则难以进行模型化表述. 本文利用区域扩张算法进行平面特征的提取,并根据平面属性及其相互间的空间关系,完成室 内场景框架的辨识.为了借鉴图像处理领域的物体识别方法, 本文提出一种基于Bearing Angle模型的激光测距数据表述方法,从而将三维点云数据转换为二维Bearing Angle图. 同一类物体中的个体形态具有多样性,同时观测视角也导致激光测距数据的显著差异.针对这些 问题,采用一种基于Gentleboost算法的有监督学习方法, 并利用物体碎片及其相对于物体中心的位置作为特征,从而完成室内场景中的物体认知. 利用室内场景框架辨识结果在Bearing Angle图中进行天棚、地面、墙壁、房门等区域的标记,并利用所产生的语义信息去除错误的认知结果,从而有助于提高识别率. 利用实际机器人平台所获得的实验结果验证了所提方法的有效性.     

Three-dimensional point cloud alignment detecting fiducial markers by structured light stereo imaging

In recent years, various methodologies of shape reconstruction have been proposed with the aim at creating Computer-Aided Design models by digitising physical objects using optical sensors. Generally, the acquisition of 3D geometrical data includes crucial tasks, such as planning scanning strategies and aligning different point clouds by multiple view approaches, which differ for user’s interaction and hardware cost. This paper describes a methodology to automatically measure three-dimensional coordinates of fiducial markers to be used as references to align point clouds obtained by an active stereo vision system based on structured light projection. Intensity-based algorithms and stereo vision principles are combined to detect passive fiducial markers localised in a scene. 3D markers are uniquely recognised on the basis of geometrical similarities. The correlation between fiducial markers and point clouds allows the digital creation of complete object surfaces. The technology has been validated by experimental tests based on nominal benchmarks and reconstructions of target objects with complex shapes.     

Acquiring a complete 3D model from specular motion under theillumination of circular-shaped light sources

We recover 3D models of objects with specular surfaces. An object is rotated and its continuous images are taken. Circular-shaped light sources that generate conic rays are used to illuminate the rotating object in such a way that highlighted stripes can be observed on most of the specular surfaces. Surface shapes can be computed from the motions of highlights in the continuous images; either specular motion stereo or single specular trace mode can be used. When the lights are properly set, each point on the object can be highlighted during the rotation. The shape for each rotation plane is measured independently using its corresponding epipolar plane image. A 3D shape model is subsequently reconstructed by combining shapes at different rotation planes. Computing a shape is simple and requires only the motion of highlight on each rotation plane. The novelty of this paper is the complete modeling of a general type of specular objects that has not been accomplished before     

A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization

Registration of 3D data is a key problem in many applications in computer vision, computer graphics and robotics. This paper provides a family of minimal solutions for the 3D-to-3D registration problem in which the 3D data are represented as points and planes. Such scenarios occur frequently when a 3D sensor provides 3D points and our goal is to register them to a 3D object represented by a set of planes. In order to compute the 6 degrees-of-freedom transformation between the sensor and the object, we need at least six points on three or more planes. We systematically investigate and develop pose estimation algorithms for several configurations, including all minimal configurations, that arise from the distribution of points on planes. We also identify the degenerate configurations in such registrations. The underlying algebraic equations used in many registration problems are the same and we show that many 2D-to-3D and 3D-to-3D pose estimation/registration algorithms involving points, lines, and planes can be mapped to the proposed framework. We validate our theory in simulations as well as in three real-world applications: registration of a robotic arm with an object using a contact sensor, registration of planar city models with 3D point clouds obtained using multi-view reconstruction, and registration between depth maps generated by a Kinect sensor.     

A novel plane extraction approach using supervised learning

This paper presents a novel approach for the classification of planar surfaces in an unorganized point clouds. A feature-based planner surface detection method is proposed which classifies a point cloud data into planar and non-planar points by learning a classification model from an example set of planes. The algorithm performs segmentation of the scene by applying a graph partitioning approach with improved representation of association among graph nodes. The planarity estimation of the points in a scene segment is then achieved by classifying input points as planar points which satisfy planarity constraint imposed by the learned model. The resultant planes have potential application in solving simultaneous localization and mapping problem for navigation of an unmanned-air vehicle. The proposed method is validated on real and synthetic scenes. The real data consist of five datasets recorded by capturing three-dimensional(3D) point clouds when a RGBD camera is moved in five different indoor scenes. A set of synthetic 3D scenes are constructed containing planar and non-planar structures. The synthetic data are contaminated with Gaussian and random structure noise. The results of the empirical evaluation on both the real and the simulated data suggest that the method provides a generalized solution for plane detection even in the presence of the noise and non-planar objects in the scene. Furthermore, a comparative study has been performed between multiple plane extraction methods.     

Toward coherent object detection and scene layout understanding

Detecting objects in complex scenes while recovering the scene layout is a critical functionality in many vision-based applications. In this work, we advocate the importance of geometric contextual reasoning for object recognition. We start from the intuition that objects' location and pose in the 3D space are not arbitrarily distributed but rather constrained by the fact that objects must lie on one or multiple supporting surfaces. We model such supporting surfaces by means of hidden parameters (i.e. not explicitly observed) and formulate the problem of joint scene reconstruction and object recognition as the one of finding the set of parameters that maximizes the joint probability of having a number of detected objects on K supporting planes given the observations. As a key ingredient for solving this optimization problem, we have demonstrated a novel relationship between object location and pose in the image, and the scene layout parameters (i.e. normal of one or more supporting planes in 3D and camera pose, location and focal length). Using a novel probabilistic formulation and the above relationship our method has the unique ability to jointly: i) reduce false alarm and false negative object detection rate; ii) recover object location and supporting planes within the 3D camera reference system; iii) infer camera parameters (view point and the focal length) from just one single uncalibrated image. Quantitative and qualitative experimental evaluation on two datasets (desk-top dataset [1] and LabelMe [2]) demonstrates our theoretical claims.     

Efficient 3D object recognition using foveated point clouds

Recent hardware technologies have enabled acquisition of 3D point clouds from real world scenes in real time. A variety of interactive applications with the 3D world can be developed on top of this new technological scenario. However, a main problem that still remains is that most processing techniques for such 3D point clouds are computationally intensive, requiring optimized approaches to handle such images, especially when real time performance is required. As a possible solution, we propose the use of a 3D moving fovea based on a multiresolution technique that processes parts of the acquired scene using multiple levels of resolution. Such approach can be used to identify objects in point clouds with efficient timing. Experiments show that the use of the moving fovea shows a seven fold performance gain in processing time while keeping 91.6% of true recognition rate in comparison with state-of-the-art 3D object recognition methods.     

Textured mesh surface reconstruction of large buildings with multi-view stereo

There are three main approaches for reconstructing 3D models of buildings. Laser scanning is accurate but expensive and limited by the laser’s range. Structure-from-motion (SfM) and multi-view stereo (MVS) recover 3D point clouds from multiple views of a building. MVS methods, especially patch-based MVS, can achieve higher density than do SfM methods. Sophisticated algorithms need to be applied to the point clouds to construct mesh surfaces. The recovered point clouds can be sparse in areas that lack features for accurate reconstruction, making recovery of complete surfaces difficult. Moreover, segmentation of the building’s surfaces from surrounding surfaces almost always requires some form of manual inputs, diminishing the ease of practical application of automatic 3D reconstruction algorithms. This paper presents an alternative approach for reconstructing textured mesh surfaces from point cloud recovered by patch-based MVS method. To a good first approximation, a building’s surfaces can be modeled by planes or curve surfaces which are fitted to the point cloud. 3D points are resampled on the fitted surfaces in an orderly pattern, whose colors are obtained from the input images. This approach is simple, inexpensive, and effective for reconstructing textured mesh surfaces of large buildings. Test results show that the reconstructed 3D models are sufficiently accurate and realistic for 3D visualization in various applications.     

平行点云: 虚实互动的点云生成与三维模型进化方法

三维信息的提取在自动驾驶等智能交通场景中正发挥着越来越重要的作用, 为了解决以激光雷达为主的深度传感器在数据采集方面面临的成本高、样本覆盖不全面等问题, 本文提出了平行点云的框架. 利用人工定义场景获取虚拟点云数据, 通过计算实验训练三维模型, 借助平行执行对模型性能进行测试, 并将结果反馈至数据生成和模型训练过程. 通过不断地迭代, 使三维模型得到充分评估并不断进化. 在平行点云的框架下, 我们以三维目标检测为例, 通过闭环迭代, 构建了虚实结合的点云数据集, 在无需人工标注的情况下, 可达到标注数据训练模型精度的72%.     

3D Hough transform for sphere recognition on point clouds

Three-dimensional object recognition on range data and 3D point clouds is becoming more important nowadays. Since many real objects have a shape that could be approximated by simple primitives, robust pattern recognition can be used to search for primitive models. For example, the Hough transform is a well-known technique which is largely adopted in 2D image space. In this paper, we systematically analyze different probabilistic/randomized Hough transform algorithms for spherical object detection in dense point clouds. In particular, we study and compare four variants which are characterized by the number of points drawn together for surface computation into the parametric space and we formally discuss their models. We also propose a new method that combines the advantages of both single-point and multi-point approaches for a faster and more accurate detection. The methods are tested on synthetic and real datasets.     

Plane-based optimization for 3D object reconstruction from single line drawings

In previous optimization-based methods of 3D planar-faced object reconstruction from single 2D line drawings, the missing depths of the vertices of a line drawing (and other parameters in some methods) are used as the variables of the objective functions. A 3D object with planar faces is derived by finding values for these variables that minimize the objective functions. These methods work well for simple objects with a small number N of variables. As N grows, however, it is very difficult for them to find expected objects. This is because with the nonlinear objective functions in a space of large dimension N, the search for optimal solutions can easily get trapped into local minima. In this paper, we use the parameters of the planes that pass through the planar faces of an object as the variables of the objective function. This leads to a set of linear constraints on the planes of the object, resulting in a much lower dimensional nullspace where optimization is easier to achieve. We prove that the dimension of this nullspace is exactly equal to the minimum number of vertex depths which define the 3D object. Since a practical line drawing is usually not an exact projection of a 3D object, we expand the nullspace to a larger space based on the singular value decomposition of the projection matrix of the line drawing. In this space, robust 3D reconstruction can be achieved. Compared with two most related methods, our method not only can reconstruct more complex 3D objects from 2D line drawings, but also is computationally more efficient.     

基于3维点云鸟瞰图的高精度实时目标检测

针对基于3维点云的目标检测问题,提出了一种高精度实时的单阶段深度神经网络,分别在网络特征提取、损失函数设计和训练数据增强等3个方面提出了新的解决方案.首先对点云直接进行体素化来构建鸟瞰图.在特征提取阶段,使用残差结构提取高层语义特征,并融合多层次特征输出稠密的特征图.在回归鸟瞰图上的目标框的同时,在损失函数中考虑二次偏移量以实现更高精度的收敛.在网络训练中,使用不同帧3维点云混合的方式进行数据增强,提高网络的泛化性能.基于KITTI鸟瞰图目标检测数据集的实验结果表明,本文提出的网络仅使用雷达点云的位置信息,在性能上不仅优于目前最先进的鸟瞰图目标检测网络,而且优于融合图像和点云的检测方案,且整个网络运行速度达到20帧/秒,满足实时性要求.     

动态环境中多帧点云融合算法及三维目标检测算法研究

低线束激光雷达扫描的点云数据较为稀疏,导致无人驾驶环境感知系统中三维目标检测效果欠佳,通过多帧点云配准可实现稀疏点云稠密化,但动态环境中的行人与移动车辆会降低激光雷达的定位精度,也会造成融合帧中运动目标上的点云偏移较大。针对上述问题,提出了一种动态环境中多帧点云融合算法,利用该算法在园区道路实况下进行三维目标检测,提高了低线束激光雷达的三维目标检测精度。利用16线和40线激光雷达采集的行驶路况数据进行实验,结果表明该算法能够增强稀疏点云密度,改善低成本激光雷达的环境感知能力。     

Reconstruction of B-spline curves and surfaces by adaptive group testing

Point clouds as measurements of 3D sensors have many applications in various fields such as object modeling, environment mapping and surface representation. Storage and processing of raw point clouds is time consuming and computationally expensive. In addition, their high dimensionality shall be considered, which results in the well known curse of dimensionality. Conventional methods either apply reduction or approximation to the captured point clouds in order to make the data processing tractable. B-spline curves and surfaces can effectively represent 2D data points and 3D point clouds for most applications. Since processing all available data for B-spline curve or surface fitting is not efficient, based on the Group Testing theory an algorithm is developed that finds salient points sequentially. The B-spline curve or surface models are updated by adding a new salient point to the fitting process iteratively until the Akaike Information Criterion (AIC) is met. Also, it has been proved that the proposed method finds a unique solution so as what is defined in the group testing theory. From the experimental results the applicability and performance improvement of the proposed method in relation to some state-of-the-art B-spline curve and surface fitting methods, may be concluded.     

3D shape acquisition and integral compact representation using optical scanning and enhanced shape parameterization

An efficient computational methodology for shape acquisition, processing and representation is developed. It includes 3D computer vision by applying triangulation and stereo-photogrammetry for high-accuracy 3D shape acquisition. Resulting huge 3D point clouds are successively parameterized into mathematical surfaces to provide for compact data-set representation, yet capturing local details sufficiently. B-spline surfaces are employed as parametric entities in fitting to point clouds resulting from optical 3D scanning. Beyond the linear best-fitting algorithm with control points as fitting variables, an enhanced non-linear procedure is developed. The set of best fitting variables in minimizing the approximation error norm between the parametric surface and the 3D cloud includes the control points coordinates. However, they are augmented by the set of position parameter values which identify the respectively closest matching points on the surface for the points in the cloud. The developed algorithm is demonstrated to be efficient on demanding test cases which encompass sharp edges and slope discontinuities originating from physical damage of the 3D objects or shape complexity.     

Instance-based object recognition in 3D point clouds using discriminative shape primitives

3D local shapes are a critical cue for object recognition in 3D point clouds. This paper presents an instance-based 3D object recognition method via informative and discriminative shape primitives. We propose a shape primitive model that measures geometrical informativity and discriminativity of 3D local shapes of an object. Discriminative shape primitives of the object are extracted automatically by model parameter optimization. We achieve object recognition from 2.5/3D scenes via shape primitive classification and recover the 3D poses of the identified objects simultaneously. The effectiveness and the robustness of the proposed method were verified on popular instance-based 3D object recognition datasets. The experimental results show that the proposed method outperforms some existing instance-based 3D object recognition pipelines in the presence of noise, varying resolutions, clutter and occlusion.     

Generic fitted shapes (GFS): Volumetric object segmentation in service robotics

In this paper, a simultaneous 3D volumetric segmentation and reconstruction method, based on the so-called Generic Fitted Shapes (GFS) is proposed. The aim of this work is to cope with the lack of volumetric information encountered in visually controlled mobile manipulation systems equipped with stereo or RGB-D cameras. Instead of using primitive volumes, such as cuboids or cylinders, for approximating objects in point clouds, their volumetric structure has been estimated based on fitted generic shapes. The proposed GFSs can capture the shapes of a broad range of object classes without the need of large a-priori shape databases. The fitting algorithm, which aims at determining the particular geometry of each object of interest, is based on a modified version of the active contours approach extended to the 3D Cartesian space. The proposed volumetric segmentation system produces comprehensive closed object surfaces which can be further used in mobile manipulation scenarios. Within the experimental setup, the proposed technique has been evaluated against two state-of-the-art methods, namely superquadrics and 3D Object Retrieval (3DOR) engines.     

Point cloud occlusion recovery with shallow feedforward neural networks

3D scenes reconstructed from point clouds, acquired by either laser scanning or photogrammetry, are subject to data voids generated by occluding objects. Modeling from incomplete data is usually a manual process in which human interpretation plays an essential role. This paper presents a machine learning algorithm based on neural networks capable of recovering point cloud occlusions for surfaces that can be approximated with injective functions. Starting from the point clouds acquired around the occlusion, a set of single-layer feedforward networks with a variable number of neurons is trained and validated with a subset of the original cloud, which is preliminarily decimated using local curvature to reduce CPU cost. The averaged result of the best neural networks is evaluated on a spatial domain that contains the 2D projection of the void, obtaining a complete 3D point cloud for the occluded volume. Criteria for choosing the number of neurons and the activation function for hidden and output layers are illustrated and discussed. Results are presented for both simulated and real occlusions, describing the pros and cons of the proposed method.     

三维激光扫描表面数据区域分割

针对现有三维激光扫描数据区域分割算法受原始碎片表面粗糙度影响较大且只适用于形状较规则、表面较平坦及断裂面较少的物体这一问题,提出了区域膨胀策略的三维扫描表面数据区域分割算法,该算法将三维激光扫描表面数据分割成若干个具有相同法矢方向的区域。首先将三维扫描表面数据转化为三维网格模型;然后利用同一区域中相邻网格具有相似法线方向这一性质,使用区域膨胀策略生成若干获选表面区域;最后通过去除候选区域中的噪声区域得到最终表面区域分割结果。通过实物表面扫描数据对上述算法进行仿真验证,结果表明该算法可对三维表面扫描数据进行有效的区域分割。