Area-wide roof plane segmentation in airborne LiDAR point clouds

Most algorithms performing segmentation of 3D point cloud data acquired by, e.g. Airborne Laser Scanning (ALS) systems are not suitable for large study areas because the huge amount of point cloud data cannot be processed in the computer’s main memory. In this study a new workflow for seamless automated roof plane detection from ALS data is presented and applied to a large study area. The design of the workflow allows area-wide segmentation of roof planes on common computer hardware but leaves the option open to be combined with distributed computing (e.g. cluster and grid environments). The workflow that is fully implemented in a Geographical Information System (GIS) uses the geometrical information of the 3D point cloud and involves four major steps: (i) The whole dataset is divided into several overlapping subareas, i.e. tiles. (ii) A raster based candidate region detection algorithm is performed for each tile that identifies potential areas containing buildings. (iii) The resulting building candidate regions of all tiles are merged and those areas overlapping one another from adjacent tiles are united to a single building area. (iv) Finally, three dimensional roof planes are extracted from the building candidate regions and each region is treated separately. The presented workflow reduces the data volume of the point cloud that has to be analyzed significantly and leads to the main advantage that seamless area-wide point cloud based segmentation can be performed without requiring a computationally intensive algorithm detecting and combining segments being part of several subareas (i.e. processing tiles). A reduction of 85% of the input data volume for point cloud segmentation in the presented study area could be achieved, which directly decreases computation time.     

3D object recognition method with multiple feature extraction from LiDAR point clouds

During autonomous driving, fast and accurate object recognition supports environment perception for local path planning of unmanned ground vehicles. Feature extraction and object recognition from large-scale 3D point clouds incur massive computational and time costs. To implement fast environment perception, this paper proposes a 3D recognition system with multiple feature extraction from light detection and ranging point clouds modified by parallel computing. Effective object feature extraction is a necessary step prior to executing an object recognition procedure. In the proposed system, multiple geometry features of a point cloud that resides in corresponding voxels are computed concurrently. In addition, a scale filter is employed to convert feature vectors from uncertain count voxels to a normalized object feature matrix, which is convenient for object-recognizing classifiers. After generating the object feature matrices of all voxels, an initialized multilayer neural network (NN) model is trained offline through a large number of iterations. Using the trained NN model, real-time object recognition is realized using parallel computing technology to accelerate computation.

     

Urban building roof segmentation from airborne lidar point clouds

This article presents a new approach to segmenting building rooftops from airborne lidar point clouds. A progressive morphological filter technique is first applied for separation between ground and non-ground points. For the non-ground points, a region-growing algorithm based on a plane-fitting technique is used to separate building points from vegetation points. Then, an adaptive Random Sample Consensus (RANSAC) algorithm based on a grid structure is developed to improve the probability of selecting an uncontained sample from the localized sampling. The distance, standard deviation and normal vector are integrated to keep topological consistency among building rooftop patches during building rooftop segmentation. Finally, the remaining points are mapped on to the extracted planes by a post-processing technique to improve the segmentation accuracy. The results for buildings with different roof complexities are presented and evaluated.     

Productive high-complexity 3D city modeling with point clouds collected from terrestrial LiDAR

To satisfy the needs of photo-realistic and ground-based representation of three-dimensional (3D) city models for a variety of applications, significant efforts have been made to automatically reconstruct detailed 3D building façades from terrestrial LiDAR data. Nonetheless, in real-world applications for high-quality 3D city modeling, three major problems are typically encountered: (1) very low productivity due to fully manual operation, (2) low geometric accuracy of 3D modeling resulting from the process of reducing original LiDAR data, and (3) system failure when importing huge LiDAR data to 3D drawing software. To overcome these limitations, the present study proposes a semi-automatic method entailing a plane component detection based on RANSAC segmentation, boundary tracing of the planar components, and manual drawing of details using the remaining, significantly reduced points. The proposed method was applied to point clouds of various buildings in a high-density area in Korea. In comparison with manual operation, the proposed method was proved to improve modeling productivity in the time-consumption aspect and to facilitate operators’ accurate object drawing. However, for additional automation and completeness of 3D modeling, further study is necessary. The proposed method requires a segmentation algorithm to heuristically determine parameters for the most desirable results as well as to detect curvilinear surfaces in modeling complex and curved façades.     

结合图像分割和点云分割的障碍物检测算法

针对单目图像检测障碍物的低可靠性和当前双目视觉检测障碍物的局限性的问题,提出一种结合图像分割和点云分割技术的双目视觉障碍物检测方法。通过设定检测深度范围,分割障碍物点云与道路点云;采用将分割出的障碍物点云对应的视差图与图像分割得到的子图进行比较的策略,有效解决对不同深度、倾斜面和不规则障碍物检测效果差的问题。通过实验验证了在获得稀疏三维点云的情况下,该方法对障碍物的检测具有较好的鲁棒性。     

Object class segmentation of massive 3D point clouds of urban areas using point cloud topology

A large number of remote-sensing techniques and image-based photogrammetric approaches allow an efficient generation of massive 3D point clouds of our physical environment. The efficient processing, analysis, exploration, and visualization of massive 3D point clouds constitute challenging tasks for applications, systems, and workflows in disciplines such as urban planning, environmental monitoring, disaster management, and homeland security. We present an approach to segment massive 3D point clouds according to object classes of virtual urban environments including terrain, building, vegetation, water, and infrastructure. The classification relies on analysing the point cloud topology; it does not require per-point attributes or representative training data. The approach is based on an iterative multi-pass processing scheme, where each pass focuses on different topological features and considers already detected object classes from previous passes. To cope with the massive amount of data, out-of-core spatial data structures and graphics processing unit (GPU)-accelerated algorithms are utilized. Classification results are discussed based on a massive 3D point cloud with almost 5 billion points of a city. The results indicate that object-class-enriched 3D point clouds can substantially improve analysis algorithms and applications as well as enhance visualization techniques.     

Slice-based building facade reconstruction from 3D point clouds

ABSTRACT

We propose a comprehensive strategy to reconstruct urban building geometry from three-dimensional (3D) point clouds. First, the point clouds are segmented using a rough-detail segmentation algorithm, and refinements guided by topological relationships are performed to rectify the segmentation mistakes. Then, the semantic features (such as facades and windows) that belong to the buildings are recognized and extracted. Next, each facade is cut into a sequence of slices. The initial models are recovered by sequentially detecting and connecting the anchor points. Finally, due to the regular arrangements of windows, a template-matching method relying on the similarity and repetitiveness of the windows is proposed to recover the details on building facades. The experimental results demonstrate that our method can automatically reconstruct the building geometry and detailed window structures are better depicted.     

基于机载激光雷达点云的交互式树木分割与建模方法研究

机载激光雷达扫描(ALS)系统可大规模获取地表树木点云,有助于较高精度树木结构参数提取和景观层面的几何重建,然而树木复杂的拓扑结构和树种的多样性给树木的准确分割与建模带来挑战.传统基于点云的自动树木分割和建模算法虽然效率高,但存在分割误差较大、建模鲁棒性较差等问题,难以满足深度学习大背景下用户对树木分割与建模结果进行精...     

Complete residential urban area reconstruction from dense aerial LiDAR point clouds

We present an automatic system to reconstruct 3D urban models for residential areas from aerial LiDAR scans. The key difference between downtown area modeling and residential area modeling is that the latter usually contains rich vegetation. Thus, we propose a robust classification algorithm that effectively classifies LiDAR points into trees, buildings, and ground. The classification algorithm adopts an energy minimization scheme based on the 2.5D characteristic of building structures: buildings are composed of opaque skyward roof surfaces and vertical walls, making the interior of building structures invisible to laser scans; in contrast, trees do not possess such characteristic and thus point samples can exist underneath tree crowns. Once the point cloud is successfully classified, our system reconstructs buildings and trees respectively, resulting in a hybrid model representing the 3D urban reality of residential areas.     

Urban building reconstruction from raw LiDAR point data

We present a method for automatic reconstruction of the volumetric structures of urban buildings, directly from raw LiDAR point clouds. Given the large-scale LiDAR data from a group of urban buildings, we take advantage of the “divide-and-conquer” strategy to decompose the entire point clouds into a number of subsets, each of which corresponds to an individual building. For each urban building, we determine its upward direction and partition the corresponding point data into a series of consecutive blocks, achieved by investigating the distributions of feature points of the building along the upward direction. Next, we propose a novel algorithm, Spectral Residual Clustering (SRC), to extract the primitive elements within the contours of blocks from the sectional point set, which is formed by registering the series of consecutive slicing points. Subsequently, we detect the geometric constraints among primitive elements through individual fitting, and perform constrained fitting over all primitive elements to obtain the accurate contour. On this basis, we execute 3D modeling operations, like extrusion, lofting or sweeping, to generate the 3D models of blocks. The final accurate 3D models are generated by applying the union Boolean operations over the block models. We evaluate our reconstruction method on a variety of raw LiDAR scans to verify its robustness and effectiveness.     

卫星云图中热带风暴云区的三维绘制技术研究

实现热带风暴场景的模拟对于灾害预防、评估等工作具有十分重要的意义。改进了一种基于图像色彩和灰度值的算法,能够较好地从卫星云图中分割提取出热带风暴的云层区域,进而利用一定规则生成了云区体数据。针对生成体数据的特点,采用了一种改进的快速光线投射算法来完成热带风暴云区的三维绘制,最终实现了对于云区的任意角度实时观察和动态绘制。     

Processing and interactive editing of huge point clouds from 3D scanners

This paper describes a new out-of-core multi-resolution data structure for real-time visualization, interactive editing and externally efficient processing of large point clouds. We describe an editing system that makes use of the novel data structure to provide interactive editing and preprocessing tools for large scanner data sets. Using the new data structure, we provide a complete tool chain for 3D scanner data processing, from data preprocessing and filtering to manual touch-up and real-time visualization. In particular, we describe an out-of-core outlier removal and bilateral geometry filtering algorithm, a toolset for interactive selection, painting, transformation, and filtering of huge out-of-core point-cloud data sets and a real-time rendering algorithm, which all use the same data structure as storage backend. The interactive tools work in real-time for small model modifications. For large scale editing operations, we employ a two-resolution approach where editing is planned in real-time and executed in an externally efficient offline computation afterwards. We evaluate our implementation on example data sets of sizes up to 63 GB, demonstrating that the proposed technique can be used effectively in real-world applications.     

Automatic segmentation of point clouds from multi-view reconstruction using graph-cut

In multi-view reconstruction systems, the recovered point cloud often consists of numerous unwanted background points. We propose a graph-cut based method for automatically segmenting point clouds from multi-view reconstruction. Based on the observation that the object of interest is likely to be central to the intended multi-view images, our method requires no user interaction except two roughly estimated parameters of objects covering in the central area of images. The proposed segmentation process is carried out in two steps: first, we build a weighted graph whose nodes represent points and edges that connect each point to its k-nearest neighbors. The potentials of each point being object and background are estimated according to distances between its projections in images and the corresponding image centers. The pairwise potentials between each point and its neighbors are computed according to their positions, colors and normals. Graph-cut optimization is then used to find the initial binary segmentation of object and background points. Second, to refine the initial segmentation, Gaussian mixture models (GMMs) are created from the color and density features of points in object and background classes, respectively. The potentials of each point being object and background are re-calculated based on the learned GMMs. The graph is updated and the segmentation of point clouds is improved by graph-cut optimization. The second step is iterated until convergence. Our method requires no manual labeling points and employs available information of point clouds from multi-view systems. We test the approach on real-world data generated by multi-view reconstruction systems.     

无需阈值支持的机载LiDAR点云数据滤波方法

点云数据滤波仍旧是现阶段机载LiDAR数据后处理的首要步骤,但其发展尚未完全成熟。在回顾和总结已有滤波算法的基础上,将统计学中偏度与峰度的概念引入到算法中,提出了一种新的基于偏度平衡的地面点与非地面点非监督分类方法,利用统计矩原理从LiDAR点云数据生成的DSM中有效地提取DTM。该方法区别传统算法的最大的优势在于无需参数或者阈值支持,并且相对于LiDAR点云数据的格式和分辨率是独立的。实验结果证明,该方法切实可行,具有较强的适应性,并且能够较好地满足精度要求。     

Virtual first and last pulse method for building detection from dense LiDAR point clouds

ABSTRACT

LiDAR (light detection and ranging) technology is one of the most important techniques used in photogrammetry and remote sensing in order to extract high quality and high density 3D point clouds. In this paper, a novel method is proposed to detect buildings without filtering the ground points, depending on a new method named Virtual First and Last Pulse (VFLP). The type of pulses is reclassified according to the vertical direction, so that the first and last pulses for each imaginary vertical column are extracted. Using the height difference between the virtual first pulse (VFP) and virtual last pulse (VLP), the vertical features can be extracted. One of these features is the building walls, which are used as a mask for the building and, in turn, is used to detect the buildings. The results show that this method is very effective for detecting buildings and removing the trees and vegetation without filtering the ground points. Also, this method obtained a promising result for per-area and per-object level. The results show the completeness of 98.75%, correctness of 97.29% and quality of 96.1% at the per-area level and completeness of 91.95%, correctness of 98.63% and quality of 90.79% at the per- object level.     

Hybrid3D: learning 3D hybrid features with point clouds and multi-view images for point cloud registration

In recent years, point cloud registration has achieved great success by learning geometric features with deep learning techniques. However, existing approaches that rely on pure geometric context still suffer from sensor noise and geometric ambiguities(e.g., flat or symmetric structure), which limit their robustness to real-world scenes. When 3D point clouds are constructed by RGB-D cameras, we can enhance the learned features with complementary texture information from RGB images. To this end, ...     

场景视点偏移的激光雷达点云分割

目的 激光雷达采集的室外场景点云数据规模庞大且包含丰富的空间结构细节信息,但是目前多数点云分割方法并不能很好地平衡结构细节信息的提取和计算量之间的关系。一些方法将点云变换到多视图或体素化网格等稠密表示形式进行处理,虽然极大地减少了计算量,但却忽略了由激光雷达成像特点以及点云变换引起的信息丢失和遮挡问题,导致分割性能降低,尤其是在小样本数据以及行人和骑行者等小物体场景中。针对投影过程中的空间细节信息丢失问题,根据人类观察机制提出了一种场景视点偏移方法,以改善三维（3D）激光雷达点云分割结果。方法 利用球面投影将3D点云转换为2维（2D）球面正视图（spherical front view,SFV）。水平移动SFV的原始视点以生成多视点序列,解决点云变换引起的信息丢失和遮挡的问题。考虑到多视图序列中的冗余,利用卷积神经网络（convolutional neural networks,CNN）构建场景视点偏移预测模块来预测最佳场景视点偏移。结果 添加场景视点偏移模块后,在小样本数据集中,行人和骑行者分割结果改善相对明显,行人和骑行者（不同偏移距离下）的交叉比相较于原方法最高提升6.5%和15.5%。添加场景视点偏移模块和偏移预测模块后,各类别的交叉比提高1.6% 3%。在公用数据集KITTI（Karlsruhe Institute of Technology and Toyota Technological Institute）上与其他算法相比,行人和骑行者的分割结果取得了较大提升,其中行人交叉比最高提升9.1%。结论 本文提出的结合人类观察机制和激光雷达点云成像特点的场景视点偏移与偏移预测方法易于适配不同的点云分割方法,使得点云分割结果更加准确。     

Alignment of continuous video onto 3D point clouds

We propose a general framework for aligning continuous (oblique) video onto 3D sensor data. We align a point cloud computed from the video onto the point cloud directly obtained from a 3D sensor. This is in contrast to existing techniques where the 2D images are aligned to a 3D model derived from the 3D sensor data. Using point clouds enables the alignment for scenes full of objects that are difficult to model; for example, trees. To compute 3D point clouds from video, motion stereo is used along with a state-of-the-art algorithm for camera pose estimation. Our experiments with real data demonstrate the advantages of the proposed registration algorithm for texturing models in large-scale semi-urban environments. The capability to align video before a 3D model is built from the 3D sensor data offers new practical opportunities for 3D modeling. We introduce a novel modeling-through-registration approach that fuses 3D information from both the 3D sensor and the video. Initial experiments with real data illustrate the potential of the proposed approach.     

三维点云法向量估计综述

由于获取方便、表示简单、灵活等优势,点云逐渐成为常用的三维模型表示方法之一。法向量作为点云必不可少的属性之一,其估计方法在点云处理中具有重要的位置。另一方面,由于点云获取过程中不可避免的噪声、误差和遮挡,点云中通常含有噪声、外点和空洞,并且部分采样模型如CAD模型,也会存在尖锐特征,这些都给法向量估计提出了挑战。对当前已有的点云法向量估计算法进行综述,分析其原理及关键技术,着重分析它们在处理噪声、外点和尖锐特征等方面的能力并给出比较,最后为未来研究提供了一些建议。