Automatic generation of high-quality building models from lidar data

Automating data acquisition for 3D city models is an important research topic in photogrammetry. In addition to techniques that rely on aerial images, generating 3D building models from point clouds provided. by light detection and ranging (Lidar) sensors is gaining importance. The progress in sensor technology has triggered this development. Airborne laser scanners can deliver dense point clouds with densities of up to one point per square meter. Using this information, it's possible to detect buildings and their approximate outlines and also to extract planar roof faces and create models that correctly resemble the roof structures. The author presents a method for automatically generating 3D building models from point clouds generated by the Lidar sensing technology.     

一种自动、快速的Kinect标定方法

针对目前Kinect传感器人工标定方法误差大、速度慢等问题,提出一种自动、快速的Kinect传感器外参标定方法。首先,根据彩色图像提取的角点,生成彩色图像的角点集合;其次,为了实现角点点云的自动提取,对点云图像进行深度分割,提取棋盘格点云,采用三维哈夫(Hough)变换检测方法将棋盘格点云投影到深度图像的模板平面上,在深度图像模板中提取深度图像中的角点;然后,将深度图像中的角点映射到棋盘格点云中,形成角点点云;最后,将角点点云与彩色图像的角点集合进行配准,得到角点的3D空间坐标,进而计算出深度相机到彩色相机的姿态变换矩阵。实验结果表明,本文提出的算法在保证相机标定精度的前提下,将相机参数的计算时间从平均218ms降低到166ms,实现了自动、快速的Kinect相机标定。     

结合体元数据结构的机载LIDAR建筑物检测

目的 目前,点云、栅格格网及不规则三角网等建筑物检测中常用的离散机载激光雷达（LIDAR）点云数据表达方式存在模型表达复杂、算法开发困难、结果表达不准确及难以表达多返回数据等缺点。为此,针对LIDAR点云体元结构模型构建及在此基础上的建筑物检测展开研究,提出一种基于体元的建筑物检测算法。方法 首先将点云数据规则化为二值（即1、0值,分别表示体元中是否包含有激光点）3D体元结构。然后利用3D滤波算法将上述体元结构中表征数据点的体元分类为地面和非地面体元。最后,依据建筑物边缘的接近直线、跳变特性从非地面体元中搜寻建筑物边缘作为种子体元进而标记与其3D连通的非地面体元集合为建筑物体元。结果 实验基于ISPRS（international society for photogrammetry and remote sensing）提供的包含了不同的建筑物类型的城区LIDAR点云数据测试了"邻域尺度"参数的敏感性及提出算法的精度。定量评价的结果表明：56邻域为最佳邻域尺度;建筑物的检测质量可达到95%以上——平均完整度可达到95.61%、平均正确率可达95.97%。定性评价的结果表明：对大型、密集、不规则形状、高低混合及其他屋顶类型比较特殊的复杂建筑物均可成功检测。结论 本文提出的建筑物检测算法采用基于体元空间邻域关系的搜索标记方式,可有效实现对各类建筑目标特别是城市建筑目标的检测,检测结果易于建模3D建筑物模型。     

Complete residential urban area reconstruction from dense aerial LiDAR point clouds

We present an automatic system to reconstruct 3D urban models for residential areas from aerial LiDAR scans. The key difference between downtown area modeling and residential area modeling is that the latter usually contains rich vegetation. Thus, we propose a robust classification algorithm that effectively classifies LiDAR points into trees, buildings, and ground. The classification algorithm adopts an energy minimization scheme based on the 2.5D characteristic of building structures: buildings are composed of opaque skyward roof surfaces and vertical walls, making the interior of building structures invisible to laser scans; in contrast, trees do not possess such characteristic and thus point samples can exist underneath tree crowns. Once the point cloud is successfully classified, our system reconstructs buildings and trees respectively, resulting in a hybrid model representing the 3D urban reality of residential areas.     

An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery

Modeling the energy performance of existing buildings enables quick identification and reporting of potential areas for building retrofit. However, current modeling practices of using energy simulation tools do not model the energy performance of buildings at their element level. As a result, potential retrofit candidates caused by construction defects and degradations are not represented. Furthermore, due to manual modeling and calibration processes, their application is often time-consuming. Current application of 2D thermography for building diagnostics is also facing several challenges due to a large number of unordered and non-geo-tagged images. To address these limitations, this paper presents a new computer vision-based method for automated 3D energy performance modeling of existing buildings using thermal and digital imagery captured by a single thermal camera. First, using a new image-based 3D reconstruction pipeline which consists of Graphic Processing Unit (GPU)-based Structure-from-Motion (SfM) and Multi-View Stereo (MVS) algorithms, the geometrical conditions of an existing building is reconstructed in 3D. Next, a 3D thermal point cloud model of the building is generated by using a new 3D thermal modeling algorithm. This algorithm involves a one-time thermal camera calibration, deriving the relative transformation by forming the Epipolar geometry between thermal and digital images, and the MVS algorithm for dense reconstruction. By automatically superimposing the 3D building and thermal point cloud models, 3D spatio-thermal models are formed, which enable the users to visualize, query, and analyze temperatures at the level of 3D points. The underlying algorithms for generating and visualizing the 3D spatio-thermal models and the 3D-registered digital and thermal images are presented in detail. The proposed method is validated for several interior and exterior locations of a typical residential building and an instructional facility. The experimental results show that inexpensive digital and thermal imagery can be converted into ubiquitous reporters of the actual energy performance of existing buildings. The proposed method expedites the modeling process and has the potential to be used as a rapid and robust building diagnostic tool.     

Textured mesh surface reconstruction of large buildings with multi-view stereo

There are three main approaches for reconstructing 3D models of buildings. Laser scanning is accurate but expensive and limited by the laser’s range. Structure-from-motion (SfM) and multi-view stereo (MVS) recover 3D point clouds from multiple views of a building. MVS methods, especially patch-based MVS, can achieve higher density than do SfM methods. Sophisticated algorithms need to be applied to the point clouds to construct mesh surfaces. The recovered point clouds can be sparse in areas that lack features for accurate reconstruction, making recovery of complete surfaces difficult. Moreover, segmentation of the building’s surfaces from surrounding surfaces almost always requires some form of manual inputs, diminishing the ease of practical application of automatic 3D reconstruction algorithms. This paper presents an alternative approach for reconstructing textured mesh surfaces from point cloud recovered by patch-based MVS method. To a good first approximation, a building’s surfaces can be modeled by planes or curve surfaces which are fitted to the point cloud. 3D points are resampled on the fitted surfaces in an orderly pattern, whose colors are obtained from the input images. This approach is simple, inexpensive, and effective for reconstructing textured mesh surfaces of large buildings. Test results show that the reconstructed 3D models are sufficiently accurate and realistic for 3D visualization in various applications.     

2D/3D information fusion for building extraction from high-resolution satellite stereo images using kernel graph cuts

This study presents a building extraction strategy from High-resolution satellite stereo images (HRSSI) using 2D and 3D information fusion. In the 2D processing strategy, a visible vegetation index (VVI) is generated. In the 3D processing, a disparity map is generated using semi-global matching (SGM). To remove defects from the disparity map, an object-based approach is proposed by using mean-shift image segmentation and extracting rectangles. By removing terrain effects, a normalized disparity map (nDM) is produced. In the next step, vegetation pixels are removed from nDM and an initial building mask is generated. As nDM does not have precise building boundaries, hybrid segmentation by the kernel graph cut (KGC) is applied to the feature space including the RGB, nDM, and VVI and the results are used in a decision level fusion step. By this methodology, segments that are highly intersected with initial building mask are classified as buildings. Finally, a building boundary refinement (BBR) algorithm is applied to buildings for removing the remaining defects. The proposed method is applied to two pairs of GeoEye-1 stereo images including residential and industrial test areas. Evaluation results show the completeness and correctness level of higher than 90% for the two test areas. Further evaluations show that the quality metric has significantly changed after decision level fusion using the KGC.     

多特征融合与几何卷积的机载LiDAR点云地物分类

目的点云分类传统方法中大量依赖人工设计特征,缺乏深层次特征,难以进一步提高精度,基于深度学习的方法大部分利用结构化网络,转化为其他表征造成了3维空间结构信息的丢失,部分利用局部结构学习多层次特征的方法也因为忽略了机载数据的几何信息,难以实现精细分类。针对上述问题,本文提出了一种基于多特征融合几何卷积神经网络(multi-feature fusion and geometric convolutional neural network,MFFGCNN)的机载Li DAR(light detection and ranging)点云地物分类方法。方法提取并融合有效的浅层传统特征,并结合坐标尺度等预处理方法,称为APD模块(airporne laser scanning point cloud design module),在输入特征层面对典型地物有针对性地进行信息补充,来提高网络对大区域、低密度的机载Li DAR点云原始数据的适应能力和基础分类精度,基于多特征融合的几何卷积模块,称为FGC(multi-feature fusion and geometric convolution)算子,...     

基于透视式增强现实的BIM建筑三维重建仿真

为了提高三维建筑模型的精准度,需要深入研究BIM建筑三维重建方法。当前方法耗时较长,得到的三维建筑模型与实际建筑之间的误差较大,存在效率低和精准度低的问题。将透视式增强现实技术应用到BIM建筑三维重建中,提出基于透视式增强现实的BIM建筑三维重建方法,通过BIM构建初始三维建筑模型,采用直接线性变换算法计算摄像机的内部参数和外部参数,完成摄像机标定。在摄像机标定结果的基础上采用LK光流计算方法得到像素在图像中的光流,根据光流的方向阈值和光流的大小筛选图像中的光流,提取到图像的匹配点,基于初始三维建筑模型针对建筑图像匹配点构成空间三维点云,采用Delaunay方法对空间三维点云进行三角化处理,针对处理后的建筑图像通过贴纹理完成BIM建筑三维重建。仿真结果表明,所提方法的效率高、精准度高。     

基于特征点和改进ICP的三维点云数据配准算法

在光学非接触三维测量中,复杂对象的重构需要多组测量数据的配准。最近点迭代(ICP)算法是三维激光扫描数据处理中点云数据配准的一种经典的数学方法,为了获得更好的配准结果,在ICP算法的基础之上,提出了结合基于特征点的等曲率预配准方法和邻近搜索ICP改进算法的精细配准,自动进行点云数据配准的算法,经对牙齿点云模型实验发现,点云数据量越大,算法的配准速度优势越明显,采用ICP算法的运行时间(194.58 s)远大于本算法的运行时间(89.13 s)。应用实例表明:该算法具有速度快、精度高的特点,算法效果良好。     

融合稀疏点云补全的3D目标检测算法

基于雷达点云的3D目标检测方法有效地解决了RGB图像的2D目标检测易受光照、天气等因素影响的问题.但由于雷达的分辨率以及扫描距离等问题,激光雷达采集到的点云往往是稀疏的,这将会影响3D目标检测精度.针对这个问题,提出一种融合稀疏点云补全的目标检测算法,采用编码、解码机制构建点云补全网络,由输入的部分稀疏点云生成完整的密...     

Urban building roof segmentation from airborne lidar point clouds

This article presents a new approach to segmenting building rooftops from airborne lidar point clouds. A progressive morphological filter technique is first applied for separation between ground and non-ground points. For the non-ground points, a region-growing algorithm based on a plane-fitting technique is used to separate building points from vegetation points. Then, an adaptive Random Sample Consensus (RANSAC) algorithm based on a grid structure is developed to improve the probability of selecting an uncontained sample from the localized sampling. The distance, standard deviation and normal vector are integrated to keep topological consistency among building rooftop patches during building rooftop segmentation. Finally, the remaining points are mapped on to the extracted planes by a post-processing technique to improve the segmentation accuracy. The results for buildings with different roof complexities are presented and evaluated.     

3D shape acquisition and integral compact representation using optical scanning and enhanced shape parameterization

An efficient computational methodology for shape acquisition, processing and representation is developed. It includes 3D computer vision by applying triangulation and stereo-photogrammetry for high-accuracy 3D shape acquisition. Resulting huge 3D point clouds are successively parameterized into mathematical surfaces to provide for compact data-set representation, yet capturing local details sufficiently. B-spline surfaces are employed as parametric entities in fitting to point clouds resulting from optical 3D scanning. Beyond the linear best-fitting algorithm with control points as fitting variables, an enhanced non-linear procedure is developed. The set of best fitting variables in minimizing the approximation error norm between the parametric surface and the 3D cloud includes the control points coordinates. However, they are augmented by the set of position parameter values which identify the respectively closest matching points on the surface for the points in the cloud. The developed algorithm is demonstrated to be efficient on demanding test cases which encompass sharp edges and slope discontinuities originating from physical damage of the 3D objects or shape complexity.     

Template Assembly for Detailed Urban Reconstruction

We propose a new framework to reconstruct building details by automatically assembling 3D templates on coarse textured building models. In a preprocessing step, we generate an initial coarse model to approximate a point cloud computed using Structure from Motion and Multi View Stereo, and we model a set of 3D templates of facade details. Next, we optimize the initial coarse model to enforce consistency between geometry and appearance (texture images). Then, building details are reconstructed by assembling templates on the textured faces of the coarse model. The 3D templates are automatically chosen and located by our optimization‐based template assembly algorithm that balances image matching and structural regularity. In the results, we demonstrate how our framework can enrich the details of coarse models using various data sets.     

Adaptive random sample consensus approach for segmentation of building roof in airborne laser scanning point cloud

ABSTRACT

This work proposes a three-step method for segmenting the roof planes of buildings in Airborne Laser Scanning (ALS) data. The first step aims at mainly avoiding the exhaustive search for planar roof faces throughout the ALS point cloud. Standard algorithms for processing ALS point cloud are used to isolate building regions. The second step of the proposed method consists in segmenting roof planes within building regions previously delimited. We use the RANdom SAmple Consensus (RANSAC) algorithm to detect roof plane points, taking into account two adaptive parameters for checking the consistency of ALS building points with the candidate planes: the distance between ALS building points and candidate planes; and the angle between the gradient vectors at ALS building points and the candidate planes’ normal vector. Each ALS building point is classified as consistent if computed parameters are below corresponding thresholds, which are automatically determined by thresholding histograms constructed for both parameters. As the RANSAC algorithm can generate fragmented results, in the third step, a post-processing is accomplished to merge planes that are approximately collinear and spatially close. The results show that the proposed method works properly. However, failures occur mainly in regions affected by local anomalies such as trees and antennas. Average rates around 90% and higher than 95% have been obtained for the completeness and correction quality parameters, respectively.     

Slice-based building facade reconstruction from 3D point clouds

ABSTRACT

We propose a comprehensive strategy to reconstruct urban building geometry from three-dimensional (3D) point clouds. First, the point clouds are segmented using a rough-detail segmentation algorithm, and refinements guided by topological relationships are performed to rectify the segmentation mistakes. Then, the semantic features (such as facades and windows) that belong to the buildings are recognized and extracted. Next, each facade is cut into a sequence of slices. The initial models are recovered by sequentially detecting and connecting the anchor points. Finally, due to the regular arrangements of windows, a template-matching method relying on the similarity and repetitiveness of the windows is proposed to recover the details on building facades. The experimental results demonstrate that our method can automatically reconstruct the building geometry and detailed window structures are better depicted.     

R-VPCG: RGB image feature fusion-based virtual point cloud generation for 3D car detection

Although 3D object detection methods based on feature fusion have made great progress, the methods still have the problem of low precision due to sparse point clouds. In this paper, we propose a new feature fusion-based method, which can generate virtual point cloud and improve the precision of car detection. Considering that RGB images have rich semantic information, this method firstly segments the cars from the image, and then projected the raw point clouds onto the segmented car image to segment point clouds of the cars. Furthermore, the segmented point clouds are input to the virtual point cloud generation module. The module regresses the direction of car, then combines the foreground points to generate virtual point clouds and superimposed with the raw point cloud. Eventually, the processed point cloud is converted to voxel representation, which is then fed into 3D sparse convolutional network to extract features, and finally a region proposal network is used to detect cars in a bird’s-eye view. Experimental results on KITTI dataset show that our method is effective, and the precision have significant advantages compared to other similar feature fusion-based methods.     

An enhanced 3D model and generative adversarial network for automated generation of horizontal building mask images and cloudless aerial photographs

Information extracted from aerial photographs is widely used in the fields of urban planning and design. An effective method for detecting buildings in aerial photographs is to use deep learning to understand the current state of a target region. However, the building mask images used to train the deep learning model must be manually generated in many cases. To overcome this challenge, a method has been proposed for automatically generating mask images by using textured three-dimensional (3D) virtual models with aerial photographs. Some aerial photographs include clouds, which degrade image quality. These clouds can be removed by using a generative adversarial network (GAN), which leads to improvements in training quality. Therefore, the objective of this research was to propose a method for automatically generating building mask images by using 3D virtual models with textured aerial photographs. In this study, using GAN to remove clouds in aerial photographs improved training quality. A model trained on datasets generated by the proposed method was able to detect buildings in aerial photographs with IoU = 0.651.     

从机载激光扫描数据中交互式提取人字形房屋模型的方法研究

快速有效地从机载激光扫描(airborne lidar)点云数据中提取房屋模型是机载激光扫描系统应用研究的一项重要课题。鉴于交互式半自动方法是从点云数据中提取简单规则房屋模型信息的一种可行的方法,为此采用3维空间中改进的Hough变换以及聚类分析,提出了一种从点云数据中交互式提取人字形房屋模型的方法。该方法分为3个步骤:第1步是用户确定房屋区域,并分割出候选的屋顶点集;第2步是对候选屋顶点集采用3维空间中改进的Hough变换,然后对Hough变换后所获得的参数集进行聚类分析,以此获得屋顶所在平面的参数表达;第3步是构造完整的房屋模型。通过屋顶平面相交得到屋脊线,通过点的范围分析确定屋顶的边缘,最后添加竖直的墙面构造完整的房屋模型。经采用Optech公司提供的数据进行实验初步证实,该方法是可行的,且整个提取过程只需要很少的用户交互,因此适合于大规模处理机载激光扫描数据。     

基于线特征及迭代最近点算法的地基建筑物点云自动配准方法

为克服迭代最近点(ICP)算法易陷入局部最优的缺陷，提出一种基于线特征及ICP算法的地基建筑物点云自动配准方法。首先，基于法向一致性进行建筑物点云平面分割；接着，采用alpha-shape算法进行点簇轮廓线提取，并拆分和拟合处理得到特征线段；然后，以线对作为配准基元，以线对夹角和距离作为相似性测度进行同名特征匹配，实现建筑物点云的粗配准；最后，以粗配准结果为初值，进一步采用ICP算法完成点云精确配准。利用两组部分重叠的建筑物点云进行配准实验，实验结果表明，采用由粗到精的配准方法能有效改善ICP算法对初值依赖的问题，实现具有部分重叠的建筑物点云的有效配准。