Automatic detection of building roofs from point clouds produced by the dense image matching technique

Three-dimensional (3D) spatial information of object points is a vital requirement for many disciplines. Laser scanning technology and techniques based on image matching have been used extensively to produce 3D dense point clouds. These data are used frequently in various applications, such as the generation of digital surface model (DSM)/digital terrain model (DTM), extracting objects (e.g., buildings, trees, and roads), 3D modelling, and detecting changes. The aim of this study was to extract the building roof points automatically from the 3D point cloud data created via the image matching techniques with optical aerial images (with red, green, and blue band (RGB) and infrared (IR)). In the first stage of the study, as an alternative to laser scanning technology, which is more expensive than optical imaging systems, the 3D point clouds were produced by matching high-resolution images using a Semi Global Matching algorithm. The normalized difference vegetation index (NDVI) values for each point were calculated using the spectral information (RGB + IR) in the 3D point cloud data, and the points that represented the vegetation cover were determined using these values. In the second stage, existing ground and non-ground points that were free of vegetation cover were determined within the point cloud. Subsequently, only the points on the roof of the building were detected automatically using the proposed algorithm. Thus, points of the roofs of buildings located in areas with different topographic characteristics were detected automatically detected using only images. It was determined that the average values of correctness (Corr), completeness (Comp), and quality (Q) of the pixel-based accuracy analysis metrics were 95%, 98%, and 93%, respectively, in the selected test areas. According to the results of the accuracy analysis, it is clear that the proposed algorithm is very successful in automatic extraction of building roof points.     

Bare-earth extraction and DTM generation from photogrammetric point clouds including the use of an existing lower-resolution DTM

A method of extracting bare-earth points from photogrammetric point clouds by partially using an existing lower resolution digital terrain model (DTM) is presented. The bare-earth points are extracted based on a threshold defined by local slope. The local slope is estimated from the lower resolution DTM. A gridded DTM is then interpolated from the extracted bare-earth points. Five different interpolation algorithms are implemented and evaluated to identify the most suitable interpolation method for such non-uniformly scattered data. The algorithm is tested on four test sites with varying topographic and ground cover characteristics. The results are evaluated against a reference DTM created using aerial laser scanning. The deviations of the extracted bare-earth points, and the interpolated DTM, from the reference DTM increases with increasing forest canopy density and terrain roughness. The DTM created by the method is significantly closer to the reference DTM than the lower resolution national DTM. The ANUDEM (Australian National University Digital Elevation Modelling) interpolation method is found to be the best performing interpolation method in terms of reducing the deviations and in terms of modelling the terrain realistically with minimum artefacts, although the differences among the interpolation methods are not considerably large.     

Small-footprint lidar estimation of sub-canopy elevation and tree height in a tropical rain forest landscape

Meso-scale digital terrain models (DTMs) and canopy-height estimates, or digital canopy models (DCMs), are two lidar products that have immense potential for research in tropical rain forest (TRF) ecology and management. In this study, we used a small-footprint lidar sensor (airborne laser scanner, ALS) to estimate sub-canopy elevation and canopy height in an evergreen tropical rain forest. A fully automated, local-minima algorithm was developed to separate lidar ground returns from overlying vegetation returns. We then assessed inverse distance weighted (IDW) and ordinary kriging (OK) geostatistical techniques for the interpolation of a sub-canopy DTM. OK was determined to be a superior interpolation scheme because it smoothed fine-scale variance created by spurious understory heights in the ground-point dataset. The final DTM had a linear correlation of 1.00 and a root-mean-square error (RMSE) of 2.29 m when compared against 3859 well-distributed ground-survey points. In old-growth forests, RMS error on steep slopes was 0.67 m greater than on flat slopes. On flatter slopes, variation in vegetation complexity associated with land use caused highly significant differences in DTM error distribution across the landscape. The highest DTM accuracy observed in this study was 0.58-m RMSE, under flat, open-canopy areas with relatively smooth surfaces. Lidar ground retrieval was complicated by dense, multi-layered evergreen canopy in old-growth forests, causing DTM overestimation that increased RMS error to 1.95 m.A DCM was calculated from the original lidar surface and the interpolated DTM. Individual and plot-scale heights were estimated from DCM metrics and compared to field data measured using similar spatial supports and metrics. For old-growth forest emergent trees and isolated pasture trees greater than 20 m tall, individual tree heights were underestimated and had 3.67- and 2.33-m mean absolute error (MAE), respectively. Linear-regression models explained 51% (4.15-m RMSE) and 95% (2.41-m RMSE) of the variance, respectively. It was determined that improved elevation and field-height estimation in pastures explained why individual pasture trees could be estimated more accurately than old-growth trees. Mean height of tree stems in 32 young agroforestry plantation plots (0.38 to 18.53 m tall) was estimated with a mean absolute error of 0.90 m (r²=0.97; 1.08-m model RMSE) using the mean of lidar returns in the plot. As in other small-footprint lidar studies, plot mean height was underestimated; however, our plot-scale results have stronger linear models for tropical, leaf-on hardwood trees than has been previously reported for temperate-zone conifer and deciduous hardwoods.     

True orthoimage generation in urban areas with very tall buildings

The orthoimage usually serves as a valuable base layer in GIS. With an increasing demand in many urban GIS applications, orthoimages in urban areas are required to represent spatial objects in their true positions. However, the traditional methods for orthoimage generation did not consider features (e.g. occlusion, shadow, etc.) of spatial objects (e.g. bridges and buildings), resulting in that spatial objects in the created orthoimages cannot be located in their true positions. This paper presents our research and experimental results of true orthoimage generation in extremely tall urban areas using lidar and multi-view large-scale aerial images. Lidar data are used for the extraction of an urban digital surface model (DSM), further for the extraction of a digital building model (DBM) and a digital terrain model (DTM). Data structure and a data model for managing urban spatial objects, such as buildings and bridges, are developed. The photogrammetric geometry is used for the detection of occluded and shadowed areas in true orthoimage generation. For the occluded and shadowed areas, lost information is compensated from a conjugate area in adjacent images, for which a new mosaicking method, which automatically chooses the 'best' imagery and automatically optimizes the seam line, has been developed. Experimental results from central Denver, Colorado and Lower Manhattan, New York City demonstrated that the proposed true orthoimage generation scheme in this paper is capable of truly orthorectifying the relief displacement in aerial images and significantly reducing occlusion and shadow defects.     

Classification of lidar bare-earth points,buildings, vegetation,and small objects based on region growing and angular classifier

In recent years, light detection and ranging (lidar) systems have been intensively used in different urban applications such as map updating, communication analysis, virtual city modelling, risk assessment, and monitoring. A prerequisite to enhance lidar data content is to differentiate ground (bare earth) points that yield digital terrain models and off-terrain points in order to classify urban objects and vegetation. The increasing demand for a fast and efficient algorithm to extract three-dimensional urban features was the motive behind this work. A new combined approach to extract bare-earth points is proposed, and a novel methodology to automatically classify airborne laser data into different objects in an urban area is presented. In addition, a new concept of angular classification is introduced to differentiate buildings from vegetation and other small objects. The new angular classifier analyses the distribution of bare-earth points around unclassified point clusters to determine whether a cluster can be classified either as building or as vegetation. The experimental results confirm the high accuracy achieved to automatically classify urban objects in flat complex areas.     

A new hierarchical moving curve-fitting algorithm for filtering lidar data for automatic DTM generation

Recent advances in laser scanning hardware have allowed rapid generation of high-resolution digital terrain models (DTMs) for large areas. However, the automatic discrimination of ground and non-ground light detection and ranging (lidar) points in areas covered by densely packed buildings or dense vegetation is difficult. In this paper, we introduce a new hierarchical moving curve-fitting filter algorithm that is designed to automatically and rapidly filter lidar data to permit automatic DTM generation. This algorithm is based on fitting a second-degree polynomial surface using flexible tiles of moving blocks and an adaptive threshold. The initial tile size is determined by the size of the largest building in the study area. Based on an adaptive threshold, non-ground points and ground points are classified and labelled step by step. In addition, we used a multi-scale weighted interpolation method to estimate the bare-earth elevation for non-ground points and obtain a recovered terrain model. Our experiments in four study areas showed that the new filtering method can separate ground and non-ground points in both urban areas and those covered by dense vegetation. The filter error ranged from 4.08% to 9.40% for Type I errors, from 2.48% to 7.63% for Type II errors, and from 5.01% to 7.40% for total errors. These errors are lower than those of triangulated irregular network filter algorithms.     

An improved topographic mapping technique from airborne lidar: application in a forested hillside

We developed a robust method to reconstruct a digital terrain model (DTM) by classifying raw light detection and ranging (lidar) points into ground and non-ground points with the help of the Progressive Terrain Fragmentation (PTF) method. PTF applies iterative steps for searching terrain points by approximating terrain surfaces using the triangulated irregular network (TIN) model constructed from ground return points. Instead of using absolute slope or offset distance, PTF uses orthogonal distance and relative angle between a triangular plane and a node. Due to this characteristic, PTF was able to classify raw lidar points into ground and non-ground points on a heterogeneous steep forested area with a small number of parameters. We tested this approach by using a lidar data set covering a part of the Angelo Coast Range Reserve on the South Fork of the Eel River in Mendocino County, California, USA. We used systematically positioned 16 reference plots to determine the optimal parameter that can be used to separate ground and non-ground points from raw lidar point clouds. We tested at different admissible hillslope angles (15° to 20°), and the minimum total error (1.6%) was acquired at the angle value of 18°. Because classifying raw lidar points into ground and non-ground points is the basis for other types of analyses, we expect that our study will provide more accurate terrain approximation and contribute to improving the extraction of other forest biophysical parameters.     

Automated extraction of street-scene objects from mobile lidar point clouds

Mobile laser scanning or lidar is a new and rapid system to capture high-density three-dimensional (3-D) point clouds. Automatic data segmentation and feature extraction are the key steps for accurate identification and 3-D reconstruction of street-scene objects (e.g. buildings and trees). This article presents a novel method for automated extraction of street-scene objects from mobile lidar point clouds. The proposed method first uses planar division to sort points into different grids, then calculates the weights of points in each grid according to the spatial distribution of mobile lidar points and generates the geo-referenced feature image of the point clouds using the inverse-distance-weighted interpolation method. Finally, the proposed method transforms the extraction of street-scene objects from 3-D mobile lidar point clouds into the extraction of geometric features from two-dimensional (2-D) imagery space, thus simplifying the automated object extraction process. Experimental results show that the proposed method provides a promising solution for automatically extracting street-scene objects from mobile lidar point clouds.     

DSM generation and evaluation from QuickBird stereo imagery with 3D physical modelling

A digital terrain model (DTM) extracted from QuickBird in-track stereo images using a three-dimensional (3D) multisensor physical model developed at the Canada Centre for Remote Sensing, Natural Resources Canada was evaluated. Firstly, the stereo photogrammetric bundle adjustment was set-up with about 10 accurate ground control points and 1-2 m errors in the three axes were obtained over 48 independent checkpoints. The DTM was then generated using an area-based multi-scale image matching method and 3D semi-automatic editing tools and then compared to lidar elevation data with 0.2-m accuracy. An elevation error with 68% confidence level (LE68) of 6.4 m was achieved over the full area. Since the DTM is in fact a digital surface model where the height, or a part, of land cover (trees, houses) is included, the accuracy depends on the land cover types. Using 3D visual classification of the stereo QuickBird images, different classes (deciduous, conifer, mixed and sparse forests, residential areas, bare soils and lakes) were generated to take into account the height of the surfaces (natural and human-made) in the accuracy evaluation. LE68 values of 3.4 m to 6.7 m were thus obtained depending on the land cover types with biases representative of the surface heights. On the other hand, LE68 values of 0.5 m and 1.3 m with no bias were obtained for lakes and bare soils respectively. These last results are more representative of the real stereo QuickBird potential for DTM and 5-m contour line generation, compliant with the highest topographic standard. Since the images were acquired in wintertime and the lidar data in summertime, better results could thus be expected when using stereo images acquired in summertime, mainly in deciduous forests to integrate the full canopy height into the DSM.     

Mapping canopy height using a combination of digital stereo‐photogrammetry and lidar

Ranging techniques such as lidar (LIght Detection And Ranging) and digital stereo‐photogrammetry show great promise for mapping forest canopy height. In this study, we combine these techniques to create hybrid photo‐lidar canopy height models (CHMs). First, photogrammetric digital surface models (DSMs) created using automated stereo‐matching were registered to corresponding lidar digital terrain models (DTMs). Photo‐lidar CHMs were then produced by subtracting the lidar DTM from the photogrammetric DSM. This approach opens up the possibility of retrospective mapping of forest structure using archived aerial photographs. The main objective of the study was to evaluate the accuracy of photo‐lidar CHMs by comparing them to reference lidar CHMs. The assessment revealed that stereo‐matching parameters and left–right image dissimilarities caused by sunlight and viewing geometry have a significant influence on the quality of the photo DSMs. Our study showed that photo‐lidar CHMs are well correlated to their lidar counterparts on a pixel‐wise basis (r up to 0.89 in the best stereo‐matching conditions), but have a lower resolution and accuracy. It also demonstrated that plot metrics extracted from the lidar and photo‐lidar CHMs, such as height at the 95th percentile of 20 m×20 m windows, are highly correlated (r up to 0.95 in general matching conditions).     

Performance evaluation of classification trees for building detection from aerial images and LiDAR data: a comparison of classification trees models

This study assesses the performance of three classification trees (CT) models (entropy, gain ratio and gini) for building detection by the fusion of airborne laser scanner data and multispectral aerial images. Data from four study areas with different sensors and scene characteristics were used to assess the performance of the models. The process of performance evaluation is based on four criteria: model validation and testing, classification accuracies, relative importance of input variables, as well as transferability of CT derived from one data set to another. The LiDAR point clouds were filtered to generate a digital terrain model (DTM) based on the orthogonal polynomials, and then a digital surface model (DSM) and the normalized digital surface model (nDSM) were generated. A set of 26 uncorrelated feature attributes were derived from the original aerial images, LiDAR intensity image, DSM and nDSM. Finally, the three CT models were used to classify buildings, trees, roads and ground from aerial images, LiDAR data and the generated attributes, with the most accurate average classifications of 95% being achieved. The entropy splitting algorithm proved to be a preferable algorithm for building detection from aerial images and LiDAR data.     

Repetitive interpolation: A robust algorithm for DTM generation from Aerial Laser Scanner Data in forested terrain

We present a new algorithm for digital terrain model (DTM) generation from an airborne laser scanning point cloud, called repetitive interpolation (REIN). It is especially applicable in steep, forested areas where other filtering algorithms typically have problems distinguishing between ground returns and off-ground points reflected in the vegetation. REIN can produce a DTM either in a vector grid or in a TIN data structure. REIN is applied after an initial filtering, which involves removal of all negative outliers and removal of many, but not necessarily all, off-ground points by some existing filtering algorithm. REIN makes use of the redundancy in the initially filtered point cloud (FPC) in order to mitigate the effect of the residual off-ground points. Multiple independent random samples are taken from the initial FPC. From each sample, ground elevation estimates are interpolated at individual DTM locations. Because the lower bounds of the distributions of the elevation estimates at each DTM location are almost insensitive to positive outliers, the true ground elevations can be approximated by adding the global mean offset to the lower bounds, which is estimated from the data. The random sampling makes REIN unique among the methods of filtering airborne laser data. While other filters behave deterministically, always generating a filter error in special situations, in REIN, because of its random aspects, these errors do not occur in each sample, and typically cancel out in the final computation of DTM elevations. Reduction of processing time by parallelization of REIN is possible. REIN was tested in a test area of 2 hectares, encompassing steep relief covered by mixed forest. An Optech ALTM 1020 lidar was used, with a flying height of 260-300 m above the ground, the beam divergence was 0.3 mrad, and the obtained point cloud density for the last returns was 8.5 m^− 2. A DTM grid was generated with 1 m horizontal resolution. The root mean square elevation error of the DTM ranged between ± 0.16 m and ± 0.37 m, depending on REIN sampling rate and number of samples taken, the lowest value achieved with 4 samples and using a 23% sampling rate. The paper also gives a short overview on existing filtering algorithms.     

Improvement of forest type classification by SPOT HRV with 20 m mesh DTM

Abstract

For multispectral analysis of forest land in mountainous areas, the estimation of true reflectance without the terrain having an effect on the sensor response is indispensable. To study this subject, the authors carried out the following experiment. First, we made a precise digital terrain model (DTM) at an interval of 10 m for a test forest site covered with Lambertian-type crown surface. Analysing the forest land from the SPOT data with the precise DTM, we obtained a classification result of forest type about 20 per cent higher accuracy than the result without application of this method.     

A semi-automated method to create a lidar-based hydro-flattened DEM

The response of water surfaces to light detection and ranging (lidar) pulses is unpredictable, which results in sparse lidar point density with varying intensity values. Due to the sparseness of the point cloud and lack of natural breaklines, lidar-derived digital elevation model (DEM) can produce unnatural surface over waterbodies. Such surfaces are not cartographically pleasing and can cause issues in the hydrologic and hydraulic modelling of a river. Hydro-flattening is the process of creating a lidar-derived DEM in which water surfaces appear and behave as they would in traditional topographic DEMs generated from photogrammetric digital terrain models. Hydro-flattened DEMs, created using lidar data, exclude the lidar points over waterbodies and include three-dimensional (3D) bank shorelines. In this article, a semi-automated method is presented for extracting bank shorelines for the purpose of creating lidar-derived hydro-flatten DEMs. Lidar point cloud and an approximate stream centreline are the primary data for this process. In the first step, a continuous bare ground surface (CBGS) is created by eliminating non-ground lidar points and by adding artificial underwater points. In the second step, the lowest elevation from the lidar point cloud within a radius distance from the river centreline is used to create a virtual water surface (VWS). This VWS is revised to consider water surface undulations such as ripples or waves, protruding underwater objects, etc. The revised VWS is then intersected with the CBGS to locate the two-dimensional (2D) bank shorelines. The 2D shorelines are assigned the elevations of the VWS and are used to produce a hydro-flattened DEM. The planimetric absolute mean separation of 0.94, 0.69, and 0.63 m for the three water surfaces is observed between the bank shoreline extracted using raw lidar points and a GPS (global positioning system) survey. The mean separation using vendor classified lidar points is 0.74, 0.67, and 0.64 m which are very similar to those using raw lidar.     

Building detection in an urban area using lidar data and QuickBird imagery

This article presents a hierarchical approach to detect buildings in an urban area through the combined usage of lidar data and QuickBird imagery. A normalized digital surface model (nDSM) was first generated on the basis of the difference between a digital surface model and the corresponding digital terrain model. Then, ground objects were removed according to a height threshold. In consideration of the relief displacement effect in very high resolution remote-sensing imagery, we segmented the nDSM by the region-growing method and used the overlap ratio to avoid over-removing building objects. Finally, the region size and spatial relation of trees and buildings were used to filter out trees occluded by buildings based on an object-based classification. Compared with previous methods directly using the normalized difference vegetation index (NDVI), our method improved the completeness from 85.94% to 90.20%. The overall accuracy of the buildings detected using the proposed method can be up to 94.31%, indicating the practical applicability of the method.     

Complete classification of raw LIDAR data and 3D reconstruction of buildings

LIDAR (LIght Detection And Ranging) data are a primary data source for digital terrain model (DTM) generation and 3D city models. This paper presents a three-stage framework for a robust automatic classification of raw LIDAR data as buildings, ground and vegetation, followed by a reconstruction of 3D models of the buildings. In the first stage the raw data are filtered and interpolated over a grid. In the second stage, first a double raw data segmentation is performed and then geometric and topological relationships among regions resulting from segmentation are computed and stored in a knowledge base. In the third stage, a rule-based scheme is applied for the classification of the regions. Finally, polyhedral building models are reconstructed by analysing the topology of building outlines, building roof slopes and eaves lines. Results obtained on data sets with different ground point density, gathered over the town of Pavia (Italy) with Toposys and Optech airborne laser scanning systems, are shown to illustrate the effectiveness of the proposed approach.     

Integration of orthoimagery and lidar data for object-based urban thematic mapping using random forests

Using high-spatial-resolution multispectral imagery alone is insufficient for achieving highly accurate and reliable thematic mapping of urban areas. Integration of lidar-derived elevation information into image classification can considerably improve classification results. Additionally, traditional pixel-based classifiers have some limitations in regard to certain landscape and data types. In this study, we take advantage of current advances in object-based image analysis and machine learning algorithms to reduce manual image interpretation and automate feature selection in a classification process. A sequence of image segmentation, feature selection, and object classification is developed and tested by the data sets in two study areas (Mannheim, Germany and Niagara Falls, Canada). First, to improve the quality of segmentation, a range image of lidar data is incorporated in an image segmentation process. Among features derived from lidar data and aerial imagery, the random forest, a robust ensemble classifier, is then used to identify the best features using iterative feature elimination. On the condition that the number of samples is at least two or three times the number of features, a segmentation scale factor has no particular effect on the selected features or classification accuracies. The results of the two study areas demonstrate that the presented object-based classification method, compared with the pixel-based classification, improves by 0.02 and 0.05 in kappa statistics, and by 3.9% and 4.5% in overall accuracy, respectively.     

Forest fuel treatment detection using multi-temporal airborne lidar data and high-resolution aerial imagery: a case study in the Sierra Nevada Mountains,California

Treatments to reduce forest fuels are often performed in forests to enhance forest health, regulate stand density, and reduce the risk of wildfires. Although commonly employed, there are concerns that these forest fuel treatments (FTs) may have negative impacts on certain wildlife species. Often FTs are planned across large landscapes, but the actual treatment extents can differ from the planned extents due to operational constraints and protection of resources (e.g. perennial streams, cultural resources, wildlife habitats). Identifying the actual extent of the treated areas is of primary importance to understand the environmental influence of FTs. Light detection and ranging (lidar) is a powerful remote-sensing tool that can provide accurate measurements of forest structures and has great potential for monitoring forest changes. This study used the canopy height model (CHM) and canopy cover (CC) products derived from multi-temporal airborne laser scanning (ALS) data to monitor forest changes following the implementation of landscape-scale FT projects. Our approach involved the combination of a pixel-wise thresholding method and an object-of-interest (OBI) segmentation method. We also investigated forest change using normalized difference vegetation index (NDVI) and standardized principal component analysis from multi-temporal high-resolution aerial imagery. The same FT detection routine was then applied to compare the capability of ALS data and aerial imagery for FT detection. Our results demonstrate that the FT detection using ALS-derived CC products produced both the highest total accuracy (93.5%) and kappa coefficient (κ) (0.70), and was more robust in identifying areas with light FTs. The accuracy using ALS-derived CHM products (the total accuracy was 91.6%, and the κ was 0.59) was significantly lower than that using ALS-derived CC, but was still higher than using aerial imagery. Moreover, we also developed and tested a method to recognize the intensity of FTs directly from pre- and post-treatment ALS point clouds.     

Urban building roof segmentation from airborne lidar point clouds

This article presents a new approach to segmenting building rooftops from airborne lidar point clouds. A progressive morphological filter technique is first applied for separation between ground and non-ground points. For the non-ground points, a region-growing algorithm based on a plane-fitting technique is used to separate building points from vegetation points. Then, an adaptive Random Sample Consensus (RANSAC) algorithm based on a grid structure is developed to improve the probability of selecting an uncontained sample from the localized sampling. The distance, standard deviation and normal vector are integrated to keep topological consistency among building rooftop patches during building rooftop segmentation. Finally, the remaining points are mapped on to the extracted planes by a post-processing technique to improve the segmentation accuracy. The results for buildings with different roof complexities are presented and evaluated.