PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds

Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers. Traditional methods for point cloud denoising largely rely on local surface fitting (e.g. jets or MLS surfaces), local or non-local averaging or on statistical assumptions about the underlying noise model. In contrast, we develop a simple data-driven method for removing outliers and reducing noise in unordered point clouds. We base our approach on a deep learning architecture adapted from PCPNet, which was recently proposed for estimating local 3D shape properties in point clouds. Our method first classifies and discards outlier samples, and then estimates correction vectors that project noisy points onto the original clean surfaces. The approach is efficient and robust to varying amounts of noise and outliers, while being able to handle large densely sampled point clouds. In our extensive evaluation, both on synthetic and real data, we show an increased robustness to strong noise levels compared to various state-of-the-art methods, enabling accurate surface reconstruction from extremely noisy real data obtained by range scans. Finally, the simplicity and universality of our approach makes it very easy to integrate in any existing geometry processing pipeline. Both the code and pre-trained networks can be found on the project page ( https://github.com/mrakotosaon/pointcleannet ).     

Extracting lines of curvature from noisy point clouds

We present a robust framework for extracting lines of curvature from point clouds. First, we show a novel approach to denoising the input point cloud using robust statistical estimates of surface normal and curvature which automatically rejects outliers and corrects points by energy minimization. Then the lines of curvature are constructed on the point cloud with controllable density. Our approach is applicable to surfaces of arbitrary genus, with or without boundaries, and is statistically robust to noise and outliers while preserving sharp surface features. We show our approach to be effective over a range of synthetic and real-world input datasets with varying amounts of noise and outliers. The extraction of curvature information can benefit many applications in CAD, computer vision and graphics for point cloud shape analysis, recognition and segmentation. Here, we show the possibility of using the lines of curvature for feature-preserving mesh construction directly from noisy point clouds.     

Feature Preserving Mesh Generation from 3D Point Clouds

We address the problem of generating quality surface triangle meshes from 3D point clouds sampled on piecewise smooth surfaces. Using a feature detection process based on the covariance matrices of Voronoi cells, we first extract from the point cloud a set of sharp features. Our algorithm also runs on the input point cloud a reconstruction process, such as Poisson reconstruction, providing an implicit surface. A feature preserving variant of a Delaunay refinement process is then used to generate a mesh approximating the implicit surface and containing a faithful representation of the extracted sharp edges. Such a mesh provides an enhanced trade‐off between accuracy and mesh complexity. The whole process is robust to noise and made versatile through a small set of parameters which govern the mesh sizing, approximation error and shape of the elements. We demonstrate the effectiveness of our method on a variety of models including laser scanned datasets ranging from indoor to outdoor scenes.     

Feature‐Preserving Surface Reconstruction From Unoriented,Noisy Point Data

We propose a robust method for surface mesh reconstruction from unorganized, unoriented, noisy and outlier‐ridden 3D point data. A kernel‐based scale estimator is introduced to estimate the scale of inliers of the input data. The best tangent planes are computed for all points based on mean shift clustering and adaptive scale sample consensus, followed by detecting and removing outliers. Subsequently, we estimate the normals for the remaining points and smooth the noise using a surface fitting and projection strategy. As a result, the outliers and noise are removed and filtered, while the original sharp features are well preserved. We then adopt an existing method to reconstruct surface meshes from the processed point data. To preserve sharp features of the generated meshes that are often blurred during reconstruction, we describe a two‐step approach to effectively recover original sharp features. A number of examples are presented to demonstrate the effectiveness and robustness of our method.     

应用于三维点云数据去噪的改进C均值算法

针对三维激光扫描仪采集到的点云数据中离群点不易区分和去噪难度大的问题,提出了一种改进的C均值算法。通过分析三维点云数据特征,在传统C均值算法中引入模糊聚类权重因子,降低类内距离和拉大类间距离,有效增强了离群点特征以降低识别难度。进而将识别出的噪声分类别处理,利用改进的C均值算法去除大尺度噪声,构造双边滤波算法去除小尺度噪声数据。与密度聚类算法、正交整体最小二乘平面拟合和基于特征选择的双边滤波点云去噪等算法相比,去噪准确度分别提升了7.3%、6.5%和6.0%,实验结果表明该算法可以有效去除大尺度噪声并能较好地保留有效数据。     

Detection of Geometric Temporal Changes in Point Clouds

Detecting geometric changes between two 3D captures of the same location performed at different moments is a critical operation for all systems requiring a precise segmentation between change and no‐change regions. Such application scenarios include 3D surface reconstruction, environment monitoring, natural events management and forensic science. Unfortunately, typical 3D scanning setups cannot provide any one‐to‐one mapping between measured samples in static regions: in particular, both extrinsic and intrinsic sensor parameters may vary over time while sensor noise and outliers additionally corrupt the data. In this paper, we adopt a multi‐scale approach to robustly tackle these issues. Starting from two point clouds, we first remove outliers using a probabilistic operator. Then, we detect the actual change using the implicit surface defined by the point clouds under a Growing Least Square reconstruction that, compared to the classical proximity measure, offers a more robust change/no‐change characterization near the temporal intersection of the scans and in the areas exhibiting different sampling density and direction. The resulting classification is enhanced with a spatial reasoning step to solve critical geometric configurations that are common in man‐made environments. We validate our approach on a synthetic test case and on a collection of real data sets acquired using commodity hardware. Finally, we show how 3D reconstruction benefits from the resulting precise change/no‐change segmentation.     

一种基于RANSAC的点云特征线提取算法

点云中提取的特征线在点云处理中具有重要的应用价值,已被应用于对称性检测、表面重建及点云与图像之间的注册等。然而,已有的点云特征线提取算法无法有效地处理点云中不可避免的噪声、外点和数据缺失,而随机采样一致性RANSAC由于具有较高的鲁棒性,在图像和三维模型处理中具有广泛的应用。为此,针对由建筑物或机械部件等具有平面特征的物体扫描得到的点云,提出了一种基于RANSAC的特征线提取算法。本算法首先基于RANSAC在点云中检测出多个平面,然后将每个平面参数化域的边界点作为候选,在这些候选点上再应用基于全局约束的RANSAC得到最终的特征线。实验结果表明,该算法对点云中的噪声、外点和数据缺失具有很强的鲁棒性。     

Robust shape extraction for automatically segmenting raw LiDAR data of outdoor scenes

ABSTRACT

Light detection and ranging (LiDAR) scanning has become a prevalent technique for digitalizing outdoor scenes with three-dimensional (3D) point clouds. An automatic segmentation of the LiDAR data is important for understanding and reconstructing the outdoor scenes. However, it is still a challenging task due to complex and various objects in the outdoor scenes and some scanning drawbacks in the LiDAR data. Observing that most of the objects, such as the ground, road, roof, and facade, can be locally described with a group of geometric shapes, e.g. plane, sphere, and cylinder, we propose an automatic method to segment raw LiDAR data by robustly extracting the shapes in this paper. Firstly, our method divides the raw LiDAR data into a number of supervoxels considering the geometric and spectral consistencies of LiDAR data. Secondly, we robustly extract shapes from each supervoxels using a random sample consensus (RANSAC)-based method and evaluates and optimises the extracted shapes with a density loss estimation technique. Finally, our method outputs the segmentation result by merging the extracted shapes into a group of complete shapes, each of them represents a meaningful object in the outdoor scene. Experiments show that the method is efficient and robust for extracting most of the shapes in the outdoor scenes from a given raw LiDAR point cloud, and no preprocessing is required.     

Interactive Pixel‐Accurate Free Viewpoint Rendering from Images with Silhouette Aware Sampling

We present an integrated, fully GPU‐based processing pipeline to interactively render new views of arbitrary scenes from calibrated but otherwise unstructured input views. In a two‐step procedure, our method first generates for each input view a dense proxy of the scene using a new multi‐view stereo formulation. Each scene proxy consists of a structured cloud of feature aware particles which automatically have their image space footprints aligned to depth discontinuities of the scene geometry and hence effectively handle sharp object boundaries and occlusions. We propose a particle optimization routine combined with a special parameterization of the view space that enables an efficient proxy generation as well as robust and intuitive filter operators for noise and outlier removal. Moreover, our generic proxy generation allows us to flexibly handle scene complexities ranging from small objects up to complete outdoor scenes. The second phase of the algorithm combines these particle clouds in real‐time into a view‐dependent proxy for the desired output view and performs a pixel‐accurate accumulation of the colour contributions from each available input view. This makes it possible to reconstruct even fine‐scale view‐dependent illumination effects. We demonstrate how all these processing stages of the pipeline can be implemented entirely on the GPU with memory efficient, scalable data structures for maximum performance. This allows us to generate new output renderings of high visual quality from input images in real‐time.     

Deep Learning for Robust Normal Estimation in Unstructured Point Clouds

Normal estimation in point clouds is a crucial first step for numerous algorithms, from surface reconstruction and scene understanding to rendering. A recurrent issue when estimating normals is to make appropriate decisions close to sharp features, not to smooth edges, or when the sampling density is not uniform, to prevent bias. Rather than resorting to manually‐designed geometric priors, we propose to learn how to make these decisions, using ground‐truth data made from synthetic scenes. For this, we project a discretized Hough space representing normal directions onto a structure amenable to deep learning. The resulting normal estimation method outperforms most of the time the state of the art regarding robustness to outliers, to noise and to point density variation, in the presence of sharp edges, while remaining fast, scaling up to millions of points.     

An adaptive normal estimation method for scanned point clouds with sharp features

Normal estimation is an essential task for scanned point clouds in various CAD/CAM applications. Many existing methods are unable to reliably estimate normals for points around sharp features since the neighborhood employed for the normal estimation would enclose points belonging to different surface patches across the sharp feature. To address this challenging issue, a robust normal estimation method is developed in order to effectively establish a proper neighborhood for each point in the scanned point cloud. In particular, for a point near sharp features, an anisotropic neighborhood is formed to only enclose neighboring points located on the same surface patch as the point. Neighboring points on the other surface patches are discarded. The developed method has been demonstrated to be robust towards noise and outliers in the scanned point cloud and capable of dealing with sparse point clouds. Some parameters are involved in the developed method. An automatic procedure is devised to adaptively evaluate the values of these parameters according to the varying local geometry. Numerous case studies using both synthetic and measured point cloud data have been carried out to compare the reliability and robustness of the proposed method against various existing methods.     

Robust exponential smoothing of multivariate time series

Multivariate time series may contain outliers of different types. In the presence of such outliers, applying standard multivariate time series techniques becomes unreliable. A robust version of multivariate exponential smoothing is proposed. The method is affine equivariant, and involves the selection of a smoothing parameter matrix by minimizing a robust loss function. It is shown that the robust method results in much better forecasts than the classic approach in the presence of outliers, and performs similarly when the data contain no outliers. Moreover, the robust procedure yields an estimator of the smoothing parameter less subject to downward bias. As a byproduct, a cleaned version of the time series is obtained, as is illustrated by means of a real data example.     

Surface meshing of underwater maps from highly defective point sets

Owing to the many possible errors that may occur during real‐world mapping, point set maps often present a huge amount of outliers and large levels of noise. We present two robust surface reconstruction techniques dealing with corrupted point sets without resorting to any prefiltering step. They are based on building an unsigned distance function, discretely evaluated on an adaptive tetrahedral grid, and defined from an outlier‐robust splat representation. To extract the surface from this volumetric view, the space is partitioned into two subsets, the surface of interest being at the boundary separating them. While both methods are based on a similar graph definition derived from the above‐mentioned grid, they differ in the partitioning procedure. First, we propose a method using S‐T cuts to separate the inside and outside of the mapped area. Second, we use a normalized cut approach to partition the volume using only the values of the unsigned distance function. We prove the validity of our methods by applying them to challenging underwater data sets (sonar and image based), and we benchmark their results against the approaches in the state of the art.     

An Optimal Transport Approach to Robust Reconstruction and Simplification of 2D Shapes

We propose a robust 2D shape reconstruction and simplification algorithm which takes as input a defect‐laden point set with noise and outliers. We introduce an optimal‐transport driven approach where the input point set, considered as a sum of Dirac measures, is approximated by a simplicial complex considered as a sum of uniform measures on 0‐ and 1‐simplices. A fine‐to‐coarse scheme is devised to construct the resulting simplicial complex through greedy decimation of a Delaunay triangulation of the input point set. Our method performs well on a variety of examples ranging from line drawings to grayscale images, with or without noise, features, and boundaries.     

Inexpensive Reconstruction and Rendering of Realistic Roadside Landscapes

In this paper, we present an inexpensive approach to create highly detailed reconstructions of the landscape surrounding a road. Our method is based on a space‐efficient semi‐procedural representation of the terrain and vegetation supporting high‐quality real‐time rendering not only for aerial views but also at road level. We can integrate photographs along selected road stretches. We merge the point clouds extracted from these photographs with a low‐resolution digital terrain model through a novel algorithm which is robust against noise and missing data. We pre‐compute plausible locations for trees through an algorithm which takes into account perceptual cues. At runtime we render the reconstructed terrain along with plants generated procedurally according to pre‐computed parameters. Our rendering algorithm ensures visual consistency with aerial imagery and thus it can be integrated seamlessly with current virtual globes.     

A novel plane extraction approach using supervised learning

This paper presents a novel approach for the classification of planar surfaces in an unorganized point clouds. A feature-based planner surface detection method is proposed which classifies a point cloud data into planar and non-planar points by learning a classification model from an example set of planes. The algorithm performs segmentation of the scene by applying a graph partitioning approach with improved representation of association among graph nodes. The planarity estimation of the points in a scene segment is then achieved by classifying input points as planar points which satisfy planarity constraint imposed by the learned model. The resultant planes have potential application in solving simultaneous localization and mapping problem for navigation of an unmanned-air vehicle. The proposed method is validated on real and synthetic scenes. The real data consist of five datasets recorded by capturing three-dimensional(3D) point clouds when a RGBD camera is moved in five different indoor scenes. A set of synthetic 3D scenes are constructed containing planar and non-planar structures. The synthetic data are contaminated with Gaussian and random structure noise. The results of the empirical evaluation on both the real and the simulated data suggest that the method provides a generalized solution for plane detection even in the presence of the noise and non-planar objects in the scene. Furthermore, a comparative study has been performed between multiple plane extraction methods.     

Feature line extraction from unorganized noisy point clouds using truncated Fourier series

The detection of feature lines is important for representing and understanding geometric features of 3D models. In this paper, we introduce a new and robust method for extracting feature lines from unorganized point clouds. We use a one-dimensional truncated Fourier series for detecting feature points. Each point and its neighbors are approximated along the principal directions by using the truncated Fourier series, and the curvature of the point is computed from the approximated curves. The Fourier coefficients are computed by Fast Fourier Transform (FFT). We apply low-pass filtering to remove noise and to compute the curvature of the point robustly. For extracting feature points from the detected potential feature points, the potential feature points are thinned using a curvature weighted Laplacian-like smoothing method. The feature lines are constructed through growing extracted points and then projected onto the original point cloud. The efficiency and robustness of our approach is illustrated by several experimental results.     

考虑法向离群的自适应双边滤波点云平滑及IMLS评价方法

针对目前在点云双边滤波平滑算法中,人工输入不合理参数导致的点云平滑效果不佳,且易导致体积收缩及现有去噪后点云质量评价方法存在表达局限性等问题,提出一种自适应参数的点云双边滤波算法和基于隐性移动最小二乘(IMLS)的质量评价方法。首先构建KD-tree数据结构用于点云拓扑,之后搜索各点邻域,利用奇异值分解法计算法向量信息,并在双边滤波公式中引入法向离群因子以剔除邻域内离群点,然后通过扩展高斯核函数的权值计算式,在点云邻域内自适应获取空间与法向特征参数,最后应用改进模型进行点云平滑并引入IMLS方法评价点云质量。实验结果表明,考虑法向离群的自适应双边滤波点云平滑算法具有良好的去噪效果,相比其他算法体积收缩更小,且IMLS评价方法客观有效。     

PCPNet Learning Local Shape Properties from Raw Point Clouds

In this paper, we propose PCPNET , a deep‐learning based approach for estimating local 3D shape properties in point clouds. In contrast to the majority of prior techniques that concentrate on global or mid‐level attributes, e.g., for shape classification or semantic labeling, we suggest a patch‐based learning method, in which a series of local patches at multiple scales around each point is encoded in a structured manner. Our approach is especially well‐adapted for estimating local shape properties such as normals (both unoriented and oriented) and curvature from raw point clouds in the presence of strong noise and multi‐scale features. Our main contributions include both a novel multi‐scale variant of the recently proposed PointNet architecture with emphasis on local shape information, and a series of novel applications in which we demonstrate how learning from training data arising from well‐structured triangle meshes, and applying the trained model to noisy point clouds can produce superior results compared to specialized state‐of‐the‐art techniques. Finally, we demonstrate the utility of our approach in the context of shape reconstruction, by showing how it can be used to extract normal orientation information from point clouds.