基于3维点云鸟瞰图的高精度实时目标检测

针对基于3维点云的目标检测问题,提出了一种高精度实时的单阶段深度神经网络,分别在网络特征提取、损失函数设计和训练数据增强等3个方面提出了新的解决方案.首先对点云直接进行体素化来构建鸟瞰图.在特征提取阶段,使用残差结构提取高层语义特征,并融合多层次特征输出稠密的特征图.在回归鸟瞰图上的目标框的同时,在损失函数中考虑二次偏移量以实现更高精度的收敛.在网络训练中,使用不同帧3维点云混合的方式进行数据增强,提高网络的泛化性能.基于KITTI鸟瞰图目标检测数据集的实验结果表明,本文提出的网络仅使用雷达点云的位置信息,在性能上不仅优于目前最先进的鸟瞰图目标检测网络,而且优于融合图像和点云的检测方案,且整个网络运行速度达到20帧/秒,满足实时性要求.     

An object detection algorithm combining semantic and geometric information of the 3D point cloud

Accurately detect vehicles or pedestrians from 3D point clouds (3D object detection) is a fast developing research topic in autonomous driving and other domains. The fundamental component for feature extraction in 3D object detection is Set Abstraction (SA), which can downsample points while aggregating points to extract features. However, the current SA ignores the geometric and semantic properties of point clouds and may miss to detect remote small objects. In this paper, FocusSA is proposed, which consists two modules for enhancing useful feature extraction in the SA layer to improve 3D object detection accuracy. At first, Focused FPS (FocFPS) is proposed to evaluate the foreground and boundary scores of the points and reweighs the Furthest Point Sampling (FPS) using the evaluated scores to retain more contextual points in downsampling. Then a Geometry-aware Feature Extraction (GeoFE) module is proposed to add geometric information to enrich the awareness of geometric structure in feature aggregation. To evaluate the performances of the proposed methods, we conduct extensive experiments on three difficulty levels of Car class in KITTI dataset. The experimental results show that on “moderate” instances, our results outperform the state-of-the-art method by 1.08%. Moreover, FocusSA is easy to be plugged in popular architectures.     

基于边缘关联点云的激光雷达与相机外参标定方法

激光雷达的点云和相机的图像经常被融合应用在多个领域。准确的外参标定是融合两者信息的前提。点云特征提取是外参标定的关键步骤。但是点云的低分辨率和低质量会影响标定结果的精度。针对这些问题,提出一种基于边缘关联点云的激光雷达与相机外参标定方法。首先,利用双回波提取标定板边缘关联点云;然后,通过优化方法从边缘关联点云中提取出与实际标定板尺寸大小兼容的标定板角点;最后,将点云中角点和图像中角点匹配。用多点透视方法求解激光雷达与相机之间的外参。实验结果表明,该方法的重投影误差为1.602px,低于同类对比方法,验证了该方法的有效性与准确性。     

基于区块特征融合的点云语义分割方法

为降低室外大规模点云场景中多类三维目标语义分割的计算复杂度,提出一种融合区块特征的语义分割方法。采用方形网格分割方法对三维点云进行区块划分、采样以及组合,求取简化的点云组合区块集,将其输入至区块特征提取和融合网络中从而获得每个区块的特征修正向量。设计点云区块全局特征修正网络,以残差的方式融合特征修正向量与原始点云全局特征,修正因分割造成的错误特征。在此基础上,将方形网格分割尺寸作为神经网络的参数引入反向传播过程中进行优化,从而建立高效的点云语义分割网络。实验结果表明,反向传播算法可以优化分割尺寸至最佳值附近,所提网络中的全局特征修正方法能够提高语义分割精度,该方法在Semantic3D数据集上的语义分割精度达到78.7%,较RandLA-Net方法提升1.3%,且在保证分割精度的前提下其点云预处理计算复杂度和网络计算时间明显降低,在处理点数为10万~100万的大规模点云时,点云语义分割速度较SPG、KPConv等方法提升2~4倍。     

Generation of a DTM and building detection based on an MPF through integrating airborne lidar data and aerial images

This study presents an approach that uses airborne light detection and ranging (lidar) data and aerial imagery for creating a digital terrain model (DTM) and for extracting building objects. The process of creating the DTM from lidar data requires four steps in this study: pre-processing, segmentation, extraction of ground points, and refinement. In the pre-processing step, raw data are transformed to raster data. For segmentation, we propose a new mean planar filter (MPF) that uses a 3 × 3 kernel to divide lidar data into planar and nonplanar surfaces. For extraction of ground points, a new method to extract additional ground points in forest areas is used, thus improving the accuracy of the DTM. The refinement process further increases the accuracy of the DTM by repeated comparison of a temporary DTM and the digital surface model. After the DTM is generated, building objects are extracted via a proposed three-step process: detection of high objects, removal of forest areas, and removal of small areas. High objects are extracted using the height threshold from the normalized digital surface model. To remove forest areas from among the high objects, an aerial image and normalized digital surface model from the lidar data are used in a supervised classification. Finally, an area-based filter eliminates small areas, such as noise, thus extracting building objects. To evaluate the proposed method, we applied this and three other methods to five sites in different environments. The experiment showed that the proposed method leads to a notable increase in accuracy over three other methods when compared with the in situ reference data.     

基于注意力的毫米波-激光雷达融合目标检测

针对自动驾驶中使用激光雷达进行目标检测时漏检被遮挡目标、远距离目标和复杂天气场景下目标的问题,提出一种基于注意力机制的毫米波-激光雷达特征融合的目标检测方法。首先,将毫米波和激光雷达各自的扫描帧数据分别聚合到它们的标注帧上,并将毫米波和激光雷达的数据点进行空间对齐;其次,对两者进行聚合和空间对齐后的数据分别进行PointPillar点云柱快速编码,转换成伪图像;最后,通过中间卷积层提取两者的传感器特征,并利用注意力机制对两者的特征图进行融合,融合后的特征图通过单阶段检测器得到检测结果。实验结果显示,该融合算法在nuScenes数据集中的平均精度均值（mAP）高于PointPillar基础网络,而且注意力融合的检测方法的性能表现优于利用拼接融合、相乘融合、相加融合的检测方法。可视化结果显示所提方法是有效的,能提高网络对被遮挡目标、远处目标和雨雾天气下目标检测的鲁棒性。     

Efficient and automatic plane detection approach for 3-D rock mass point clouds

The detection of planar regions from three-dimensional (3-D) laser scanning point clouds has become more and more significant in many scientific fields, including 3-D reconstruction, augmented reality and analysis of discontinuities. In rock engineering, planes extracted from rock mass point clouds are the foundational step to build 3-D numerical models of rock mass, which is significant in analysis of rock stability. In the past, several approaches have been proposed for detecting planes from TLS point clouds. However, these methods have difficulties in processing rock points because of the uniqueness of rock. This paper introduces a novel and efficient method for plane detection from 3-D rock mass point clouds. Firstly, after filtering the raw point clouds of rock mass acquired through laser scanning, the point cloud is split into some small voxels according to the specified resolution. Then, for the purpose of acquisition of high-quality growth units, an accurate coplanarity test process is used in each voxel. Meanwhile, the accurate neighborhood information can be built according to the result of coplanarity test. Finally, small voxels are clustered into a completed plane by region growing and the procedure of postprecessing. The performance of this method was tested in one icosahedron point cloud and three rock mass point clouds. Compared with the existing methods, the results demonstrate superior performance of our method in the field of plane detection.

     

Roof plane extraction from airborne lidar point clouds

Planar patches are important primitives for polyhedral building models. One of the key challenges for successful reconstruction of three-dimensional (3D) building models from airborne lidar point clouds is achieving high quality recognition and segmentation of the roof planar points. Unfortunately, the current automatic extraction processes for planar surfaces continue to suffer from limitations such as sensitivity to the selection of seed points and the lack of computational efficiency. In order to address these drawbacks, a new fully automatic segmentation method is proposed in this article, which is capable of the following: (1) processing a roof point dataset with an arbitrary shape; (2) robustly selecting the seed points in a parameter space with reduced dimensions; and (3) segmenting the planar patches in a sub-dataset with similar attributes when region growing in the object space. The detection of seed points in the parameter space was improved by mapping the accumulator array to a 1D space. The range for region growing in the object space was reduced by an attribute similarity measure that split the roof dataset into candidate and non-candidate subsets. The experimental results confirmed that the proposed approach can extract planar patches of building roofs robustly and efficiently.     

基于邻域曲率的低特征辨识度点云配准方法

在对特征辨识度低的点云进行配准的过程中,传统的基于局部特征提取和匹配的方法通常精度不高,而基于全局特征匹配的方法精度和效率也难以保证。针对这一问题,提出一种改进的局部特征配准方法。在初步配准阶段,设计了一种基于法向量投影协方差分析的关键点提取方法,结合快速特征直方图(FPFH)对关键点进行特征描述,定义多重匹配条件对特征点进行筛选,最后将对应点的最近距离之和作为优化目标进行粗匹配;在精配准阶段,采用以点到平面的最小距离作为迭代优化对象的改进迭代最近点(ICP)算法进行精确配准。实验结果表明,在配准特征辨识度低的点云时,相较于其他三种配准方法,该方法能保持高配准精度的同时降低配准时间。     

An improved topographic mapping technique from airborne lidar: application in a forested hillside

We developed a robust method to reconstruct a digital terrain model (DTM) by classifying raw light detection and ranging (lidar) points into ground and non-ground points with the help of the Progressive Terrain Fragmentation (PTF) method. PTF applies iterative steps for searching terrain points by approximating terrain surfaces using the triangulated irregular network (TIN) model constructed from ground return points. Instead of using absolute slope or offset distance, PTF uses orthogonal distance and relative angle between a triangular plane and a node. Due to this characteristic, PTF was able to classify raw lidar points into ground and non-ground points on a heterogeneous steep forested area with a small number of parameters. We tested this approach by using a lidar data set covering a part of the Angelo Coast Range Reserve on the South Fork of the Eel River in Mendocino County, California, USA. We used systematically positioned 16 reference plots to determine the optimal parameter that can be used to separate ground and non-ground points from raw lidar point clouds. We tested at different admissible hillslope angles (15° to 20°), and the minimum total error (1.6%) was acquired at the angle value of 18°. Because classifying raw lidar points into ground and non-ground points is the basis for other types of analyses, we expect that our study will provide more accurate terrain approximation and contribute to improving the extraction of other forest biophysical parameters.     

基于 SHOT 特征描述子的自动提取球形标靶方法研究

针对复杂场景下的三维激光点云球形标靶精确自动化提取问题,提出了一种基于 SHOT 特征的 自动精确提取球形标靶的方法。该方法设计了粗提取和精提取处理过程,粗提取过程首先采用 SHOT 特征描述 子提取场景内全部的球形标靶点云;其次,利用欧氏聚类分割球形标靶点云,并采用最小二乘方法计算球形标 靶的粗略参数。精提取过程依据迭代最小二乘方法和法向滤波剔除非球面点,得到球形标靶点云和精确的球形 标靶参数。设计了含有 4 个球形标靶的实验场景,使用德国 Z+F Image 5016 扫描仪进行场景数据采集,自动 提取得到实验场景中的球形标靶点云和球形标靶参数。结果表明,在 10 m 范围内,该方法自动提取的球形标 靶半径中误差为 0.25~0.33 mm,较人工提取球形标靶点云的半径中误差减小 0.02~0.06 mm,较基于微分方法减 少 0.01~0.09 mm;该方法能够得到较高的球形标靶定位精度和稳健地去除场景点云中的噪声,可在 30 s 内完 成百万级点云球形标靶的自动提取任务。     

基于Kinect的改进移动机器人视觉SLAM

针对传统ICP（iterative closest points,迭代最近点算法）存在易陷入局部最优、匹配误差大等问题,提出了一种新的欧氏距离和角度阈值双重限制方法,并在此基础上构建了基于Kinect的室内移动机器人RGB-D SLAM（simultaneous localization and mapping）系统。首先,使用Kinect获取室内环境的彩色信息和深度信息,通过图像特征提取与匹配,结合相机内参与像素点深度值,建立三维点云对应关系;然后,利用RANSAC（random sample consensus）算法剔除外点,完成点云的初匹配;采用改进的点云配准算法完成点云的精匹配;最后,在关键帧选取中引入权重,结合g2o（general graph optimization）算法对机器人位姿进行优化。实验证明该方法的有效性与可行性,提高了三维点云地图的精度,并估计出了机器人运行轨迹。     

基于点云-模型匹配的激光雷达目标识别

为实现激光雷达自动目标识别,本文给出了一种基于点云模型匹配的方法。将点云进行三视投影,对投影点云进行二值化处理得到二值图像,采用Sobel算子和Hough变换提取点云轮廓边界及获得边界直线参数,然后以投影点云轮廓信息为约束提取包围矩形,完成目标姿态估计和几何特征提取。在此基础上以点云到CAD模型面元的欧氏距离最小为优化目标,采用单位四元数法计算点云与模型之间的刚体变换,通过迭代实现点云和候选目标CAD模型的匹配,并以归一化平均欧氏距离作为相似性度量完成目标识别。采用五种地面装甲目标在不同激光雷达视角下的点云进行目标识别实验,统计结果表明目标类别的正确识别率为100%,目标型号的正确识别率大于91%,因而本文方法具有较好的识别性能和较高的应用价值。     

A novel plane extraction approach using supervised learning

This paper presents a novel approach for the classification of planar surfaces in an unorganized point clouds. A feature-based planner surface detection method is proposed which classifies a point cloud data into planar and non-planar points by learning a classification model from an example set of planes. The algorithm performs segmentation of the scene by applying a graph partitioning approach with improved representation of association among graph nodes. The planarity estimation of the points in a scene segment is then achieved by classifying input points as planar points which satisfy planarity constraint imposed by the learned model. The resultant planes have potential application in solving simultaneous localization and mapping problem for navigation of an unmanned-air vehicle. The proposed method is validated on real and synthetic scenes. The real data consist of five datasets recorded by capturing three-dimensional(3D) point clouds when a RGBD camera is moved in five different indoor scenes. A set of synthetic 3D scenes are constructed containing planar and non-planar structures. The synthetic data are contaminated with Gaussian and random structure noise. The results of the empirical evaluation on both the real and the simulated data suggest that the method provides a generalized solution for plane detection even in the presence of the noise and non-planar objects in the scene. Furthermore, a comparative study has been performed between multiple plane extraction methods.     

基于LiDAR/INS的野外移动机器人组合导航方法

移动机器人在地形复杂等野外环境跨区域运动时,机器人运动特性和环境特征变化更为明显,由此引起的点云畸变和特征点稀疏等问题尤为突出,有必要结合传感器标定误差、车轮打滑和车体颠簸等因素进一步改进机器人的位姿估计精度。本文对基于LiDAR/INS的移动机器人环境建模和自主导航方法展开研究,针对LeGO-LOAM等在处理车体姿态快速变化时的性能退化问题,提出一种适用于野外移动机器人运动特性的点云特征分析和多传感融合方法,利用IMU的预积分与LiDAR的scan-to-map构成优化函数,进而迭代更新机器人的位姿。野外环境实验结果表明,当机器人以较高速度做转弯运动或在短时间内多次转向时,本文所提方法仍可以为优化提供良好的初值估计,相比LeGO-LOAM等方法具有更高的位姿估计精度。     

一种基于RANSAC的点云特征线提取算法

点云中提取的特征线在点云处理中具有重要的应用价值,已被应用于对称性检测、表面重建及点云与图像之间的注册等。然而,已有的点云特征线提取算法无法有效地处理点云中不可避免的噪声、外点和数据缺失,而随机采样一致性RANSAC由于具有较高的鲁棒性,在图像和三维模型处理中具有广泛的应用。为此,针对由建筑物或机械部件等具有平面特征的物体扫描得到的点云,提出了一种基于RANSAC的特征线提取算法。本算法首先基于RANSAC在点云中检测出多个平面,然后将每个平面参数化域的边界点作为候选,在这些候选点上再应用基于全局约束的RANSAC得到最终的特征线。实验结果表明,该算法对点云中的噪声、外点和数据缺失具有很强的鲁棒性。     

Multi-class US traffic signs 3D recognition and localization via image-based point cloud model using color candidate extraction and texture-based recognition

Continuous condition monitoring and inspection of traffic signs are essential to ensure that safety and performance criteria are met. The use of 3D point cloud modeling by the construction industry has been significantly increased in recent years especially for recording the as-is conditions of facilities. The high-precision and dense 3D point clouds generated by photogrammetry can facilitate the process of asset condition assessment. This paper presents an automated computer-vision based method that detects, classifies, and localizes traffic signs via street-level image-based 3D point cloud models. The proposed pipeline integrates 3D object detection algorithm. An improved Structure-from-Motion (SfM) procedure is developed to create a 3D point cloud of roadway assets from the street level imagery. In order to assist with accurate 3D recognition and localization by color and texture features extraction, an automated process of point cloud cleaning and noise removal is proposed. Using camera pose information from SfM, the points within the bounding box of detected traffic signs are then projected into the cleaned point cloud by using the triangulation method (linear and non-linear) and the 3D points corresponding to the traffic sign in question are labeled and visualized in 3D. The proposed framework is validated using real-life data, which represent the most common types of traffic signs. The robustness of the proposed pipeline is evaluated by analyzing the accuracy in detection of traffic signs as well as the accuracy in localization in 3D point cloud model. The results promise to better and more accurate visualize the location of the traffic signs with respect to other roadway assets in 3D environment.     

基于改进 L-K 光流的 WebAR 信息可视分析方法

摘 要：信息可视化技术结合移动增强现实(MAR)技术在目标跟踪领域仍然存在设备计算 负载过大的问题。若仍坚持采用同跟踪平面图像特征点的方案来跟踪立体对象各角度的特征点, 则目标跟踪过程所需要获取的多角度特征点数据无疑会加重跟踪过程的计算压力,进而导致移 动设备负载过大,最终影响模型渲染,所渲染的模型常出现剧烈抖动、卡顿或运动滞后于目标 物的现象。针对上述问题,提出了一种基于改进的 L-K (Lucas-Kanade)光流跟踪算法的 WebAR (基于 Web 端的 MAR 技术)解决方案,将特征点的跟踪问题转化为光流估计问题以及一种优化 的三维信息可视化交互策略。实验结果表明,该方法能够提高 MAR 在跟踪目标时的计算效率 和稳定性,丰富信息可视化的呈现效果和交互方式。     

基于三维激光雷达的道路边界提取和障碍物检测算法

为了快速有效地提取智能车辆在不同环境下的道路环境信息,提出基于三维激光雷达的道路边界提取和障碍物检测算法.首先,对三维激光雷达点云数据进行栅格化滤波处理,利用单束激光点云空间邻域联合分割的方法进行空间分析,得到点云平滑度特征图像.然后,采用自适应方向搜索算法获取道路边界候选点,并进行聚类分析和曲线拟合.最后,对道路边界约束下可通行区域内点云进行聚类分割,获得道路内障碍物方位和距离信息.实验表明,文中算法能够实时准确地提取道路边界和障碍物位置信息,满足智能车环境建模和路径规划的需求.     

基于自注意力机制的点云分类分割

3D点云由于其无序性以及缺少拓扑信息使得点云的分类与分割仍具有挑战性.针对上述问题,我们设计了一种基于自注意力机制的3D点云分类算法,可学习点云的特征信息,用于目标分类与分割.首先,设计适用于点云的自注意力模块,用于点云的特征提取.通过构建领域图来加强输入嵌入,使用自注意力机制进行局部特征的提取与聚合.最后,通过多层感知机以及解码器-编码器的方式将局部特征进行结合,实现3D点云的分类与分割.该方法考虑了输入嵌入时单个点在点云中的局部语境信息,构建局部长距离下的网络结构,最终得到的结果更具区分度.在ShapeNetPart、RoofN3D等数据集上的实验证实所提方法的分类与分割性能较优.