平行点云: 虚实互动的点云生成与三维模型进化方法

三维信息的提取在自动驾驶等智能交通场景中正发挥着越来越重要的作用, 为了解决以激光雷达为主的深度传感器在数据采集方面面临的成本高、样本覆盖不全面等问题, 本文提出了平行点云的框架. 利用人工定义场景获取虚拟点云数据, 通过计算实验训练三维模型, 借助平行执行对模型性能进行测试, 并将结果反馈至数据生成和模型训练过程. 通过不断地迭代, 使三维模型得到充分评估并不断进化. 在平行点云的框架下, 我们以三维目标检测为例, 通过闭环迭代, 构建了虚实结合的点云数据集, 在无需人工标注的情况下, 可达到标注数据训练模型精度的72%.     

Neural networks for the recognition and pose estimation of 3D objects from a single 2D perspective view

In this paper we present a neural network (NN) based system for recognition and pose estimation of 3D objects from a single 2D perspective view. We develop an appearance based neural approach for this task. First the object is represented in a feature vector derived by a principal component network. Then a NN classifier trained with Resilient backpropagation (Rprop) algorithm is applied to identify it. Next pose parameters are obtained by four NN estimators trained on the same feature vector. Performance on recognition and pose estimation for real images under occlusions are shown. Comparative studies with two other approaches are carried out.     

基于邻域曲率的低特征辨识度点云配准方法

在对特征辨识度低的点云进行配准的过程中,传统的基于局部特征提取和匹配的方法通常精度不高,而基于全局特征匹配的方法精度和效率也难以保证。针对这一问题,提出一种改进的局部特征配准方法。在初步配准阶段,设计了一种基于法向量投影协方差分析的关键点提取方法,结合快速特征直方图(FPFH)对关键点进行特征描述,定义多重匹配条件对特征点进行筛选,最后将对应点的最近距离之和作为优化目标进行粗匹配;在精配准阶段,采用以点到平面的最小距离作为迭代优化对象的改进迭代最近点(ICP)算法进行精确配准。实验结果表明,在配准特征辨识度低的点云时,相较于其他三种配准方法,该方法能保持高配准精度的同时降低配准时间。     

自适应邻域选择的FPFH特征提取算法

在使用点云FPFH(Fast Point Feature Histograms)特征进行三维物体识别或配准时,人为主观调整邻域半径计算FPFH特征描述符具有随意性、低效性,整个过程不能自动化完成。针对该问题,提出了自适应邻域选择的FPFH特征提取算法。首先,对多对点云估算点云密度;然后,计算多个邻域半径以提取FPFH特征用于SAC-IA配准,统计配准性能最优时的半径与点云密度值,使用三次样条插值拟合法求出函数表达式,形成自适应邻域选择的FPFH特征提取算法。实验结果表明,该算法根据点云密度自适应选择合适的邻域半径,提升了FPFH特征匹配的性能,同时 加快了运算速度,具有指导价值。     

基于3维点云鸟瞰图的高精度实时目标检测

针对基于3维点云的目标检测问题,提出了一种高精度实时的单阶段深度神经网络,分别在网络特征提取、损失函数设计和训练数据增强等3个方面提出了新的解决方案.首先对点云直接进行体素化来构建鸟瞰图.在特征提取阶段,使用残差结构提取高层语义特征,并融合多层次特征输出稠密的特征图.在回归鸟瞰图上的目标框的同时,在损失函数中考虑二次偏移量以实现更高精度的收敛.在网络训练中,使用不同帧3维点云混合的方式进行数据增强,提高网络的泛化性能.基于KITTI鸟瞰图目标检测数据集的实验结果表明,本文提出的网络仅使用雷达点云的位置信息,在性能上不仅优于目前最先进的鸟瞰图目标检测网络,而且优于融合图像和点云的检测方案,且整个网络运行速度达到20帧/秒,满足实时性要求.     

An object detection algorithm combining semantic and geometric information of the 3D point cloud

Accurately detect vehicles or pedestrians from 3D point clouds (3D object detection) is a fast developing research topic in autonomous driving and other domains. The fundamental component for feature extraction in 3D object detection is Set Abstraction (SA), which can downsample points while aggregating points to extract features. However, the current SA ignores the geometric and semantic properties of point clouds and may miss to detect remote small objects. In this paper, FocusSA is proposed, which consists two modules for enhancing useful feature extraction in the SA layer to improve 3D object detection accuracy. At first, Focused FPS (FocFPS) is proposed to evaluate the foreground and boundary scores of the points and reweighs the Furthest Point Sampling (FPS) using the evaluated scores to retain more contextual points in downsampling. Then a Geometry-aware Feature Extraction (GeoFE) module is proposed to add geometric information to enrich the awareness of geometric structure in feature aggregation. To evaluate the performances of the proposed methods, we conduct extensive experiments on three difficulty levels of Car class in KITTI dataset. The experimental results show that on “moderate” instances, our results outperform the state-of-the-art method by 1.08%. Moreover, FocusSA is easy to be plugged in popular architectures.     

Three-dimensional object representation and invariant recognitionusing continuous distance transform neural networks

3D object recognition under partial object viewing is a difficult pattern recognition task. In this paper, we introduce a neural-network solution that is robust to partial viewing of objects and noise corruption. This method directly utilizes the acquired 3D data and requires no feature extraction. The object is first parametrically represented by a continuous distance transform neural network (CDTNN) trained by the surface points of the exemplar object. The CDTNN maps any 3D coordinate into a value that corresponds to the distance between the point and the nearest surface point of the object. Therefore, a mismatch between the exemplar object and an unknown object can be easily computed. When encountered with deformed objects, this mismatch information can be backpropagated through the CDTNN to iteratively determine the deformation in terms of affine transform. Application to 3D heart contour delineation and invariant recognition of 3D rigid-body objects is presented.     

基于局部曲面特征直方图的点云识别

针对三维点云的快速识别问题,文中提出基于局部曲面特征直方图的点云识别算法.首先,采用循环体素滤波算法,将不同分辨率的点云滤波至指定分辨率.再基于邻域曲率均值最大的关键点查找算法选取点云局部特征较明显的点作为关键点,根据关键点邻域内点云重心与邻域曲面内各点的法线和距离的关系计算关键点的特征描述符.然后,根据临近关键点间的空间关系和特征描述符欧氏距离进行特征匹配.最后,采用多线程识别框架,加快在线识别速度.实验表明文中算法识别速度较快.     

面向点云的三维物体识别方法综述

随着三维扫描技术的快速发展,获取各类场景的点云数据已经非常简单快捷;加之点云数据具备不受光照、阴影、纹理的影响等优势,基于点云的三维物体识别已成为计算机视觉领域的研究热点。首先,对近年来面向点云数据的三维物体识别方法进行归纳和总结;然后,对已有方法的优势及缺点进行分析;最后,指出点云物体识别中所面临的挑战及进一步的研究方向。     

基于原始点云的三维目标检测算法

针对当前三维目标检测中存在的数据降采样难、特征提取不充分、感受野有限、候选包围盒回归质量不高等问题,基于3DSSD三维目标检测算法,提出了一种基于原始点云、单阶段、无锚框的三维目标检测算法RPV-SSD(random point voxel single stage object detector),该算法由随机体素采样层、3D稀疏卷积层、特征聚合层、候选点生成层、区域建议网络层共五个部分组成,主要通过聚合随机体素采样的关键点逐点特征、体素稀疏卷积特征、鸟瞰图特征,进而实现对物体类别、3D包围盒以及物体朝向的预测。在KITTI数据集上的实验表明,该算法整体表现良好,不仅能够命中真值标签中的目标并且回归较好的包围盒,还能够从物体的不完整点云推测出物体的类别及其完整形状,提高目标检测性能。     

快速不变矩算法基于CUDA的并行实现研究

不变矩自提出以来被广泛应用于目标识别系统中进行特征描述,这需要能够实时计算不变矩值。虽然人们提出了许多不变矩的快速算法,仍无法在单台PC机上实现不变矩的实时计算。本文分析了基于差分矩因子的不变矩快速算法的并行性,提出了一种基于CUDA（Compute Unified Device Architecture）的快速不变矩并行实现方法,并在NVIDIA Tesla C1060 GPU（Graphic Processing Unit）上实现。对所提出算法的计算性能与普通串行算法进行了对比分析。实验结果表明,本文所提出的并行计算方法极大地提高了不变矩的计算速度,可有效地用来进行实时特征提取。     

Automatic model-based 3D object recognition by combining feature matching with tracking

We propose a vision-based robust automatic 3D object recognition, which provides object identification and 3D pose information by combining feature matching with tracking. For object identification, we propose a robust visual feature and a probabilistic voting scheme. An initial object pose is estimated using correlations between the model image and the 3D CAD model, which are predefined, and the homography, byproduct of the identification. In tracking, a Lie group formalism is used for robust and fast motion computation. Experimental results show that object recognition by the proposed method improves the recognition range considerably. Sungho Kim received the B.S. degree in Electrical Engineering from Korea University, Korea in 2000 and the M.S. degree in Electrical Engineering and Computer Science from Korea Advanced Institute of Science and Technology, Korea in 2002. He is currently pursuing his Ph.D. at the latter institution, concentrating on 3D object recognition and tracking. In So Kweon received the Ph.D. degree in robotics from Carnegie Mellon University, Pittsburgh, PA, in 1990. Since 1992, he has been a Professor of Electrical Engineering at KAIST. His current research interests include human visual perception, object recognition, real-time tracking, vision-based mobile robot localization, volumetric 3D reconstruction, and camera calibration. He is a member of the IEEE, and Korea Robotics Society (KRS).     

Multiple 3D object position estimation and tracking using double filtering on multi-core processor

We present a new algorithm to tracking multiple 3D objects that has robustness, real-time processing ability and fast object registration. Usually, many augmented reality applications want to track 3D object using natural features in real-time, more accuracy and want to register target object immediately in few seconds. Prevalent object tracking algorithm uses FERN for feature extraction that takes long time to register and learning target object for high quality performance. Our method provides not only high accuracy but also fast target object registering time about 0.3 ms in same environment and real-time processing. These features are presented by using SURF, ROI, double robust filtering and optimized multi-core parallelization. Using our methods, tracking multiple 3D objects with fast and high accuracy is available.     

Stacked convolutional auto-encoders for surface recognition based on 3d point cloud data

This paper addresses the problem of feature extraction for 3d point cloud data using a deep-structured auto-encoder. As one of the most focused research areas in human–robot interaction (HRI), the vision-based object recognition is very important. To recognize object using the most common geometry feature, surface condition that can be obtained from 3d point cloud data could decrease the error during the HRI. In this research, the surface normal vectors are used to convert 3D point cloud data to a surface-condition-feature map, and a sub-route stacked convolution auto-encoder (sCAE) is designed to classify the difference between the surfaces. The result of the trained filters and the classification of sCAE shows the surface-condition-feature and the specified sCAE are very effective in the variation of surface condition.     

Efficient 3D object recognition using foveated point clouds

Recent hardware technologies have enabled acquisition of 3D point clouds from real world scenes in real time. A variety of interactive applications with the 3D world can be developed on top of this new technological scenario. However, a main problem that still remains is that most processing techniques for such 3D point clouds are computationally intensive, requiring optimized approaches to handle such images, especially when real time performance is required. As a possible solution, we propose the use of a 3D moving fovea based on a multiresolution technique that processes parts of the acquired scene using multiple levels of resolution. Such approach can be used to identify objects in point clouds with efficient timing. Experiments show that the use of the moving fovea shows a seven fold performance gain in processing time while keeping 91.6% of true recognition rate in comparison with state-of-the-art 3D object recognition methods.     

HDRNet: High-Dimensional Regression Network for Point Cloud Registration

Abstract-3D point cloud registration is a crucial topic in the reverse engineering, computer vision and robotics fields. The core of this problem is to estimate a transformation matrix for aligning the source point cloud with a target point cloud. Several learning-based methods have achieved a high performance. However, they are challenged with both partial overlap point clouds and multiscale point clouds, since they use the singular value decomposition (SVD) to find the rotation matrix without fully considering the scale information. Furthermore, previous networks cannot effectively handle the point clouds having large initial rotation angles, which is a common practical case. To address these problems, this paper presents a learning-based point cloud registration network, namely HDRNet, which consists of four stages: local feature extraction, correspondence matrix estimation, feature embedding and fusion and parametric regression. HDRNet is robust to noise and large rotation angles, and can effectively handle the partial overlap and multi-scale point clouds registration. The proposed model is trained on the ModelNet40 dataset, and compared with ICP, SICP, FGR and recent learning-based methods (PCRNet, IDAM, RGMNet and GMCNet) under several settings, including its performance on moving to invisible objects, with higher success rates. To verify the effectiveness and generality of our model, we also further tested our model on the Stanford 3D scanning repository.     

基于毫米波雷达稀疏点云的人体行为识别方法

目前利用毫米波雷达进行人体行为识别的方法在复杂场景下无法很好的区分相似动作,与此同时模型的鲁棒性和抗干扰能力也相对较差;针对以上两个问题,提出了一种通用的基于毫米波雷达稀疏点云的人体行为识别方法,该方法首先利用K-means++聚类算法对点云进行采样,然后使用基于注意力特征融合的点云活动分类网络进行人体行为特征的提取和识别,该网络可以兼顾点云的空间特征以及时序特征,对稀疏点云的运动有灵敏的感知能力;为了验证所提出方法的有效性和鲁棒性,分别在MMActivity数据集和MMGesture数据集上进行了实验,其在两个数据集上取得97.50%和94.10%的准确率,均优于其它方法;此外,进一步验证了K-means++点云采样方法的有效性,相较于随机采样,准确率提升了0.4个百分点,实验结果表明所提出方法能够有效的提升人体行为识别的准确率,且模型具有较好的泛化能力。     

A 3D shape segmentation approach for robot grasping by parts

Neuro-psychological findings have shown that human perception of objects is based on part decomposition. Most objects are made of multiple parts which are likely to be the entities actually involved in grasp affordances. Therefore, automatic object recognition and robot grasping should take advantage from 3D shape segmentation. This paper presents an approach toward planning robot grasps across similar objects by part correspondence. The novelty of the method lies in the topological decomposition of objects that enables high-level semantic grasp planning.In particular, given a 3D model of an object, the representation is initially segmented by computing its Reeb graph. Then, automatic object recognition and part annotation are performed by applying a shape retrieval algorithm. After the recognition phase, queries are accepted for planning grasps on individual parts of the object. Finally, a robot grasp planner is invoked for finding stable grasps on the selected part of the object. Grasps are evaluated according to a widely used quality measure. Experiments performed in a simulated environment on a reasonably large dataset show the potential of topological segmentation to highlight candidate parts suitable for grasping.