Temporal Upsampling of Point Cloud Sequences by Optimal Transport for Plant Growth Visualization

Plant growth visualization from a series of 3D scanner measurements is a challenging task. Time intervals between successive measurements are typically too large to allow a smooth animation of the growth process. Therefore, obtaining a smooth animation of the plant growth process requires a temporal upsampling of the point cloud sequence in order to obtain approximations of the intermediate states between successive measurements. Additionally, there are suddenly arising structural changes due to the occurrence of new plant parts such as new branches or leaves. We present a novel method that addresses these challenges via semantic segmentation and the generation of a segment hierarchy per scan, the matching of the hierarchical representations of successive scans and the segment-wise computation of optimal transport. The transport problems' solutions yield the information required for a realistic temporal upsampling, which is generated in real time. Thereby, our method does not require shape templates, good correspondences or huge databases of examples. Newly grown and decayed parts of the plant are detected as unmatched segments and are handled by identifying corresponding bifurcation points and introducing virtual segments in the previous, respectively successive time step. Our method allows the generation of realistic upsampled growth animations with moderate computational effort.     

FARM: Functional Automatic Registration Method for 3D Human Bodies

We introduce a new method for non-rigid registration of 3D human shapes. Our proposed pipeline builds upon a given parametric model of the human, and makes use of the functional map representation for encoding and inferring shape maps throughout the registration process. This combination endows our method with robustness to a large variety of nuisances observed in practical settings, including non-isometric transformations, downsampling, topological noise and occlusions; further, the pipeline can be applied invariably across different shape representations (e.g. meshes and point clouds), and in the presence of (even dramatic) missing parts such as those arising in real-world depth sensing applications. We showcase our method on a selection of challenging tasks, demonstrating results in line with, or even surpassing, state-of-the-art methods in the respective areas.     

PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds

Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers. Traditional methods for point cloud denoising largely rely on local surface fitting (e.g. jets or MLS surfaces), local or non-local averaging or on statistical assumptions about the underlying noise model. In contrast, we develop a simple data-driven method for removing outliers and reducing noise in unordered point clouds. We base our approach on a deep learning architecture adapted from PCPNet, which was recently proposed for estimating local 3D shape properties in point clouds. Our method first classifies and discards outlier samples, and then estimates correction vectors that project noisy points onto the original clean surfaces. The approach is efficient and robust to varying amounts of noise and outliers, while being able to handle large densely sampled point clouds. In our extensive evaluation, both on synthetic and real data, we show an increased robustness to strong noise levels compared to various state-of-the-art methods, enabling accurate surface reconstruction from extremely noisy real data obtained by range scans. Finally, the simplicity and universality of our approach makes it very easy to integrate in any existing geometry processing pipeline. Both the code and pre-trained networks can be found on the project page ( https://github.com/mrakotosaon/pointcleannet ).     

基于点云配准的人脸三维尺寸测量

接触式人脸三维尺寸测量易损坏表面特征,依赖于特征点标定,常含冗余信息.针对该问题,提出一种基于结构光与多视图图像点云配准的非接触式人脸三维尺寸测量方法.首先利用改进的迭代最近点算法建立转换函数,求出尺度因子、旋转矩阵和平移向量;然后基于模糊C均值算法对人脸面部进行聚类分割以获得候选区域;针对人脸表面离散点云不平整问题,...     

Wavelet‐based Heat Kernel Derivatives: Towards Informative Localized Shape Analysis

In this paper, we propose a new construction for the Mexican hat wavelets on shapes with applications to partial shape matching. Our approach takes its main inspiration from the well‐established methodology of diffusion wavelets. This novel construction allows us to rapidly compute a multi‐scale family of Mexican hat wavelet functions, by approximating the derivative of the heat kernel. We demonstrate that this leads to a family of functions that inherit many attractive properties of the heat kernel (e.g. local support, ability to recover isometries from a single point, efficient computation). Due to its natural ability to encode high‐frequency details on a shape, the proposed method reconstructs and transfers ‐functions more accurately than the Laplace‐Beltrami eigenfunction basis and other related bases. Finally, we apply our method to the challenging problems of partial and large‐scale shape matching. An extensive comparison to the state‐of‐the‐art shows that it is comparable in performance, while both simpler and much faster than competing approaches.     

ZerNet: Convolutional Neural Networks on Arbitrary Surfaces Via Zernike Local Tangent Space Estimation

In this paper, we propose a novel formulation extending convolutional neural networks (CNN) to arbitrary two-dimensional manifolds using orthogonal basis functions called Zernike polynomials. In many areas, geometric features play a key role in understanding scientific trends and phenomena, where accurate numerical quantification of geometric features is critical. Recently, CNNs have demonstrated a substantial improvement in extracting and codifying geometric features. However, the progress is mostly centred around computer vision and its applications where an inherent grid-like data representation is naturally present. In contrast, many geometry processing problems deal with curved surfaces and the application of CNNs is not trivial due to the lack of canonical grid-like representation, the absence of globally consistent orientation and the incompatible local discretizations. In this paper, we show that the Zernike polynomials allow rigourous yet practical mathematical generalization of CNNs to arbitrary surfaces. We prove that the convolution of two functions can be represented as a simple dot product between Zernike coefficients and the rotation of a convolution kernel is essentially a set of 2 × 2 rotation matrices applied to the coefficients. The key contribution of this work is in such a computationally efficient but rigorous generalization of the major CNN building blocks.     

RFNet: 用于三维点云分类的卷积神经网络

由于点云的非结构性和无序性, 目前已有的点云分类网络在精度上仍然需要进一步提高. 通过考虑局部结构的构建、全局特征聚合和损失函数改进三个方面, 构造一个有效的点云分类网络. 首先, 针对点云的非结构性, 通过学习中心点特征与近邻点特征之间的关系, 为不规则的近邻点分配不同的权重, 以此构建局部结构; 然后, 使用注意力思想, 提出加权平均池化(Weighted average pooling, WAP), 通过自注意力方式, 学习每个高维特征的注意力分数, 在应对点云无序性的同时, 可以有效地聚合冗余的高维特征; 最后, 利用交叉熵损失与中心损失之间的互补关系, 提出联合损失函数(Joint loss function, JL), 在增大类间距离的同时, 减小类内距离, 进一步提高了网络的分类能力. 在合成数据集ModelNet40、ShapeNetCore和真实世界数据集ScanObjectNN上进行实验, 与目前性能最好的多个网络相比较, 验证了该整体网络结构的优越性.     

结合方向包围框改进的三维点云配准算法

为了提升源点云和模板点云在初始相对偏转角度过大时的配准精度,提出了一种结合方向包围框的改进 PointNetLK算法PointNetLK-OBB。该算法用三维点云的方向包围框表示源点云和模板点云的宏观特征,在最近点迭代算法的引导下,对齐源点云和模板点云的方向包围框,并在源点云和模板点云间产生镜面对称效应;根据源点云和模板点云的拟合度探测镜面对称的对称面,得到源点云自身的最佳旋转和平移,完成三维点云配准任务。为了验证算法的有效性,在公开数据集ModelNet40上进行对比实验,实验结果显示,PointNetLK-OBB与PointNetLK相比,提升了源点云和模板点云在初始相对偏转角度过大时的配准精度,对源点云和模板点云间的初始相对位置敏感度降低。创新在于,利用PointNetLK绕开传统点云配准的非凸问题,借助于方向包围框的规整性避开PointNetLK语境下出现的局部最优问题。     

3D点云形状特征的二维主流形描述

首先,对空间分布不均匀且无序的三维点云构造其二维主流形,并以与球面同胚的封闭曲面网格形式给出其二维主流形的二次优化逼近,以主流形网格有序均匀的结点分布表示三维点云空间分布无序且不均匀的形状特征,降低了三维形状描述的难度;然后,以基本几何变换作为快速粗对齐、以迭代最近法向点(ICNP)方法作为精准对齐,确定两个主曲面网格之间最佳刚性变换,ICNP方法在寻找最近点时考虑法向夹角,利用了更多的几何信息,实现快速精准的刚性对齐,兼顾计算精度和速度;最后,以对齐误差作为两个3D点云之间形状差异测度.实验结果表明:所提出的基于主流形二次曲面网格优化逼近的三维点云模型形状描述方法对三维点云的分辨率和噪声等干扰因素具有较高的健壮性,可以用于三维检索的形状描述.     

基于GPU的实时三维点云数据配准研究

在三维重建中,不同摄像机坐标系下点云配准耗时过多。为此,提出一种基于图形处理单元(GPU)的实时三维点云数据配准算法。利用投影映射法获取匹配点对,使用点到切平面距离最小化方法计算变换矩阵,通过GPU多线程并行处理大规模图像数据。实验结果表明,对于分别包含307 200个数据的2帧点云,在保持原有配准效果的基础上,该算法的最优耗时仅为基于CPU的最近邻迭代算法的11.9%。     

A Novel Representation and Feature Matching Algorithm for Automatic Pairwise Registration of Range Images

Automatic registration of range images is a fundamental problem in 3D modeling of free-from objects. Various feature matching algorithms have been proposed for this purpose. However, these algorithms suffer from various limitations mainly related to their applicability, efficiency, robustness to resolution, and the discriminating capability of the used feature representation. We present a novel feature matching algorithm for automatic pairwise registration of range images which overcomes these limitations. Our algorithm uses a novel tensor representation which represents semi-local 3D surface patches of a range image by third order tensors. Multiple tensors are used to represent each range image. Tensors of two range images are matched to identify correspondences between them. Correspondences are verified and then used for pairwise registration of the range images. Experimental results show that our algorithm is accurate and efficient. Moreover, it is robust to the resolution of the range images, the number of tensors per view, the required amount of overlap, and noise. Comparisons with the spin image representation revealed that our representation has more discriminating capabilities and performs better at a low resolution of the range images. This work has been provisionally patented under Australian patent number 2004902436 and is sponsored by ARC grant number DP0344338.     

基于点云匹配的AR饰面作业系统跟踪注册方法

针对建筑机器人饰面作业过程中常因视觉遮挡导致作业效率低的问题,使用增强现实解决遮挡并提出一种基于点云匹配的增强现实跟踪注册方法。利用目标模型点云与作业环境点云的匹配进行目标的初始定位;利用改进的相关滤波跟踪算法对目标进行跟踪获取目标位置;基于迭代最近点法对目标位姿进行估计。在跟踪注册过程中加入位姿优化,保证目标位姿估计精度。为了更加准确地跟踪目标位置,提出一种特征融合和尺度自适应的改进相关滤波目标跟踪算法。通过板材安装实验,表明跟踪注册方法精确性、实时性均较好,最小识别误差达到2.88 mm,具有良好的虚实融合效果。     

基于旋转不变深度层次聚类网络的点云分析

由于解决了三维点云的排列不变性问题,基于三维点云的深度学习方法在计算机三维视觉领域中取得了重大的突破,人们逐渐倾向于使用三维点云来描述物体并基于神经网络结构来提取点云的特征.然而,现有的方法依然无法解决旋转不变性问题,使得目前的模型鲁棒性较差;同时,神经网络结构的设计过于启发式,没有合理利用三维点云的几何结构与分布特性,导致网络结构的表达能力有待提升.鉴于此,提出了一种具有良好兼容性的严格旋转不变性表达以及深度层次类簇网络,试图从理论与实践两个层面解决上述问题.在点云识别、部件分割、语义分割这3个经典任务上进行了旋转鲁棒性对比实验,均取得了最优的效果.     

二维点轮廓与矢量轮廓配准研究

在平面类零件的光学测量中,二维点轮廓与矢量轮廓的配准是关键算法,配准精度 直接影响测量精度。针对平面类零件的配准问题,提出了基于形状特征函数的粗配准算法和二维 矢量最近点迭代(ICP)精配准算法。利用角度距离图法将矢量图形的几何信息转化为独立于坐标系 的连续函数,进而实现粗配准算法。基于平面上点与曲线的最近距离算法计算配准目标函数,给 出了不同于传统的ICP 算法的直接求解目标函数的解析方法,有效提高了算法效率。利用实例验 证分析了该算法的高效性和可靠性。     

三维点云中的二维标记检测

特征检测在物体识别、数据配准等应用中具有至关重要的作用。同一场景中不同采集数据的配准和融合,必须已知或者估算不同数据中的共同特征对应点。然而,许多场景缺少有效对应特征点。解决该问题的一种有效的方法是在场景中添加标记以增加特征。文章提出一种在只含有位置信息的三维点云中自动检测二维标记的方法。该方法首先在三维场景添加黑色圆形薄纸片作为二维标记,利用区域增长法将获取的三维场景的点云数据分割成不同类别,然后基于随机抽样一致性算法的扩展方法依次对分割后的点云进行形状拟合,最后通过检测形状检测该二维标记。该方法能够有效地检测出三维场景中的二维标记,并避免了遮挡、形变等问题,为缺少特征的场景提供了简单可行的特征,可广泛应用于数据配准、物体识别、物体追踪、三维重建等领域。     

Machine vision for industry: Tasks,tools and techniques

The ‘why’, ‘how’ and ‘what’ of industrial machine vision systems are surveyed-why vision is important, how it is accomplished and what sorts of tasks it is being applied to. Examples are given of vision techniques and applications from Japan, France, the GDR and the USA.     

基于超点图网络的三维点云室内场景分割模型

针对点云数据集样本不均衡及PointNet网络无法充分利用点云邻域信息的问题,提出一种三维点云场景分割模型。根据几何信息将原始点云块同质分割为超点,利用小型PointNet网络将点云原始特征映射到高维空间中,并挖掘场景中深层语义信息。在此基础上,构建自归一化属性门控单元优化点云上下文语义分割效果,采用二维图像领域中的Focal Loss损失函数实现点云场景分割。实验结果表明,该模型在S3DIS数据集上的平均交并比、总体精度、平均精度分别达到63.8%、86.4%、74.3%,较SPG模型分别提升1.7、0.9、1.3个百分点。     

基于点云的球面三维逆向建模

球面拟合是三维逆向建模中面临的亟待解决的复杂难题之一。它广泛应用 于零件检测、建筑物结构恢复建模、医学血管和细胞模拟领域。论文改进了对三维球面点云 进行直接拟合的方法,得到球面的相关几何参量,并在速度和精确度上都获得了提高。论文 对拟合相关参数和噪声影响进行分析,并否定了对选定的数据集分组整合的算法。LM (Levenberg-Marquardt)算法是最广泛应用的最小二乘法二次曲面拟合的方法之一,本算法在 与LM 算法的对比中凸显了算法在时间和某些情况下精度的优越性,为进一步研究三维图形 拟合重建恢复等工作打下了基础。