Simultaneous nonrigid registration of multiple point sets and atlas construction

Group-wise registration of a set of shapes represented by unlabeled point-sets is a challenging problem since, usually this involves solving for point correspondence in a nonrigid motion setting. In this paper, we propose a novel and robust algorithm that is capable of simultaneously computing the mean shape represented by a probability density function from multiple unlabeled point-sets and registering them non-rigidly to this emerging mean shape. This algorithm avoids the correspondence problem by minimizing the Jensen-Shannon (JS) divergence between the point sets. We motivate the use of the JS divergence by pointing out its close relationship to hypothesis testing. We derive the analytic gradient of the cost function in order to efficiently achieve the optimal solution. JS-divergence is symmetric with no bias toward any of the given shapes to be registered and whose mean is being sought. A by product of the registration process is a probabilistic atlas defined as the convex combination of the probability densities of the input point sets being aligned. Our algorithm can be especially useful for creating atlases of various shapes present in images as well as for simultaneously (rigidly or non-rigidly) registering 3D range data sets without having to establish any correspondence. We present experimental results on real and synthetic data.     

A Novel Kernel Correlation Model with the Correspondence Estimation

We present a novel multiple-linked iterative closest point method to estimate correspondences and the rigid/non-rigid transformations between point-sets or shapes. The estimation task is carried out by maximizing a symmetric similarity function, which is the product of the square roots of correspondences and a kernel correlation. The local mean square error analysis and robustness analysis are provided to show our method’s superior performance to the kernel correlation method.     

Semi-Supervised 3D Shape Recognition via Multimodal Deep Co-training

3D shape recognition has been actively investigated in the field of computer graphics. With the rapid development of deep learning, various deep models have been introduced and achieved remarkable results. Most 3D shape recognition methods are supervised and learn only from the large amount of labeled shapes. However, it is expensive and time consuming to obtain such a large training set. In contrast to these methods, this paper studies a semi-supervised learning framework to train a deep model for 3D shape recognition by using both labeled and unlabeled shapes. Inspired by the co-training algorithm, our method iterates between model training and pseudo-label generation phases. In the model training phase, we train two deep networks based on the point cloud and multi-view representation simultaneously. In the pseudo-label generation phase, we generate the pseudo-labels of the unlabeled shapes using the joint prediction of two networks, which augments the labeled set for the next iteration. To extract more reliable consensus information from multiple representations, we propose an uncertainty-aware consistency loss function to combine the two networks into a multimodal network. This not only encourages the two networks to give similar predictions on the unlabeled set, but also eliminates the negative influence of the large performance gap between the two networks. Experiments on the benchmark ModelNet40 demonstrate that, with only 10% labeled training data, our approach achieves competitive performance to the results reported by supervised methods.     

基于权重能量自适应分布的三维形状分割算法

针对三维形状分割问题,提出一种引入权重能量自适应分布参与深度神经网络训练的全监督分割算法.首先对三维形状表面进行过分割得到若干小块,提取每一个小块的特征描述符向量作为神经网络的输入,计算权重能量自适应分布,将经过加权后的分割标签作为神经网络的输出,训练深度神经网络.对于新的未分割的三维模型,提取模型表面三角面片的特征向量后输入到神经网络中进行预测分割后,对预测分割的边缘进行修整得到分割结果,实现三维模型的自动分割.在普林斯顿三维模型分割数据集上的实验结果表明,算法通过在训练过程中引入权重能量自适应分布,可以大幅降低神经网络训练时的均方误差,提高神经网络预测结果的准确率;与传统算法相比,该算法具有高准确率、强鲁棒性、强学习扩展能力等优点.     

LRA: Local Rigid Averaging of Stretchable Non-rigid Shapes

We present a novel algorithm for generating the mean structure of non-rigid stretchable shapes. Following an alignment process, which supports local affine deformations, we translate the search of the mean shape into a diagonalization problem where the structure is hidden within the kernel of a matrix. This is the first step required in many practical applications, where one needs to model bendable and stretchable shapes from multiple observations.     

EM-GPA: Generalized Procrustes analysis with hidden variables for 3D shape modeling

Aligning shapes is essential in many computer vision problems and generalized Procrustes analysis (GPA) is one of the most popular algorithms to align shapes. However, if some of the shape data are missing, GPA cannot be applied. In this paper, we propose EM-GPA, which extends GPA to handle shapes with hidden (missing) variables by using the expectation-maximization (EM) algorithm. For example, 2D shapes can be considered as 3D shapes with missing depth information due to the projection of 3D shapes into the image plane. For a set of 2D shapes, EM-GPA finds scales, rotations and 3D shapes along with their mean and covariance matrix for 3D shape modeling. A distinctive characteristic of EM-GPA is that it does not enforce any rank constraint often appeared in other work and instead uses GPA constraints to resolve the ambiguity in finding scales, rotations, and 3D shapes. The experimental results show that EM-GPA can recover depth information accurately even when the noise level is high and there are a large number of missing variables. By using the images from the FRGC database, we show that EM-GPA can successfully align 2D shapes by taking the missing information into consideration. We also demonstrate that the 3D mean shape and its covariance matrix are accurately estimated. As an application of EM-GPA, we construct a 2D + 3D AAM (active appearance model) using the 3D shapes obtained by EM-GPA, and it gives a similar success rate in model fitting compared to the method using real 3D shapes. EM-GPA is not limited to the case of missing depth information, but it can be easily extended to more general cases.     

点集模型的直接可见性测试与基于视点的绘制

点集模型作为一种新兴的三维几何形体表示形式,近年来备受关注.本文运用点集法向计算与凸包构建等技术,对原始点集模型进行直接可视性计算,并利用可见性计算的结果对点集模型进行基于视点的绘制.算法首先对原始模型进行基于视点的精简,剔除大部分不可见点;再对精简后的模型进行球面对称变换,并构建变换后点集的凸包,进而提取出可见点集;最后运用真实感图形绘制技术实现可见点集的快速绘制.实验证明,本文算法能够快速地计算点集模型中采样点的可见性.该算法可应用于点集模型基于视点的绘制与曲面重建,以及点集模型的阴影绘制等领域.     

断层图象插值的二维形变法

提出了一种新的二维断层图象层间插值方法。该方法将有向距离变换和二维形变理论应用于断层图象的层间插值，继而应用于三维重建。该插值方法有３个步骤。首先，计算目标形状的位置和方向；其次，计算目标形状的距离映射图，并进行形状变换，最后，用迭代法对目标的位置和方向进行线性插值，对形状进行非线性插值。文中给出该插值方法应用于模拟图象和ＭＲＩ图象的形状插值和三维重建的结果。     

ArchiDNA: An interactive system for creating 2D and 3D conceptual drawings in architectural design

We present an interactive system called ArchiDNA for creating 2D and 3D conceptual drawings in architectural design. We developed a novel principle of shape generation called match-and-attach by analyzing drawing styles of a contemporary architect, Peter Eisenman. The process consists of user interaction techniques and a set of rules that decide how one or more shapes attach to another shape. One key ingredient of our process is a unique concept for the interactive semi-automatic shape generation that uses the combination of algorithmic rules of a computer and designers’ manual inputs. These techniques enable designers to use CAD software in the early stages of architectural designs to explore conceptual building forms. ArchiDNA dynamically responds to drawing inputs, configures 2D shapes, and converts them to 3D shapes in a similar style. We intend to complement existing CAD software and computational drawing pipelines for intuitive 2D and 3D conceptual drawing creation.     

Shape statistics in kernel space for variational image segmentation

We present a variational integration of nonlinear shape statistics into a Mumford-Shah based segmentation process. The nonlinear statistics are derived from a set of training silhouettes by a novel method of density estimation which can be considered as an extension of kernel PCA to a probabilistic framework.We assume that the training data forms a Gaussian distribution after a nonlinear mapping to a higher-dimensional feature space. Due to the strong nonlinearity, the corresponding density estimate in the original space is highly non-Gaussian.Applications of the nonlinear shape statistics in segmentation and tracking of 2D and 3D objects demonstrate that the segmentation process can incorporate knowledge on a large variety of complex real-world shapes. It makes the segmentation process robust against misleading information due to noise, clutter and occlusion.     

Generating shapes by analogies: An application to hearing aid design

3D shape modeling is a crucial component of rapid prototyping systems that customize shapes of implants and prosthetic devices to a patient’s anatomy. In this paper, we present a solution to the problem of customized 3D shape modeling using a statistical shape analysis framework. We design a novel method to learn the relationship between two classes of shapes, which are related by certain operations or transformation. The two associated shape classes are represented in a lower dimensional manifold, and the reduced set of parameters obtained in this subspace is utilized in an estimation, which is exemplified by a multivariate regression in this paper. We demonstrate our method with a felicitous application to the estimation of customized hearing aid devices.     

Robust Structure‐Based Shape Correspondence

We present a robust method to find region‐level correspondences between shapes, which are invariant to changes in geometry and applicable across multiple shape representations. We generate simplified shape graphs by jointly decomposing the shapes, and devise an adapted graph‐matching technique, from which we infer correspondences between shape regions. The simplified shape graphs are designed to primarily capture the overall structure of the shapes, without reflecting precise information about the geometry of each region, which enables us to find correspondences between shapes that might have significant geometric differences. Moreover, due to the special care we take to ensure the robustness of each part of our pipeline, our method can find correspondences between shapes with different representations, such as triangular meshes and point clouds. We demonstrate that the region‐wise matching that we obtain can be used to find correspondences between feature points, reveal the intrinsic self‐similarities of each shape and even construct point‐to‐point maps across shapes. Our method is both time and space efficient, leading to a pipeline that is significantly faster than comparable approaches. We demonstrate the performance of our approach through an extensive quantitative and qualitative evaluation on several benchmarks where we achieve comparable or superior performance to existing methods.     

3D Morphing Using Strain Field Interpolation

In this paper, we present a new technique based on strain fields to carry out 3D shape morphing for applications in computer graphics and related areas. Strain is an important geometric quantity used in mechanics to describe the deformation of objects. We apply it in a novel way to analyze and control deformation in morphing. Using position vector fields, the strain field relating source and target shapes can be obtained. By interpolating this strain field between zero and a final desired value we can obtain the position field for intermediate shapes. This method ensures that the 3D morphing process is smooth. Locally, volumes suffer minimal distortion, and no shape jittering or wobbling happens： other methods do not necessarily have these desirable properties. We also show how to control the method so that changes of shape （in particular, size changes） vary linearly with time.     

Partial 3‐D Correspondence from Shape Extremities

We present a 3‐D correspondence method to match the geometric extremities of two shapes which are partially isometric. We consider the most general setting of the isometric partial shape correspondence problem, in which shapes to be matched may have multiple common parts at arbitrary scales as well as parts that are not similar. Our rank‐and‐vote‐and‐combine algorithm identifies and ranks potentially correct matches by exploring the space of all possible partial maps between coarsely sampled extremities. The qualified top‐ranked matchings are then subjected to a more detailed analysis at a denser resolution and assigned with confidence values that accumulate into a vote matrix. A minimum weight perfect matching algorithm is finally iterated to combine the accumulated votes into an optimal (partial) mapping between shape extremities, which can further be extended to a denser map. We test the performance of our method on several data sets and benchmarks in comparison with state of the art.     

Multiple Shape Correspondence by Dynamic Programming

We present a multiple shape correspondence method based on dynamic programming, that computes consistent bijective maps between all shape pairs in a given collection of initially unmatched shapes. As a fundamental distinction from previous work, our method aims to explicitly minimize the overall distortion, i.e., the average isometric distortion of the resulting maps over all shape pairs. We cast the problem as optimal path finding on a graph structure where vertices are maps between shape extremities. We exploit as much context information as possible using a dynamic programming based algorithm to approximate the optimal solution. Our method generates coarse multiple correspondences between shape extremities, as well as denser correspondences as by‐product. We assess the performance on various mesh sequences of (nearly) isometric shapes. Our experiments show that, for isometric shape collections with non‐uniform triangulation and noise, our method can compute relatively dense correspondences reasonably fast and outperform state of the art in terms of accuracy.     

Patch‐type Segmentation of Voxel Shapes using Simplified Surface Skeletons

We present a new method for decomposing a 3D voxel shape into disjoint segments using the shape's simplified surface‐skeleton. The surface skeleton of a shape consists of 2D manifolds inside its volume. Each skeleton point has a maximally inscribed ball that touches the boundary in at least two contact points. A key observation is that the boundaries of the simplified fore‐ and background skeletons map one‐to‐one to increasingly fuzzy, soft convex, respectively concave, edges of the shape. Using this property, we build a method for segmentation of 3D shapes which has several desirable properties. Our method segments both noisy shapes and shapes with soft edges which vanish over low‐curvature regions. Multiscale segmentations can be obtained by varying the simplification level of the skeleton. We present a voxel‐based implementation of our approach and illustrate it on several realistic examples.     

On the shape of a set of points and lines in the plane

Detailed geometric models of the real world are in increasing demand. LiDAR data is appropriate to reconstruct urban models. In urban scenes, the individual surfaces can be reconstructed and connected to form the scene geometry. There are various methods for reconstructing the free‐form shape of a point sample on a single surface. However, these methods do not take the context of the surface into account. We present the guided α‐shape: an extension of the well known α‐shape that uses lines (guides) to indicate preferred locations for the boundary of the shape. The guided α‐shape uses (parts of) these lines as boundary where the points suggest that this is appropriate. We prove that the guided α‐shape can be constructed in O((n + m) log (n + m)) time, from an input of n points and m guides. We apply guided α‐shapes to urban reconstruction from LiDAR, where neighboring surfaces can be connected conveniently along their intersection lines into adjacent surfaces of a 3D model. We analyze guided α‐shapes of both synthetic and real data and show they are consistently better than α‐shapes for this application.     

Data‐Driven Sparse Priors of 3D Shapes

We present a sparse optimization framework for extracting sparse shape priors from a collection of 3D models. Shape priors are defined as point‐set neighborhoods sampled from shape surfaces which convey important information encompassing normals and local shape characterization. A 3D shape model can be considered to be formed with a set of 3D local shape priors, while most of them are likely to have similar geometry. Our key observation is that the local priors extracted from a family of 3D shapes lie in a very low‐dimensional manifold. Consequently, a compact and informative subset of priors can be learned to efficiently encode all shapes of the same family. A comprehensive library of local shape priors is first built with the given collection of 3D models of the same family. We then formulate a global, sparse optimization problem which enforces selecting representative priors while minimizing the reconstruction error. To solve the optimization problem, we design an efficient solver based on the Augmented Lagrangian Multipliers method (ALM). Extensive experiments exhibit the power of our data‐driven sparse priors in elegantly solving several high‐level shape analysis applications and geometry processing tasks, such as shape retrieval, style analysis and symmetry detection.     

PCA-based 3D shape reconstruction of human foot using multiple viewpoint cameras

This paper describes a multiple camera-based method to reconstruct the 3D shape of a human foot. From a foot database, an initial 3D model of the foot represented by a cloud of points is built. The shape parameters, which can characterize more than 92% of a foot, are defined by using the principal component analysis method. Then, using ＂active shape models＂, the initial 3D model is adapted to the real foot captured in multiple images by applying some constraints （edge points＇ distance and color variance）. We insist here on the experiment part where we demonstrate the efficiency of the proposed method on a plastic foot model, and also on real human feet with various shapes. We propose and compare different ways of texturing the foot which is needed for reconstruction. We present an experiment performed on the plastic foot model and on human feet and propose two different ways to improve the final 3D shapers accuracy according to the previous experiments＇ results. The first improvement proposed is the densification of the cloud of points used to represent the initial model and the foot database. The second improvement concerns the projected patterns used to texture the foot. We conclude by showing the obtained results for a human foot with the average computed shape error being only 1.06 mm.     

A level-set approach for the metamorphosis of solid models

We present a new approach to 3D shape metamorphosis. We express the interpolation of two shapes as a process where one shape deforms to maximize its similarity with another shape. The process incrementally optimizes an objective function while deforming an implicit surface model. We represent the deformable surface as a level set (iso-surface) of a densely sampled scalar function of three dimensions. Such level-set models have been shown to mimic conventional parametric deformable surface models by encoding surface movements as changes in the grayscale values of a volume data set. Thus, a well-founded mathematical structure leads to a set of procedures that describes how voxel values can be manipulated to create deformations that are represented as a sequence of volumes. The result is a 3D morphing method that offers several advantages over previous methods, including minimal need for user input, no model parameterization, flexible topology, and subvoxel accuracy