Symmetry-seeking models and 3D object reconstruction

We propose models of 3D shape which may be viewed as deformable bodies composed of simulated elastic material. In contrast to traditional, purely geometric models of shape, deformable models are active—their shapes change in response to externally applied forces. We develop a deformable model for 3D shape which has a preference for axial symmetry. Symmetry is represented even though the model does not belong to a parametric shape family such as (generalized) cylinders. Rather, a symmetry-seeking property is designed into internal forces that constrain the deformations of the model. We develop a framework for 3D object reconstruction based on symmetry-seeking models. Instances of these models are formed from monocular image data through the action of external forces derived from the data. The forces proposed in this paper deform the model in space so that the shape of its projection into the image plane is consistent with the 2D silhouette of an object of interest. The effectiveness of our approach is demonstrated using natural images.     

3D object recognition using invariants of 2D projection curves

This paper presents a new method for recognizing 3D objects based on the comparison of invariants of their 2D projection curves. We show that Euclidean equivalent 3D surfaces imply affine equivalent 2D projection curves that are obtained from the projection of cross-section curves of the surfaces onto the coordinate planes. Planes used to extract cross-section curves are chosen to be orthogonal to the principal axes of the defining surfaces. Projection curves are represented using implicit polynomial equations. Affine algebraic and geometric invariants of projection curves are constructed and compared under a variety of distance measures. Results are verified by several experiments with objects from different classes and within the same class.     

Surface deformation models for nonrigid 3D shape recovery

Three-dimensional detection and shape recovery of a nonrigid surface from video sequences require deformation models to effectively take advantage of potentially noisy image data. Here, we introduce an approach to creating such models for deformable 3D surfaces. We exploit the fact that the shape of an inextensible triangulated mesh can be parameterized in terms of a small subset of the angles between its facets. We use this set of angles to create a representative set of potential shapes, which we feed to a simple dimensionality reduction technique to produce low-dimensional 3D deformation models. We show that these models can be used to accurately model a wide range of deforming 3D surfaces from video sequences acquired under realistic conditions.     

Implicit simplicial models for adaptive curve reconstruction

Parametric deformable models have been extensively and very successfully used for reconstructing free-form curves and surfaces, and for tracking nonrigid deformations, but they require previous knowledge of the topological type of the data, and good initial curve or surface estimates. With deformable models, it is also computationally expensive to check for and to prevent self-intersections while tracking deformations. The implicit simplicial models that we introduce in this paper are implicit curves and surfaces defined by piecewise linear functions. This representation allows for local deformations, control of the topological type, and prevention of self-intersections during deformations. As a first application, we also describe an algorithm for 2D curve reconstruction from unorganized sets of data points. The topology, the number of connected components, and the geometry of the data are all estimated using an adaptive space subdivision approach. The main four components of the algorithm are topology estimation, curve fitting, adaptive space subdivision, and mesh relaxation     

Practical reliable Bayesian recognition of 2D and 3D objects usingimplicit polynomials and algebraic invariants

We treat the use of more complex higher degree polynomial curves and surfaces of degree higher than 2, which have many desirable properties for object recognition and position estimation, and attack the instability problem arising in their use with partial and noisy data. The scenario discussed in this paper is one where we have a set of objects that are modeled as implicit polynomial functions, or a set of representations of classes of objects with each object in a class modeled as an implicit polynomial function, stored in the database. Then, given partial data from one of the objects, we want to recognize the object (or the object class) or collect more data in order to get better parameter estimates for more reliable recognition. Two problems arising in this scenario are discussed: 1) the problem of recognizing these polynomials by comparing them in terms of their coefficients; and 2) the problem of where to collect data so as to improve the parameter estimates as quickly as possible. We use an asymptotic Bayesian approximation for solving the two problems. The intrinsic dimensionality of polynomials and the use of the Mahalanobis distance are discussed     

Implicit Surface-Based Geometric Fusion

This paper introduces a general purpose algorithm for reliable integration of sets of surface measurements into a single 3D model. The new algorithm constructs a single continuous implicit surface representation which is the zero-set of a scalar field function. An explicit object model is obtained using any implicit surface polygonization algorithm. Object models are reconstructed from both multiple view conventional 2.5D range images and hand-held sensor range data. To our knowledge this is the first geometric fusion algorithm capable of reconstructing 3D object models from noisy hand-held sensor range data.This approach has several important advantages over existing techniques. The implicit surface representation allows reconstruction of unknown objects of arbitrary topology and geometry. A continuous implicit surface representation enables reliable reconstruction of complex geometry. Correct integration of overlapping surface measurements in the presence of noise is achieved using geometric constraints based on measurement uncertainty. The use of measurement uncertainty ensures that the algorithm is robust to significant levels of measurement noise. Previous implicit surface-based approaches use discrete representations resulting in unreliable reconstruction for regions of high curvature or thin surface sections. Direct representation of the implicit surface boundary ensures correct reconstruction of arbitrary topology object surfaces. Fusion of overlapping measurements is performed using operations in 3D space only. This avoids the local 2D projection required for many previous methods which results in limitations on the object surface geometry that is reliably reconstructed. All previous geometric fusion algorithms developed for conventional range sensor data are based on the 2.5D image structure preventing their use for hand-held sensor data. Performance evaluation of the new integration algorithm against existing techniques demonstrates improved reconstruction of complex geometry.     

Describing complicated objects by implicit polynomials

This paper introduces and focuses on two problems. First is the representation power of closed implicit polynomials of modest degree for curves in 2-D images and surfaces in 3-D range data. Super quadrics are a small subset of object boundaries that are well fitted by these polynomials. The second problem is the stable computationally efficient fitting of noisy data by closed implicit polynomial curves and surfaces. The attractive features of these polynomials for Vision is discussed     

Learning 3D Deformation of Animals from 2D Images

Understanding how an animal can deform and articulate is essential for a realistic modification of its 3D model. In this paper, we show that such information can be learned from user‐clicked 2D images and a template 3D model of the target animal. We present a volumetric deformation framework that produces a set of new 3D models by deforming a template 3D model according to a set of user‐clicked images. Our framework is based on a novel locally‐bounded deformation energy, where every local region has its own stiffness value that bounds how much distortion is allowed at that location. We jointly learn the local stiffness bounds as we deform the template 3D mesh to match each user‐clicked image. We show that this seemingly complex task can be solved as a sequence of convex optimization problems. We demonstrate the effectiveness of our approach on cats and horses, which are highly deformable and articulated animals. Our framework produces new 3D models of animals that are significantly more plausible than methods without learned stiffness.     

Object tracking using deformable templates

We propose a method for object tracking using prototype-based deformable template models. To track an object in an image sequence, we use a criterion which combines two terms: the frame-to-frame deviations of the object shape and the fidelity of the modeled shape to the input image. The deformable template model utilizes the prior shape information which is extracted from the previous frames along with a systematic shape deformation scheme to model the object shape in a new frame. The following image information is used in the tracking process: 1) edge and gradient information: the object boundary consists of pixels with large image gradient, 2) region consistency: the same object region possesses consistent color and texture throughout the sequence, and 3) interframe motion: the boundary of a moving object is characterized by large interframe motion. The tracking proceeds by optimizing an objective function which combines both the shape deformation and the fidelity of the modeled shape to the current image (in terms of gradient, texture, and interframe motion). The inherent structure in the deformable template, together with region, motion, and image gradient cues, makes the proposed algorithm relatively insensitive to the adverse effects of weak image features and moderate amounts of occlusion     

Deformable M-Reps for 3D Medical Image Segmentation

M-reps (formerly called DSLs) are a multiscale medial means for modeling and rendering 3D solid geometry. They are particularly well suited to model anatomic objects and in particular to capture prior geometric information effectively in deformable models segmentation approaches. The representation is based on figural models, which define objects at coarse scale by a hierarchy of figures—each figure generally a slab representing a solid region and its boundary simultaneously. This paper focuses on the use of single figure models to segment objects of relatively simple structure.A single figure is a sheet of medial atoms, which is interpolated from the model formed by a net, i.e., a mesh or chain, of medial atoms (hence the name m-reps), each atom modeling a solid region via not only a position and a width but also a local figural frame giving figural directions and an object angle between opposing, corresponding positions on the boundary implied by the m-rep. The special capability of an m-rep is to provide spatial and orientational correspondence between an object in two different states of deformation. This ability is central to effective measurement of both geometric typicality and geometry to image match, the two terms of the objective function optimized in segmentation by deformable models. The other ability of m-reps central to effective segmentation is their ability to support segmentation at multiple levels of scale, with successively finer precision. Objects modeled by single figures are segmented first by a similarity transform augmented by object elongation, then by adjustment of each medial atom, and finally by displacing a dense sampling of the m-rep implied boundary. While these models and approaches also exist in 2D, we focus on 3D objects.The segmentation of the kidney from CT and the hippocampus from MRI serve as the major examples in this paper. The accuracy of segmentation as compared to manual, slice-by-slice segmentation is reported.     

Minimal surfaces based object segmentation

A geometric approach for 3D object segmentation and representation is presented. The segmentation is obtained by deformable surfaces moving towards the objects to be detected in the 3D image. The model is based on curvature motion and the computation of surfaces with minimal areas, better known as minimal surfaces. The space where the surfaces are computed is induced from the 3D image (volumetric data) in which the objects are to be detected. The model links between classical deformable surfaces obtained via energy minimization, and intrinsic ones derived from curvature based flows. The new approach is stable, robust, and automatically handles changes in the surface topology during the deformation     

3D anatomical shape atlas construction using mesh quality preserved deformable models

3D anatomical shape atlas construction has been extensively studied in medical image analysis research, owing to its importance in model-based image segmentation, longitudinal studies and populational statistical analysis, etc. Among multiple steps of 3D shape atlas construction, establishing anatomical correspondences across subjects, i.e., surface registration, is probably the most critical but challenging one. Adaptive focus deformable model (AFDM) [1] was proposed to tackle this problem by exploiting cross-scale geometry characteristics of 3D anatomy surfaces. Although the effectiveness of AFDM has been proved in various studies, its performance is highly dependent on the quality of 3D surface meshes, which often degrades along with the iterations of deformable surface registration (the process of correspondence matching). In this paper, we propose a new framework for 3D anatomical shape atlas construction. Our method aims to robustly establish correspondences across different subjects and simultaneously generate high-quality surface meshes without removing shape details. Mathematically, a new energy term is embedded into the original energy function of AFDM to preserve surface mesh qualities during deformable surface matching. More specifically, we employ the Laplacian representation to encode shape details and smoothness constraints. An expectation–maximization style algorithm is designed to optimize multiple energy terms alternatively until convergence. We demonstrate the performance of our method via a set of diverse applications, including a population of sparse cardiac MRI slices with 2D labels, 3D high resolution CT cardiac images and rodent brain MRIs with multiple structures. The constructed shape atlases exhibit good mesh qualities and preserve fine shape details. The constructed shape atlases can further benefit other research topics such as segmentation and statistical analysis.     

New feature points based on geometric invariants for 3D image registration

We introduce in this paper a new type of feature points of 3D surfaces, based on geometric invariants. We call this new type of feature points the extremal points of the 3D surfaces, and we show that the relative positions of those 3D points are invariant according to 3D rigid transforms (rotation and translation). We show also how to extract those points from 3D images, such as Magnetic Resonance images (MRI) or Cat-Scan images, and also how to use them to perform precise 3D registration. Previously, we described a method, called the Marching Lines algorithm, which allow us to extract the extremal lines, which are geometric invariant 3D curves, as the intersection of two implicit surfaces: the extremal points are the intersection of the extremal lines with a third implicit surface. We present an application of the extremal points extraction to the fully automatic registration of two 3D images of the same patient, taken in two different positions, to show the accuracy and robustness of the extracted extremal points.     

Animation of deformable models using implicit surfaces

The paper presents a general approach for designing and animating complex deformable models with implicit surfaces. Implicit surfaces are introduced as an extra layer coating any kind of structure that moves and deforms over time. Offering a compact definition of a smooth surface around an object, they provide an efficient collision detection mechanism. The implicit layer deforms in order to generate exact contact surfaces between colliding bodies. A simple physically based model approximating elastic behavior is then used for computing collision response. The implicit formulation also eases the control of the object's volume with a new method based on local controllers. We present two different applications that illustrate the benefits of these techniques. First, the animation of simple characters made of articulated skeletons coated with implicit flesh exploits the compactness and enhanced control of the model. The second builds on the specific properties of implicit surfaces for modeling soft inelastic substances capable of separation and fusion that maintain a constant volume when animated     

A new method for machining feature extracting of objects using 2D technical drawings

In this study, a new method called “3-Space” has been developed to obtain 3D models of prismatic machine parts using 2D technical drawings. This method uses 3 surfaces named Base, Top and Sweep Base for the determination of the objects. The edge colors have been given to the surfaces with meanings, and are used for the determination of the type and characteristics of the objects. 4 characteristics of the processing objects (namely external access direction, exit status, boundary geometry and the processing order of the features) have been determined. First, the object is considered a prismatic raw material, and the features to be processed on it are removed. Then the remaining object itself is created and thus a 3D view of the object is obtained.     

Motion estimation of elastic articulated objects from points and contours with volume invariable constraint

This paper presents a model of elastic articulated objects based on revolving conic surface and a method of model-based motion estimation. The model includes 3D object skeleton and deformable surfaces that can represent the deformation of human body surfaces. In each limb, surface deformation is represented by adjusting one or two deformation parameters. Then, the 3D deformation parameters are determined by corresponding 2D image points and contours with volume invariable constraint from a sequence of stereo images. The 3D motion parameters are estimated based on the 3D model. The algorithm presented in this paper includes model-based parameter estimation of motion and parameter determination of deformable surfaces.     

3D Object Tracking with Adaptively Weighted Local Bundles

The 3D object tracking from a monocular RGB image is a challenging task.Although popular color and edge-based methods have been well studied,they are only applicable to certain cases and new solutions to the challenges in real environment must be developed.In this paper,we propose a robust 3D object tracking method with adaptively weighted local bundles called AWLB tracker to handle more complicated cases.Each bundle represents a local region containing a set of local features.To alleviate the negative effect of the features in low-confidence regions,the bundles are adaptively weighted using a spatially-variant weighting function based on the confidence values of the involved energy terms.Therefore,in each frame,the weights of the energy items in each bundle are adapted to different situations and different regions of the same frame.Experiments show that the proposed method can improve the overall accuracy in challenging cases.We then verify the effectiveness of the proposed confidence-based adaptive weighting method using ablation studies and show that the proposed method overperforms the existing single-feature methods and multi-feature methods without adaptive weighting.