Bayesian 3D shape from silhouettes

This paper introduces a smooth posterior density function for inferring shapes from silhouettes. Both the likelihood and the prior are modelled using kernel density functions and optimisation is performed using gradient ascent algorithms. Adding a prior allows for the recovery of concave areas of the shape that are usually lost when estimating the visual hull. This framework is also extended to use colour information when it is available in addition to the silhouettes. In these cases, the modelling not only allows for the shape to be recovered but also its colour information. Our new algorithms are assessed by reconstructing 2D shapes from 1D silhouettes and 3D faces from 2D silhouettes. Experimental results show that using the prior can assist in reconstructing concave areas and also illustrate the benefits of using colour information even when only small numbers of silhouettes are available.     

An octree-based method for shape from inconsistent silhouettes

Shape-from-Silhouette (SfS) is the widely known problem of obtaining the 3D structure of an object from its silhouettes. Two main approaches can be employed: those based on voxel sets, which perform an exhaustive search of the working space, and those based on octrees, which perform a top-down analysis that speeds up the computation. The main problem of both approaches is the need for perfect silhouettes to obtain accurate results. Perfect background subtraction hardly ever happens in realistic scenarios, so these techniques are restricted to controlled environments where the consistency hypothesis can be assumed. Recently, some approaches (all of them based on voxel sets) have been proposed to solve the problem of inconsistency. Their main drawback is the high computational cost required to perform an exhaustive analysis of the working space. This paper proposes a novel approach to solve SfS with inconsistent silhouettes from an octree based perspective. The inconsistencies are dealt by means of the Dempster–Shafer (DS) theory and we employ a Butterworth function for adapting threshold values in each resolution level of the octree. The results obtained show that our proposal provides higher reconstruction quality than the standard octree based methods in realistic environments. When compared to voxel set approaches that manage inconsistency, our method obtains similar results with a reduction in the computing time of an order of magnitude.     

3D reconstruction using silhouettes from unordered viewpoints

In this paper, we present a novel approach for reconstructing an object surface from its silhouettes. The proposed approach directly estimates the differential structure of the surface, and results in a higher accuracy than existing volumetric approaches for object reconstruction. Compared with other existing differential approaches, our approach produces relatively complete 3D models similar to volumetric approaches, with the topology conforming to what is observed from the silhouettes. In addition, the method neither assumes nor depends on the spatial order of viewpoints. Experimental results on both synthetic and real world data are presented, and comparison is made with other existing approaches to demonstrate the superiority of the proposed approach.     

Three-dimensional shape reconstruction of abdominal aortic aneurysm

This paper presents an effective computational technique for reconstructing a three-dimensional shape of an abdominal aortic aneurysm (AAA), from a limited number of computed tomography (CT) images. The three-dimensional template geometry of a healthy abdominal aorta is used as a priori knowledge, and the template geometry is deformed by extended free-form deformation (EFFD), to generate a patient-specific AAA geometry. A two-step optimization scheme is devised to find an optimal set of EFFD parameters that match the cross-section of a deformed template with an AAA contour shown in a CT image. The geometric continuity of a deformed model is maintained by raising the order of the polynomial function used in EFFD. Experimental results show that the proposed method creates the three-dimensional shape of AAA suitable for structural finite element analysis and computational fluid dynamics for medical diagnosis.     

Model-based human shape reconstruction from multiple views

Image-based modelling allows the reconstruction of highly realistic digital models from real-world objects. This paper presents a model-based approach to recover animated models of people from multiple view video images. Two contributions are made, a multiple resolution model-based framework is introduced that combines multiple visual cues in reconstruction. Second, a novel mesh parameterisation is presented to preserve the vertex parameterisation in the model for animation. A prior humanoid surface model is first decomposed into multiple levels of detail and represented as a hierarchical deformable model for image fitting. A novel mesh parameterisation is presented that allows propagation of deformation in the model hierarchy and regularisation of surface deformation to preserve vertex parameterisation and animation structure. The hierarchical model is then used to fuse multiple shape cues from silhouette, stereo and sparse feature data in a coarse-to-fine strategy to recover a model that reproduces the appearance in the images. The framework is compared to physics-based deformable surface fitting at a single resolution, demonstrating an improved reconstruction accuracy against ground-truth data with a reduced model distortion. Results demonstrate realistic modelling of real people with accurate shape and appearance while preserving model structure for use in animation.     

Coarse-to-fine surface reconstruction from silhouettes and range data using mesh deformation

We present a coarse-to-fine surface reconstruction method based on mesh deformation to build watertight surface models of complex objects from their silhouettes and range data. The deformable mesh, which initially represents the object visual hull, is iteratively displaced towards the triangulated range surface using the line-of-sight information. Each iteration of the deformation algorithm involves smoothing and restructuring operations to regularize the surface evolution process. We define a non-shrinking and easy-to-compute smoothing operator that fairs the surface separately along its tangential and normal directions. The mesh restructuring operator, which is based on edge split, collapse and flip operations, enables the deformable mesh to adapt its shape to the object geometry without suffering from any geometrical distortions. By imposing appropriate minimum and maximum edge length constraints, the deformable mesh, hence the object surface, can be represented at increasing levels of detail. This coarse-to-fine strategy, that allows high resolution reconstructions even with deficient and irregularly sampled range data, not only provides robustness, but also significantly improves the computational efficiency of the deformation process. We demonstrate the performance of the proposed method on several real objects.     

A volumetric fusion technique for surface reconstruction from silhouettes and range data

Optical triangulation, an active reconstruction technique, is known to be an accurate method but has several shortcomings due to occlusion and laser reflectance properties of the object surface, that often lead to holes and inaccuracies on the recovered surface. Shape from silhouette, on the other hand, as a passive reconstruction technique, yields robust, hole-free reconstruction of the visual hull of the object. In this paper, a hybrid surface reconstruction method that fuses geometrical information acquired from silhouette images and optical triangulation is presented. Our motivation is to recover the geometry from silhouettes on those parts of the surface which the range data fail to capture. A volumetric octree representation is first obtained from the silhouette images and then carved by range points to amend the missing cavity information. An isolevel value on each surface cube of the carved octree structure is accumulated using local surface triangulations obtained separately from range data and silhouettes. The marching cubes algorithm is then applied for triangulation of the volumetric representation. The performance of the proposed technique is demonstrated on several real objects.     

Three-dimensional shape from color photometric stereo

Computer vision systems can be used to determine the shapes of real three-dimensional objects for purposes of object recognition and pose estimation or for CAD applications. One method that has been developed is photometric stereo. This method uses several images taken from the same viewpoint, but with different lightings, to determine the three-dimensional shape of an object. Most previous work in photometric stereo has been with gray-tone images; color images have only been used for dielectric materials. In this paper we describe a procedure for color photometric stereo, which recovers the shape of a colored object from two or more color images of the object under white illumination. This method can handle different types of materials, such as composites and metals, and can employ various reflection models such as the Lambertian, dichromatic, and Torrance-Sparrow models. For composite materials, colored metals, and dielectrics, there are two advantages of utilizing color information: at each pixel, there are more constraints on the orientation, and the result is less sensitive to noise. Consequently, the shape can be found more accurately. The method has been tested on both artificial and real images of objects of various materials, and on real images of a multi-colored object.     

Three-dimensional object reconstruction from orthogonal projections

Techniques for the reconstruction of three-dimensional objects from orthogonal projections are described. When two projections are available, the object is divided into thin slices and the slices are individually reconstructed. When three projections are available, heuristic reconstruction techniques are discussed.     

Free-form object reconstruction from silhouettes, occluding edges and texture edges: a unified and robust operator based on duality

In this paper, the duality in differential form is developed between a 3D primal surface and its dual manifold formed by the surface's tangent planes, i.e., each tangent plane of the primal surface is represented as a four-dimensional vector which constitutes a point on the dual manifold. The iterated dual theorem shows that each tangent plane of the dual manifold corresponds to a point on the original 3D surface, i.e., the dual of the dual goes back to the primal. This theorem can be directly used to reconstruct 3D surface from image edges by estimating the dual manifold from these edges. In this paper we further develop the work in our original conference papers resulting in the robust differential dual operator. We argue that the operator makes good use of the information available in the image data, by using both points of intensity discontinuity and their edge directions; we provide a simple physical interpretation of what the abstract algorithm is actually estimating and why it makes sense in terms of estimation accuracy; our algorithm operates on all edges in the images, including silhouette edges, self occlusion edges, and texture edges, without distinguishing their types (thus resulting in improved accuracy and handling locally concave surface estimation if texture edges are present); the algorithm automatically handles various degeneracies; and the algorithm incorporates new methodologies for implementing the required operations such as appropriately relating edges in pairs of images, evaluating and using the algorithm's sensitivity to noise to determine the accuracy of an estimated 3D point. Experiments with both synthetic and real images demonstrate that the operator is accurate, robust to degeneracies and noise, and general for reconstructing free-form objects from occluding edges and texture edges detected in calibrated images or video sequences.     

Curvature-augmented tensor voting for shape inference from noisy 3Ddata

Improves the basic tensor voting formalism to infer the sign and direction of principal curvatures at each input site from noisy 3D data. Unlike most previous approaches, no local surface fitting, partial derivative computation, nor oriented normal vector recovery is performed in our method. These approaches are known to be noise-sensitive, since accurate partial derivative information is often required, which is usually unavailable from real data. Also, unlike approaches that detect signs of Gaussian curvature, we can handle points with zero Gaussian curvature uniformly, without first localizing them in a separate process. The tensor-voting curvature estimation is non-iterative, does not require initialization, and is robust to a considerable amount of outlier noise, as its effect is reduced by collecting a large number of tensor votes. Qualitative and quantitative results on synthetic and real complex data are presented     

Three-dimensional reconstruction of geoscientific objects from serial sections

Serial section reconstruction is widely used for visualising complex three-dimensional objects, but little research has been applied to modelling geoscientific applications. We review previous work and highlight the correspondece problem, particularly, important in reconstructions from geoscientific data. We propose an automatic solution to the correspondence problem, based on a minimum-spanningtree algoithm. The improved results stem from the use of topological information to help decide which edges appear in the final correspondence graph. We then reconstruct some invertebrate fossil samples, before outlining future possibilities in deriving solutions for complex samples, using richer information for earch specimen.     

Three-dimensional motion and reconstruction of coronary arteries from biplane cineangiography

A new approach is described for reconstructing coronary arteries from two sequences of projection images. The estimation of motion is performed on three-dimensional line segments (or centrelines), and is based on a ‘predictionprojection-optimization’ loop. The method copes with time varying properties, deformations and superpositions of vessels. Experiments using simulated and real data have been carried out. and the results found to be robust over a full cycle of a human heart. Local and global kinetic features can then be derived to obtain a greater insight on the cardiac functional state     

On the intrinsic reconstruction of shape from its symmetries

The main question we address is: What is the minimal information required to generate closed, nonintersecting planar boundaries? For this paper, we restrict "shape" to this meaning. More precisely, we examine whether the medial axis, together with dynamics, can serve as a language to design shapes and to effect shape changes. We represent the medial axis together with a direction of flow along the axis as the shock graph and examine the reconstruction of shape along each of the three types of medial axis points, A/sub 1//sup 2/, A/sub 1//sup 3/, A/sub 3/, and the associated six types of shock points. First, we show that the tangent and curvature of the medial axis and the speed and acceleration of the shock with respect to time of propagation are sufficient to determine the boundary tangent and curvature at corresponding points of the boundary. This implies that a rather coarse sampling of the symmetry axis, its tangent, curvature, speed, and acceleration is sufficient to regenerate accurately a local neighborhood of shape at regular axis points (A/sub 1//sup 2/). Second, we examine the reconstruction of shape at branch points (A/sub 1//sup 3/) where three regular branches are joined. We show that the three pairs of geometry (that is, curvature) and dynamics (that is, acceleration) must satisfy certain constraints. Finally, we derive similar results for the end points of shock branches (A/sub 3/ points). These formulas completely specify the local reconstruction of a shape from its shock-graph or medial axis and the conditions required to form a coherent shape from the medial axis.     

Robust deformable shape reconstruction from monocular video with manifold forests

Existing approaches to recover structure of 3D deformable objects and camera motion parameters from an uncalibrated images assume the object’s shape could be modelled well by a linear subspace. These methods have been proven effective and well suited when the deformations are relatively small, but fail to reconstruct the objects with relatively large deformations. This paper describes a novel approach for 3D non-rigid shape reconstruction, based on manifold decision forest technique. The use of this technique can be justified by noting that a specific type of shape variations might be governed by only a small number of parameters, and therefore can be well represented in a low-dimensional manifold. The key contributions of this work are the use of random decision forests for the shape manifold learning and robust metric for calculation of the re-projection error. The learned manifold defines constraints imposed on the reconstructed shapes. Due to a nonlinear structure of the learned manifold, this approach is more suitable to deal with large and complex object deformations when compared to the linear constraints. The robust metric is applied to reduce the effect of measurement outliers on the quality of the reconstruction. In many practical applications outliers cannot be completely removed and therefore the use of robust techniques is of particular practical interest. The proposed method is validated on 2D points sequences projected from the 3D motion capture data for ground truth comparison and also on real 2D video sequences. Experiments show that the newly proposed method provides better performance compared to previously proposed ones, including the robustness with respect to measurement noise, missing measurements and outliers present in the data.