A volumetric fusion technique for surface reconstruction from silhouettes and range data

Optical triangulation, an active reconstruction technique, is known to be an accurate method but has several shortcomings due to occlusion and laser reflectance properties of the object surface, that often lead to holes and inaccuracies on the recovered surface. Shape from silhouette, on the other hand, as a passive reconstruction technique, yields robust, hole-free reconstruction of the visual hull of the object. In this paper, a hybrid surface reconstruction method that fuses geometrical information acquired from silhouette images and optical triangulation is presented. Our motivation is to recover the geometry from silhouettes on those parts of the surface which the range data fail to capture. A volumetric octree representation is first obtained from the silhouette images and then carved by range points to amend the missing cavity information. An isolevel value on each surface cube of the carved octree structure is accumulated using local surface triangulations obtained separately from range data and silhouettes. The marching cubes algorithm is then applied for triangulation of the volumetric representation. The performance of the proposed technique is demonstrated on several real objects.     

Motion-based mesh segmentation using augmented silhouettes

Motion-based segmentation, the problem of detecting rigid parts of an articulated three-dimensional shape, is an open challenge that has several applications in mesh animation, compression, and interpolation. We present a novel approach that uses the visual perception of the shape and its motion to distinguish the rigid from the deformable parts of the object. Using two-dimensional projections of the different shape poses with respect to a number of different view points, we derive a set of one-dimensional curves, which form a superset of the mesh silhouettes. Analysing these augmented silhouettes, we identify the vertices of the mesh that correspond to the deformable parts, and a subsequent clustering approach, which is based on the diffusion distance, yields a motion-based segmentation of the shape.     

3D reconstruction using silhouettes from unordered viewpoints

In this paper, we present a novel approach for reconstructing an object surface from its silhouettes. The proposed approach directly estimates the differential structure of the surface, and results in a higher accuracy than existing volumetric approaches for object reconstruction. Compared with other existing differential approaches, our approach produces relatively complete 3D models similar to volumetric approaches, with the topology conforming to what is observed from the silhouettes. In addition, the method neither assumes nor depends on the spatial order of viewpoints. Experimental results on both synthetic and real world data are presented, and comparison is made with other existing approaches to demonstrate the superiority of the proposed approach.     

Robust deformation capture from temporal range data for surface rendering

Imagine an object such as a paper sheet being waved in front of some sensor. Reconstructing the time‐varying 3D shape of the object finds direct applications in computer animation. The goal of this paper is to provide such a deformation capture system for surfaces. It uses temporal range data obtained by sensors such as those based on structured light or stereo. So as to deal with many different kinds of material, we do not make the usual assumption that the object surface has textural information. This rules out those techniques based on detecting and matching keypoints or directly minimizing color discrepancy. The proposed method is based on a planar mesh that is deformed so as to fit each of the range images. We show how to achieve this by minimizing a compound cost function combining several data and regularization terms, needed to make the overall system robust so that it can deal with low quality datasets. Carefully examining the parameter to residual relationship shows that this cost function can be minimized very efficiently by coupling nonlinear least squares methods with sparse matrix operators. Experimental results for challenging datasets coming from different kinds of range sensors are reported. The algorithm is reasonably fast and is shown to be robust to missing and erroneous data points. Copyright © 2008 John Wiley & Sons, Ltd.     

Coarse-to-fine multiview 3d face reconstruction using multiple geometrical features

Multimedia Tools and Applications - 3D face reconstruction from multi-view video sequences has become a hotspot in computer vision for the last decades. Structure from Motion (SfM) methods, which...     

Robust edge-preserving surface mesh polycube deformation

Computational Visual Media - Polycube construction and deformation are essential problems in computer graphics. In this paper, we present a robust, simple, efficient, and automatic algorithm to...     

Three-dimensional human shape inference from silhouettes: reconstruction and validation

Silhouettes are robust image features that provide considerable evidence about the three-dimensional (3D) shape of a human body. The information they provide is, however, incomplete and prior knowledge has to be integrated to reconstruction algorithms in order to obtain realistic body models. This paper presents a method that integrates both geometric and statistical priors to reconstruct the shape of a subject assuming a standardized posture from a frontal and a lateral silhouette. The method is comprised of three successive steps. First, a non-linear function that connects the silhouette appearances and the body shapes is learnt and used to create a first approximation. Then, the body shape is deformed globally along the principal directions of the population (obtained by performing principal component analysis over 359 subjects) to follow the contours of the silhouettes. Finally, the body shape is deformed locally to ensure it fits the input silhouettes as well as possible. Experimental results showed a mean absolute 3D error of 8 mm with ideal silhouettes extraction. Furthermore, experiments on body measurements (circumferences or distances between two points on the body) resulted in a mean error of 11 mm.     

从表面重构的体数据实现3维网格剖分

目的 针对有限元分析中网格最优化问题,提出一种改进的生成四面体网格的自组织算法。方法 该算法首先应用几何方法将三角形表面模型重新构造成规定大小的分类体数据,同时由该表面模型建立平衡八叉树,计算用以控制网格尺寸的3维数组;然后将体数据转换成邻域内不同等值面的形态一致的边界指示数组;结合改进的自组织算法和相关3维数据的插值函数,达到生成四面体网格的目的。结果 实验结果对比表明,该方法能够生成更高比例的优质四面体,增强了对扁平面体的抑制能力,同时很好地保证了边界的一致。结论 在对封闭的3维表面网格进行有限元建模时,本文算法为其提供了一种有效、可靠的途径。     

Invariant surface and motion estimation from sparse range data

In this paper, we present a system for the estimation of the surface structure and the motion parameters of a free-flying object in a tele-robotics experiment. The system consists of two main components: (i) a vision-based invariant-surface and motion estimator and (ii) a Kalman filter state estimator. We present a new algorithm for motion estimation from sparse multi-sensor range data. The motion estimates from the vision-based estimator are input to a Kalman filter state estimator for continuously tracking a free-flying object in space under zero-gravity conditions. The predicted position and orientation parameters are then fed back to the vision module of the system and serve as an initial guess in the search for optimal motion parameters. The task of the vision module is two-fold: (i) estimating a piecewise-smooth surface from a single frame of multi-sensor data and (ii) determining the most likely (in the Bayesian sense) object motion that makes data in subsequent time frames to have been sampled from the same piecewise-smooth surface. With each incoming data frame, the piecewise-smooth surface is incrementally refined. The problem is formulated as an energy minimization and solved numerically resulting in a surface estimate invariant to 3D rigid motion and the vector of motion parameters. Performance of the system is depicted on simulated and real range data.     

从表面重构的体数据实现三维网格剖分

针对有限元分析中网格最优化问题,本文提出一种改进的生成四面体网格的自组织算法。该算法首先应用几何方法将三角形表面模型重新构造成规定大小的分类体数据,同时由该表面模型建立平衡八叉树,计算用以控制网格尺寸的三维数组;然后将体数据转换成邻域内不同等值面的形态一致的边界指示数组;结合改进的自组织算法和相关三维数据的插值函数,达到生成四面体网格的目的。实验对比表明,该方法能够生成更高比例的优质四面体,同时很好地保证了边界的一致。在对封闭的三维表面网格进行有限元建模时,本文算法为其提供了一种有效、可靠的途径。     

Image reconstruction from limited range projections using orthogonal moments

A set of orthonormal polynomials is proposed for image reconstruction from projection data. The relationship between the projection moments and image moments is discussed in detail, and some interesting properties are demonstrated. Simulation results are provided to validate the method and to compare its performance with previous works.     

Material-aware differential mesh deformation using sketching interface

In this paper, we present a material-aware mesh deformation method using a sketching interface. Guided by user-specified material properties, our method can deform the surface mesh in a non-uniform way, while previous deformation techniques are mainly designed for uniform materials. The non-uniform deformation is achieved by material-dependent gradient field manipulation and Poisson-based reconstruction. Compared with previous material-oblivious deformation techniques, our method supplies better control of the deformation process and can generate more realistic results. We propose a novel detail representation that transforms geometric details between successive surface levels as a combination of dihedral angles and barycentric coordinates. This detail representation is similarity-invariant and fully compatible with material properties. Based on these two methods, we implement a multi-resolution deformation tool, allowing the user to edit a mesh inside a hierarchy in a material-aware manner. We demonstrate the effectiveness and robustness of our methods by several examples with real-world data.     

Interactive mesh deformation using equality-constrained least squares

Mesh deformation techniques that preserve the differential properties have been intensively studied. In this paper, we propose an equality-constrained least squares approach for stably deforming mesh models while approximately preserving mean curvature normals and strictly satisfying other constraints such as positional constraints. We solve the combination of hard and soft constraints by constructing a typical least squares system using QR decomposition. A well-known problem of hard constraints is over-constraints. We show that the equality-constrained least squares approach is useful for resolving such over-constrained situations. In our framework, the rotations of mean curvature normals are treated using the logarithms of unit quaternions in . During deformation, mean curvature normals can be rotated while preserving their magnitudes. In addition, we introduce a new modeling constraints called rigidity constraints and show that rigidity constraints can effectively preserve the shapes of feature regions during deformation. Our framework achieves good performance for interactive deformation of mesh models.     

Quasi-angle-preserving mesh deformation using the least-squares approach

We propose an angle-based mesh representation, which is invariant under translation, rotation, and uniform scaling, to encode the geometric details of a triangular mesh. Angle-based mesh representation consists of angle quantities defined on the mesh, from which the mesh can be reconstructed uniquely up to translation, rotation, and uniform scaling. The reconstruction process requires solving three sparse linear systems： the first system encodes the length of edges between vertices on the mesh, the second system encodes the relationship of local frames between two adjacent vertices on the mesh, and the third system defines the position of the vertices via the edge length and the local frames. From this angle-based mesh representation, we propose a quasi-angle-preserving mesh deformation system with the least-squares approach via detail-preserving mesh editing examples are presented to handle translation, rotation, and uniform scaling. Several demonstrate the effectiveness of the proposed method.     

Detection and classification of vehicles from omnidirectional videos using multiple silhouettes

To detect and classify vehicles in omnidirectional videos, we propose an approach based on the shape (silhouette) of the moving object obtained by background subtraction. Different from other shape-based classification techniques, we exploit the information available in multiple frames of the video. We investigated two different approaches for this purpose. One is combining silhouettes extracted from a sequence of frames to create an average silhouette, the other is making individual decisions for all frames and use consensus of these decisions. Using multiple frames eliminates most of the wrong decisions which are caused by a poorly extracted silhouette from a single video frame. The vehicle types we classify are motorcycle, car (sedan) and van (minibus). The features extracted from the silhouettes are convexity, elongation, rectangularity and Hu moments. We applied two separate methods of classification. First one is a flowchart-based method that we developed and the second is K-nearest neighbour classification. 60% of the samples in the dataset are used for training. To ensure randomization in the experiments, threefold cross-validation is applied. The results indicate that using multiple silhouettes increases the classification performance.     

Manifold surface reconstruction of an environment from sparse Structure-from-Motion data

The majority of methods for the automatic surface reconstruction of an environment from an image sequence have two steps: Structure-from-Motion and dense stereo. From the computational standpoint, it would be interesting to avoid dense stereo and to generate a surface directly from the sparse cloud of 3D points and their visibility information provided by Structure-from-Motion. The previous attempts to solve this problem are currently very limited: the surface is non-manifold or has zero genus, the experiments are done on small scenes or objects using a few dozens of images. Our solution does not have these limitations. Furthermore, we experiment with hand-held or helmet-held catadioptric cameras moving in a city and generate 3D models such that the camera trajectory can be longer than one kilometer.     

Textured mesh surface reconstruction of large buildings with multi-view stereo

There are three main approaches for reconstructing 3D models of buildings. Laser scanning is accurate but expensive and limited by the laser’s range. Structure-from-motion (SfM) and multi-view stereo (MVS) recover 3D point clouds from multiple views of a building. MVS methods, especially patch-based MVS, can achieve higher density than do SfM methods. Sophisticated algorithms need to be applied to the point clouds to construct mesh surfaces. The recovered point clouds can be sparse in areas that lack features for accurate reconstruction, making recovery of complete surfaces difficult. Moreover, segmentation of the building’s surfaces from surrounding surfaces almost always requires some form of manual inputs, diminishing the ease of practical application of automatic 3D reconstruction algorithms. This paper presents an alternative approach for reconstructing textured mesh surfaces from point cloud recovered by patch-based MVS method. To a good first approximation, a building’s surfaces can be modeled by planes or curve surfaces which are fitted to the point cloud. 3D points are resampled on the fitted surfaces in an orderly pattern, whose colors are obtained from the input images. This approach is simple, inexpensive, and effective for reconstructing textured mesh surfaces of large buildings. Test results show that the reconstructed 3D models are sufficiently accurate and realistic for 3D visualization in various applications.     

Capturing tree crown formation through implicit surface reconstruction using airborne lidar data

Forest structure data derived from lidar is being used in forest science and management for inventory analysis, biomass estimation, and wildlife habitat analysis. Regression analysis dominated previous approaches to the derivation of tree stem and crown parameters from lidar. The regression model for tree parameters is locally applied based on vertical lidar point density, the tree species involved, and stand structure in the specific research area. The results of this approach, therefore, are location-specific, limiting its applicability to other areas. For a more widely applicable approach to derive tree parameters, we developed an innovative method called ‘wrapped surface reconstruction’ that employs radial basis functions and an isosurface. Utilizing computer graphics, we capture the exact shape of an irregular tree crown of various tree species based on the lidar point cloud and visualize their exact crown formation in three-dimensional space. To validate the tree parameters given by our wrapped surface approach, survey-grade equipment (a total station) was used to measure the crown shape. Four vantage points were established for each of 55 trees to capture whole-tree crown profiles georeferenced with post-processed differential GPS points. The observed tree profiles were linearly interpolated to estimate crown volume. These fieldwork-generated profiles were compared with the wrapped surface to assess goodness of fit. For coniferous trees, the following tree crown parameters derived by the wrapped surface method were highly correlated (p < 0.05) with the total station-derived measurements: tree height (R² = 0.95), crown width (R² = 0.80), live crown base (R² = 0.92), height of the lowest branch (R² = 0.72), and crown volume (R² = 0.84). For deciduous trees, wrapped surface-derived parameters of tree height (R² = 0.96), crown width (R² = 0.75), live crown base (R² = 0.53), height of the lowest branch (R² = 0.51), and crown volume (R² = 0.89) were correlated with the total station-derived measurements. The wrapped surface technique is less susceptible to errors in estimation of tree parameters because of exact interpolation using the radial basis functions. The effect of diminished energy return causes the low correlation for lowest branches in deciduous trees (R² = 0.51), even though leaf-off lidar data was used. The wrapped surface provides fast and automated detection of micro-scale tree parameters for specific applications in areas such as tree physiology, fire modeling, and forest inventory.     

Optimized triangle mesh reconstruction from unstructured points

A variety of approaches have been proposed for polygon mesh reconstruction from a set of unstructured sample points. Suffering from severe aliases at sharp features and having a large number of unnecessary faces, most resulting meshes need to be optimized using input sample points in a postprocess. In this paper, we propose a fast algorithm to reconstruct high-quality meshes from sample data. The core of our proposed algorithm is a new mesh evaluation criterion which takes full advantage of the relation between the sample points and the reconstructed mesh. Based on our proposed evaluation criterion, we develop necessary operations to efficiently incorporate the functions of data preprocessing, isosurface polygonization, mesh optimization and mesh simplification into one simple algorithm, which can generate high-quality meshes from unstructured point clouds with time and space efficiency. Published online: 28 January 2003 Correspondence to: Y.-J. Liu