Incorporating object-centered sampling and Delaunay tetrahedrization for visual hull reconstruction

In this paper we present a novel shape from silhouette algorithm. For an object to be modeled, the algorithm first computes a cloud of points located on a pencil of rays and distributed evenly on the visual hull surface, inside and outside the visual hull. Then Delaunay tetrahedrization is applied to the point cloud to partition its convex hull into a set of tetrahedrons. Finally, outlier tetrahedrons are removed by tetrahedron peeling, and a mesh model of the visual hull is extracted. The algorithm is robust, free from discretization artifacts, and produces a mesh model composed of well-shaped triangles.     

The visual hull concept for silhouette-based image understanding

Many algorithms for both identifying and reconstructing a 3-D object are based on the 2-D silhouettes of the object. In general, identifying a nonconvex object using a silhouette-based approach implies neglecting some features of its surface as identification clues. The same features cannot be reconstructed by volume intersection techniques using multiple silhouettes of the object. This paper addresses the problem of finding which parts of a nonconvex object are relevant for silhouette-based image understanding. For this purpose, the geometric concept of visual hull of a 3-D object is introduced. This is the closest approximation of object S that can be obtained with the volume intersection approach; it is the maximal object silhouette-equivalent to S, i.e., which can be substituted for S without affecting any silhouette. Only the parts of the surface of S that also lie on the surface of the visual hull can be reconstructed or identified using silhouette-based algorithms. The visual hull depends not only on the object but also on the region allowed to the viewpoint. Two main viewing regions result in the external and internal visual hull. In the former case the viewing region is related to the convex hull of S, in the latter it is bounded by S. The internal visual hull also admits an interpretation not related to silhouettes. Algorithms for computing visual hulls are presented and their complexity analyzed. In general, the visual hull of a 3-D planar face object turns out to be bounded by planar and curved patches     

Reduced Depth and Visual Hulls of Complex 3D Scenes

Depth and visual hulls are useful for quick reconstruction and rendering of a 3D object based on a number of reference views. However, for many scenes, especially multi‐object, these hulls may contain significant artifacts known as phantom geometry. In depth hulls the phantom geometry appears behind the scene objects in regions occluded from all the reference views. In visual hulls the phantom geometry may also appear in front of the objects because there is not enough information to unambiguously imply the object positions. In this work we identify which parts of the depth and visual hull might constitute phantom geometry. We define the notion of reduced depth hull and reduced visual hull as the parts of the corresponding hull that are phantom‐free. We analyze the role of the depth information in identification of the phantom geometry. Based on this, we provide an algorithm for rendering the reduced depth hull at interactive frame‐rates and suggest an approach for rendering the reduced visual hull. The rendering algorithms take advantage of modern GPU programming techniques. Our techniques bypass explicit reconstruction of the hulls, rendering the reduced depth or visual hull directly from the reference views.     

Enclosing Surfaces for Point Clusters Using 3D Discrete Voronoi Diagrams

Point clusters occur in both spatial and non-spatial data. In the former context they may represent segmented particle data, in the latter context they may represent clusters in scatterplots. In order to visualize such point clusters, enclosing surfaces lead to much better comprehension than pure point renderings.
We propose a flexible system for the generation of enclosing surfaces for 3D point clusters. We developed a GPU-based 3D discrete Voronoi diagram computation that supports all surface extractions. Our system provides three different types of enclosing surfaces. By generating a discrete distance field to the point cluster and extracting an isosurface from the field, an enclosing surface with any distance to the point cluster can be generated. As a second type of enclosing surfaces, a hull of the point cluster is extracted. The generation of the hull uses a projection of the discrete Voronoi diagram of the point cluster to an isosurface to generate a polygonal surface. Generated hulls of non-convex clusters are also non-convex. The third type of enclosing surfaces can be created by computing a distance field to the hull and extracting an isosurface from the distance field. This method exhibits reduced bumpiness and can extract surfaces arbitrarily close to the point cluster without losing connectedness.
We apply our methods to the visualization of multidimensional spatial and non-spatial data. Multidimensional clusters are extracted and projected into a 3D visual space, where the point clusters are visualized. The respective clusters can also be visualized in object space when dealing with multidimensional particle data.     

基于图像轮廓的三维重建方法

根据平面镜成像原理,提出了一种基于单幅图像的侧影轮廓线可见外壳重建方法。利用成角度平面镜装置模拟实现多相机同时拍摄,对图像中各轮廓线之间极线几何关系的分析得到对应的相机参数,实现目标物体的三维重建。实验结果表明,该方法简单有效,不需要通过实验室条件下的特殊仪器进行标定即可实现目标物体的三维重建,具有一定的实用价值。     

Volumetric stereo and silhouette fusion for image-based modeling

This paper presents a volumetric stereo and silhouette fusion algorithm for acquiring high quality models from multiple calibrated photographs. Our method is based on computing and merging depth maps. Different from previous methods of this category, the silhouette information is also applied in our algorithm to recover the shape information on the textureless and occluded areas. The proposed algorithm starts by computing visual hull using a volumetric method in which a novel projection test method is proposed for visual hull octree construction. Then, the depth map of each image is estimated by an expansion-based approach that returns a 3D point cloud with outliers and redundant information. After generating an oriented point cloud from stereo by rejecting outlier, reducing scale, and estimating surface normal for the depth maps, another oriented point cloud from silhouette is added by carving the visual hull octree structure using the point cloud from stereo to restore the textureless and occluded surfaces. Finally, Poisson Surface Reconstruction approach is applied to convert the oriented point cloud both from stereo and silhouette into a complete and accurate triangulated mesh model. The proposed approach has been implemented and the performance of the approach is demonstrated on several real data sets, along with qualitative comparisons with the state-of-the-art image-based modeling techniques according to the Middlebury benchmark.     

联合LBS和Snake的3D人体外形和运动跟踪方法

为了解决基于多目视频轮廓信息的3D人体外形和运动跟踪问题,提出一种联合线性混合蒙皮和Snake变形模型的算法框架.首先建立人物对象的蒙皮模型,以每一帧多目同步视频的轮廓作为输入,采用一种基于剪影轮廓的可视外壳重建算法,使得作为3D特征的可视外壳保持了局部细节且更加光滑;并使用关节型迭代最近点算法进行匹配以捕获出每一帧骨架子空间下的人物3D外形及运动;再一次使用当前帧的多目轮廓信息,让Snake内外力共同作用于人物网格模型上的顶点,使之自由地趋近于目标对象.使用带ground-truth的合成数据进行对比实验的结果表明,该方法因同时使用3D误差约束和2D误差约束,提高了跟踪精度.     

The Theory and Practice of Coplanar Shadowgram Imaging for Acquiring Visual Hulls of Intricate Objects

Acquiring 3D models of intricate objects (like tree branches, bicycles and insects) is a challenging task due to severe self-occlusions, repeated thin structures, and surface discontinuities. In theory, a shape-from-silhouettes (SFS) approach can overcome these difficulties and reconstruct visual hulls that are close to the actual shapes, regardless of the complexity of the object. In practice, however, SFS is highly sensitive to errors in silhouette contours and the calibration of the imaging system, and has therefore not been used for obtaining accurate shapes with a large number of views. In this work, we present a practical approach to SFS using a novel technique called coplanar shadowgram imaging that allows us to use dozens to even hundreds of views for visual hull reconstruction. A point light source is moved around an object and the shadows (silhouettes) cast onto a single background plane are imaged. We characterize this imaging system in terms of image projection, reconstruction ambiguity, epipolar geometry, and shape and source recovery. The coplanarity of the shadowgrams yields unique geometric properties that are not possible in traditional multi-view camera-based imaging systems. These properties allow us to derive a robust and automatic algorithm to recover the visual hull of an object and the 3D positions of the light source simultaneously, regardless of the complexity of the object. We demonstrate the acquisition of several intricate shapes with severe occlusions and thin structures, using 50 to 120 views. Electronic Supplementary Material  The online version of this article () contains supplementary material, which is available to authorized users. This is an extension and consolidation of our previous work on coplanar shadowgram imaging system (Yamazaki et al. 2007) presented at IEEE International Conference on Computer Vision 2007.     

Multi-Camera Active-Vision for Markerless Shape Recovery of Unknown Deforming Objects

A novel multi-camera active-vision reconfiguration method is proposed for the markerless shape recovery of unknown deforming objects. The proposed method implements a model fusion technique to obtain a complete 3D mesh-model via triangulation and a visual hull. The model is tracked using an adaptive particle filtering algorithm, yielding a deformation estimate that can, then, be used to reconfigure the cameras for improved surface visibility. The objective of reconfiguration is maximization of the total surface area visible through stereo triangulation. The surface area based objective function directly relates to maximizing the accuracy of the shape recovered, as stereo triangulation is more accurate than visual hull building when the number of viewpoints is limited. The reconfiguration process comprises workspace discretization, visibility estimation, optimal stereo-pose selection, and path planning to ensure 2D tracked feature consistency. In contrast to other reconfiguration techniques that rely on a priori known, and at times static, object models, our method focuses on a priori unknown deforming objects. The proposed method operates on-line and has been shown to outperform static-camera based systems through extensive simulations and experiments with an increased surface visibility in the presence of occluding obstacles.     

多先验融合的图像显著性目标检测算法

为了更加准确地检测出图像中的显著性目标，提出了多先验融合的显著性目标检测算法。针对传统中心先验对偏离图像中心的显著性目标会出现检测失效的情况，提出在多颜色空间下求显著性目标的最小凸包交集来确定目标的大致位置，以凸包区域中心计算中心先验。同时通过融合策略将凸包区域中心先验、颜色对比先验和背景先验融合并集成到特征矩阵中。最后通过低秩矩阵恢复模型生成结果显著图。在公开数据集MSRA1000和ESSCD上的仿真实验结果表明，MPLRR能够得到清晰高亮的显著性目标视觉效果图，同时F，AUC，MAE等评价指标也比现有的许多方法有明显提升。     

Understanding the curve-polygon object

In this paper the new method of understanding of the curve-polygon object is presented. The method of understanding of the curve-polygon object is part of the research aimed at developing a shape understanding method able to perform complex visual tasks connected with visual thinking. The shape understanding method is implemented as the shape understanding system (SUS). Understanding includes, among others, obtaining the visual concept in process of the visual reasoning, naming and visual explanation by generation an object from a required class. In this paper generation of the object from the selected well defined class, the curve-polygon class, as well as assigning the visual object into one of the shape classes is presented. The generation of the visual objects is used in SUS during learning of the visual concept, explanatory process and self-correcting process. The visual object is assigned into one of the shape classes during the visual reasoning process. The visual reasoning, presented in this paper, in contrast to other forms of reasoning depends on the type of objects which are analysed. In this paper visual reasoning that assigns an object to the curve-polygon class is presented. The shape understanding system consists of different types of experts that perform different processing and reasoning tasks. The self-correcting expert, that implements the new method of testing and reasoning, is invoked to test ability of the system to understand the concept of the curve-polygon shape.     

A fast convex hull algorithm with maximum inscribed circle affine transformation

This paper presents a fast convex hull algorithm for a large point set. The algorithm imitates the procedure of human visual attention derived in a psychological experiment. The merit of human visual attention is to neglect most inner points directly. The proposed algorithm achieves a significant saving in time and space in comparison with the two best convex hull algorithms mentioned in a latest review proposed by Chadnov and Skvortsov in 2004. Furthermore, we propose to use an affine transformation to solve the narrow shape problem for computing the convex hull faster.     

基于可视外壳的cage 生成

借助cage 作为代理几何来处理高精度复杂模型,正成为计算机动画与几何造型中的一种重要方法.目前,已有的cage 生成算法尚缺乏普适性,很大程度上依赖于几何的表示和模型的复杂度.因此,提出一种基于可视外壳的cage 生成算法.通过逆向模拟计算机视觉领域可视外壳的生成过程获取cage,以实现cage 生成与几何表示和模型复杂度无关的目标.实验结果表明,该方法易于实现且效率高.     

Acquiring a complete 3D model from specular motion under theillumination of circular-shaped light sources

We recover 3D models of objects with specular surfaces. An object is rotated and its continuous images are taken. Circular-shaped light sources that generate conic rays are used to illuminate the rotating object in such a way that highlighted stripes can be observed on most of the specular surfaces. Surface shapes can be computed from the motions of highlights in the continuous images; either specular motion stereo or single specular trace mode can be used. When the lights are properly set, each point on the object can be highlighted during the rotation. The shape for each rotation plane is measured independently using its corresponding epipolar plane image. A 3D shape model is subsequently reconstructed by combining shapes at different rotation planes. Computing a shape is simple and requires only the motion of highlight on each rotation plane. The novelty of this paper is the complete modeling of a general type of specular objects that has not been accomplished before     

How far 3D shapes can be understood from 2D silhouettes

Each 2D silhouette of a 3D unknown object O constrains O inside the volume obtained by back-projecting the silhouette from the corresponding viewpoint. A set of silhouettes specifies a boundary volume R obtained by intersecting the volumes due to each silhouette. R more or less closely approximates O, depending on the viewpoints and the object itself. This approach to the reconstruction of 3D objects is usually referred to as volume intersection. This correspondence addresses the problem of inferring the shape of the unknown object O from the reconstructed object R. For doing this, the author divides the points of the surface of R into hard points, which belong to the surface of any possible object originating R, and soft points, which may or may not belong to O. The author considers two cases: In the first case R is the closest approximation of O which can be obtained from its silhouettes, i.e., its visual hull; in the second case, R is a generic reconstructed object. In both cases the author supplies necessary and sufficient conditions for a point to be hard and gives rules for computing the hard surfaces     

一种应用参数曲面的动态自由变形方法

本文提出了一种动态自由变形方法，被变形物体首先附在两张参数曲面上，当参数曲面的形状发生变化时，物体会随之变形。这两张参数曲面分别被称为形状曲面和高度曲面，其中形状曲面用于控制变形物体的形状，高度曲面用于控制物体沿着形状曲面法向的高度。     

Coarse-to-fine surface reconstruction from silhouettes and range data using mesh deformation

We present a coarse-to-fine surface reconstruction method based on mesh deformation to build watertight surface models of complex objects from their silhouettes and range data. The deformable mesh, which initially represents the object visual hull, is iteratively displaced towards the triangulated range surface using the line-of-sight information. Each iteration of the deformation algorithm involves smoothing and restructuring operations to regularize the surface evolution process. We define a non-shrinking and easy-to-compute smoothing operator that fairs the surface separately along its tangential and normal directions. The mesh restructuring operator, which is based on edge split, collapse and flip operations, enables the deformable mesh to adapt its shape to the object geometry without suffering from any geometrical distortions. By imposing appropriate minimum and maximum edge length constraints, the deformable mesh, hence the object surface, can be represented at increasing levels of detail. This coarse-to-fine strategy, that allows high resolution reconstructions even with deficient and irregularly sampled range data, not only provides robustness, but also significantly improves the computational efficiency of the deformation process. We demonstrate the performance of the proposed method on several real objects.     

The enumerative geometry of projective algebraic surfaces and the complexity of aspect graphs

The aspect graph is a popular viewer-centered representation that enumerates all the topologically distinct views of an object. Building the aspect graph requires partitioning viewpoint space in view-equivalent cells by a certain number of visual event surfaces. If the object is piecewise-smooth algebraic, then all visual event surfaces are either made of lines having specified contacts with the object or made of lines supporting the points of contacts of planes having specified contacts with the object. In this paper, we present a general framework for studying the enumerative properties of line and plane systems. The context is that of enumerative geometry and intersection theory. In particular, we give exact results for the degrees of all visual event surfaces coming up in the construction of aspect graphs of piecewise-smooth algebraic bodies. We conclude by giving a bound on the number of topologically distinet views of such objects.This work was supported by a fellowship from the Ministry of Higher Education and Research of France.