The visual hull concept for silhouette-based image understanding

Many algorithms for both identifying and reconstructing a 3-D object are based on the 2-D silhouettes of the object. In general, identifying a nonconvex object using a silhouette-based approach implies neglecting some features of its surface as identification clues. The same features cannot be reconstructed by volume intersection techniques using multiple silhouettes of the object. This paper addresses the problem of finding which parts of a nonconvex object are relevant for silhouette-based image understanding. For this purpose, the geometric concept of visual hull of a 3-D object is introduced. This is the closest approximation of object S that can be obtained with the volume intersection approach; it is the maximal object silhouette-equivalent to S, i.e., which can be substituted for S without affecting any silhouette. Only the parts of the surface of S that also lie on the surface of the visual hull can be reconstructed or identified using silhouette-based algorithms. The visual hull depends not only on the object but also on the region allowed to the viewpoint. Two main viewing regions result in the external and internal visual hull. In the former case the viewing region is related to the convex hull of S, in the latter it is bounded by S. The internal visual hull also admits an interpretation not related to silhouettes. Algorithms for computing visual hulls are presented and their complexity analyzed. In general, the visual hull of a 3-D planar face object turns out to be bounded by planar and curved patches     

Shape recovery from turntable sequence using rim reconstruction

This paper makes use of both feature points and silhouettes to deliver fast 3D shape recovery. The algorithm exploits object silhouettes in two views to establish a 3D rim curve, which is defined with respect to the two frontier points arising from two views. The images of this 3D rim curve in the two views are matched using cross-correlation technique. A 3D planar rim curve is then reconstructed using point-based reconstruction method. A set of 3D rim curves enclosing the object can be obtained from an image sequence captured under circular motion. Silhouettes are further utilized to check for mismatched rim points. The proposed method solves the problem of reconstruction of concave object surface, which is usually left unresolved in general silhouette-based reconstruction methods. In addition, the property of the reconstructed 3D rim curves allows fast surface extraction. Experimental results with real data are presented.     

三维模型轮廓线抽取算法

虽然三维模型的轮廓线在图形交流中起着重要的作用,但由于轮廓线是视点依赖的,当物体在空间运动时,用现有算法从复杂的三维模型中抽取轮廓线需耗费大量时间,为了提高抽取轮廓线的速度,提出了两种新的抽取三维模型轮廓线算法,这两种算法是先利用轮廓线的局部极值特性来获得部分轮廓边,然后利用轮廓线的连通性,通过简单的比较运算,即可获得三维模型的外部轮廓线,实验结果表明,该两种算法都可快速获得三维模型的外围轮廓线,最后还将本算法与相关算法进行了详细的比较分析。     

Silhouette and stereo fusion for 3D object modeling

In this paper, we present a new approach to high quality 3D object reconstruction. Starting from a calibrated sequence of color images, the algorithm is able to reconstruct both the 3D geometry and the texture. The core of the method is based on a deformable model, which defines the framework where texture and silhouette information can be fused. This is achieved by defining two external forces based on the images: a texture driven force and a silhouette driven force. The texture force is computed in two steps: a multi-stereo correlation voting approach and a gradient vector flow diffusion. Due to the high resolution of the voting approach, a multi-grid version of the gradient vector flow has been developed. Concerning the silhouette force, a new formulation of the silhouette constraint is derived. It provides a robust way to integrate the silhouettes in the evolution algorithm. As a consequence, we are able to recover the contour generators of the model at the end of the iteration process. Finally, a texture map is computed from the original images for the reconstructed 3D model.     

Introducing a new problem: shape-from-silhouette when the relative positions of the viewpoints is unknown

3D shapes can be reconstructed from 2D silhouettes by back-projecting them from the corresponding viewpoints and intersecting the resulting solid cones. However, in many practical cases as observing an aircraft or an asteroid, the positions of the viewpoints with respect to the object are not known. In these cases, the relative position of the solid cones is not known and the intersection cannot be performed. The purpose of this paper is introducing and stating in a theoretical framework the problem of understanding 3D shapes from silhouettes when the relative positions of the viewpoints are unknown. The results presented provide a first insight into the problem. In particular, the case of orthographic viewing directions parallel to the same plane is thoroughly discussed, and sets of inequalities are presented which allow determining objects compatible with the silhouettes.     

A volumetric fusion technique for surface reconstruction from silhouettes and range data

Optical triangulation, an active reconstruction technique, is known to be an accurate method but has several shortcomings due to occlusion and laser reflectance properties of the object surface, that often lead to holes and inaccuracies on the recovered surface. Shape from silhouette, on the other hand, as a passive reconstruction technique, yields robust, hole-free reconstruction of the visual hull of the object. In this paper, a hybrid surface reconstruction method that fuses geometrical information acquired from silhouette images and optical triangulation is presented. Our motivation is to recover the geometry from silhouettes on those parts of the surface which the range data fail to capture. A volumetric octree representation is first obtained from the silhouette images and then carved by range points to amend the missing cavity information. An isolevel value on each surface cube of the carved octree structure is accumulated using local surface triangulations obtained separately from range data and silhouettes. The marching cubes algorithm is then applied for triangulation of the volumetric representation. The performance of the proposed technique is demonstrated on several real objects.     

Silhouette coherence for camera calibration under circular motion

We present a new approach to camera calibration as a part of a complete and practical system to recover digital copies of sculpture from uncalibrated image sequences taken under turntable motion. In this paper, we introduce the concept of the silhouette coherence of a set of silhouettes generated by a 3D object. We show how the maximization of the silhouette coherence can be exploited to recover the camera poses and focal length. Silhouette coherence can be considered as a generalization of the well-known epipolar tangency constraint for calculating motion from silhouettes or outlines alone. Further, silhouette coherence exploits all the geometric information encoded in the silhouette (not just at epipolar tangency points) and can be used in many practical situations where point correspondences or outer epipolar tangents are unavailable. We present an algorithm for exploiting silhouette coherence to efficiently and reliably estimate camera motion. We use this algorithm to reconstruct very high quality 3D models from uncalibrated circular motion sequences, even when epipolar tangency points are not available or the silhouettes are truncated. The algorithm has been integrated into a practical system and has been tested on more than 50 uncalibrated sequences to produce high quality photo-realistic models. Three illustrative examples are included in this paper. The algorithm is also evaluated quantitatively by comparing it to a state-of-the-art system that exploits only epipolar tangents     

三维封闭三角网格孔洞边界识别算法

针对三维封闭三角网格模型的缺失实体孔洞,提出一种边界识别算法。以手绘曲线作为输入,结合轮廓线法得到目标孔洞的近似边界点集,生成连续的孔洞轮廓线。按照曲线点集并以孔洞轮廓线辅助搜索孔洞的上边界,根据上边界向下搜索得到孔洞的下边界。该算法可用于满足二维流形、可定向及封闭的三角网格模型上孔洞边界的定位,能够简化孔洞边界的定位过程,提高设计系统的易用性。     

联合LBS和Snake的3D人体外形和运动跟踪方法

为了解决基于多目视频轮廓信息的3D人体外形和运动跟踪问题,提出一种联合线性混合蒙皮和Snake变形模型的算法框架.首先建立人物对象的蒙皮模型,以每一帧多目同步视频的轮廓作为输入,采用一种基于剪影轮廓的可视外壳重建算法,使得作为3D特征的可视外壳保持了局部细节且更加光滑;并使用关节型迭代最近点算法进行匹配以捕获出每一帧骨架子空间下的人物3D外形及运动;再一次使用当前帧的多目轮廓信息,让Snake内外力共同作用于人物网格模型上的顶点,使之自由地趋近于目标对象.使用带ground-truth的合成数据进行对比实验的结果表明,该方法因同时使用3D误差约束和2D误差约束,提高了跟踪精度.     

Experimenting with nonintrusive motion capture in a virtual environment

A growing number of promising applications requires recognizing human posture and motion. Conventional techniques require us to attach foreign objects to the body, which in some applications is disturbing or even impossible. New, nonintrusive motion capture approaches are called for. The well-known shape-from-silhouette technique for understanding 3D shapes could also be effective for human bodies. We present a novel technique for model-based motion capture that uses silhouettes extracted from multiple views. A 3D reconstruction of the performer can be computed from a silhouette with a technique known as volume intersection. We can recover the posture by fitting a model of the human body to the reconstructed volume. The purpose of this work is to test the effectiveness of this approach in a virtual environment by investigating the precision of the posture and motion obtained with various numbers and arrangements of stationary cameras. An average 1% position error has been obtained with five cameras.     

基于侧影轮廓的图像三维重建方法

针对自标定单幅图像的三维重建问题,提出一种基于侧影轮廓的图像三维重建方法。使用成角度平面镜装置拍摄目标物体,对所得图像进行边缘跟踪,得到闭合的轮廓曲线,利用满足平面镜成像原理的2对侧影轮廓计算灭点,根据灭点间的约束关系计算相机参数,在此基础上重建得到物体的三维模型。实验结果表明,该方法能快速重建逼真的三维模型。     

Shape-From-Silhouette Across Time Part I: Theory and Algorithms

Shape-From-Silhouette (SFS) is a shape reconstruction method which constructs a 3D shape estimate of an object using silhouette images of the object. The output of a SFS algorithm is known as the Visual Hull (VH). Traditionally SFS is either performed on static objects, or separately at each time instant in the case of videos of moving objects. In this paper we develop a theory of performing SFS across time: estimating the shape of a dynamic object (with unknown motion) by combining all of the silhouette images of the object over time. We first introduce a one dimensional element called a Bounding Edge to represent the Visual Hull. We then show that aligning two Visual Hulls using just their silhouettes is in general ambiguous and derive the geometric constraints (in terms of Bounding Edges) that govern the alignment. To break the alignment ambiguity, we combine stereo information with silhouette information and derive a Temporal SFS algorithm which consists of two steps: (1) estimate the motion of the objects over time (Visual Hull Alignment) and (2) combine the silhouette information using the estimated motion (Visual Hull Refinement). The algorithm is first developed for rigid objects and then extended to articulated objects. In the Part II of this paper we apply our temporal SFS algorithm to two human-related applications: (1) the acquisition of detailed human kinematic models and (2) marker-less motion tracking.     

Silhouette smoothing for real-time rendering of mesh surfaces

Coarse piecewise linear approximation of surfaces causes undesirable polygonal appearance of silhouettes. We present an efficient method for smoothing the silhouettes of coarse triangle meshes using efficient 3D curve reconstruction and simple local re-meshing. It does not assume the availability of a fine mesh and generates only moderate amount of additional data at run time. Furthermore, polygonal feature edges are also smoothed in a unified framework. Our method is based on a novel interpolation scheme over silhouette triangles and this ensures that smooth silhouettes are faithfully reconstructed and always change continuously with respect to continuous movement of the view point or objects. We speed up computation with GPU assistance to achieve real-time rendering of coarse meshes with the smoothed silhouettes. Experiments show that this method outperforms previous methods for silhouette smoothing.     

Model construction and shape recognition from occluding contours

A technique is presented for recognizing a 3D object (a model in an image library) from a single 2D silhouette using information such as corners (points with high positive curvatures) and occluding contours, rather than straight line segments. The silhouette is assumed to be a parallel projection of the object. Each model is stored as a set of the principal quadtrees, from which the volume/surface octree of the model is generated. Feature points (i.e. corners) are extracted to guide the recognition process. Four-point correspondences between the 2D feature points of the observed object and 3D feature points of each model are hypothesized, and then verified by applying a variety of constraints to their associated viewing parameters. The result of the hypothesis and verification process is further validated by 2D contour matching. This approach allows for a method of handling both planar and curved objects in a uniform manner, and provides a solution to the recognition of multiple objects with occlusion as demonstrated by the experimental results     

用轮廓的点分布特征分析和识别步态

提出了一种轮廓点分布特征匹配方法来分析和识别行人步态。首先通过设计一种轮廓采样点区域分布直方图分析了步态的周期性特征,提取了一个步态周期的帧图像轮廓;继而采用一种局部轮廓描述子得到帧图像轮廓的点分布直方图阵列作为轮廓特征、用轮廓点集间的Hausdorff距离结合动态时间规整技术求取测试序列和参考序列间的匹配相似度;最后通过分类实现了人体步态识别。在Soton步态数据库进行了实验,正确分类识别率最高达到90.27%。相关文献的对比分析表明：该方法的识别率有较大的提高,是有效的。     

基于多幅实拍照片为真实景物模型添加纹理

利用实拍照片为基于真实景物创建的几何模型添加纹理的方法正在受到广泛的关注,实拍照片与几何模型的配准是这项技术的关键.以往方法采用3D-2D特征点匹配或侧影轮廓线匹配的方法进行配准,因此对空间物体的表面特征或轮廓线形状有特殊的要求.提出了一种新的配准方法来解决这一问题,由于采用了基于图像重建的采样点模型与已知几何模型在空间中匹配的方法实现配准,因而充分利用了物体几何形状本身的拓扑和曲率等信息,并可以一次性地实现所有图像与空间物体的配准.实验结果表明,该方法可以解决一部分用以往的方法尚无法处理的实际问题,且在重建空间采样点分布较为合理的情形下,纹理映射效果非常理想.     

Reconstruction of 3D Object Meshes from Silhouette Images

In this work we propose a method for computing mesh representations of 3D objects reconstructed from a set of silhouette images. Our method is based on the polygonization of volumetric reconstructions by using a modified version of the dual contouring method. In order to apply dual contouring on volumetric reconstruction from silhouettes we devised a method that is able to determine the discrete topology of the surface in relation to the octree cells. We also developed a new scheme for computing hermitian data representing the intersections of conic volumes with the octree cells and their corresponding normals with subpixel accuracy. Due to the discrete and extremely noisy nature of the data used in the reconstruction we had to devise a different criterion for mesh simplification that applies topological consistency tests only when the geometric error measure is beyond a given tolerance. We present results of the application of the proposed method in the extraction of a mesh corresponding to the surface of objects of a real scene.     

Perspective silhouette of a general swept volume

We present an efficient and robust algorithm for computing the perspective silhouette of the boundary of a general swept volume. We also construct the topology of connected components of the silhouette. At each instant t, a three-dimensional object moving along a trajectory touches the envelope surface of its swept volume along a characteristic curve K^t. The same instance of the moving object has a silhouette curve L^t on its own boundary. The intersection K^t∩L^t contributes to the silhouette of the general swept volume. We reformulate this problem as a system of two polynomial equations in three variables. The connected components of the resulting silhouette curves are constructed by detecting the instances where the two curves K^t and L^t intersect each other tangentially on the surface of the moving object. We also consider a general case where the eye position changes while moving along a predefined path. The problem is reformulated as a system of two polynomial equations in four variables, where the zero-set is a two-manifold. By analyzing the topology of the zero-set, we achieve an efficient algorithm for generating a continuous animation of perspective silhouettes of a general swept volume.     

基于侧影轮廓的三维模型快速重建

提出一种基于侧影轮廓进行三维模型重建的新方法,将传统的三维锥形交叉问题转换成二维轮廓交叉问题。首先,将不同视角下的二维侧影轮廓反投影到若干个平行的三维平面上,然后在三维平面上计算所有反投影轮廓的交叉轮廓,最后对相邻三维平面上的交叉轮廓进行匹配并重建物体的表面。理论分析和实验结果表明该算法的时间复杂度和视角数目呈线性关系。由于该方法主要以增加视角数目来提高模型的精确度,所以比三维锥形交叉的方法能更快速地重建物体精确的三维模型。