How far 3D shapes can be understood from 2D silhouettes

Each 2D silhouette of a 3D unknown object O constrains O inside the volume obtained by back-projecting the silhouette from the corresponding viewpoint. A set of silhouettes specifies a boundary volume R obtained by intersecting the volumes due to each silhouette. R more or less closely approximates O, depending on the viewpoints and the object itself. This approach to the reconstruction of 3D objects is usually referred to as volume intersection. This correspondence addresses the problem of inferring the shape of the unknown object O from the reconstructed object R. For doing this, the author divides the points of the surface of R into hard points, which belong to the surface of any possible object originating R, and soft points, which may or may not belong to O. The author considers two cases: In the first case R is the closest approximation of O which can be obtained from its silhouettes, i.e., its visual hull; in the second case, R is a generic reconstructed object. In both cases the author supplies necessary and sufficient conditions for a point to be hard and gives rules for computing the hard surfaces     

A quasi-dense approach to surface reconstruction from uncalibrated images

This paper proposes a quasi-dense approach to 3D surface model acquisition from uncalibrated images. First, correspondence information and geometry are computed based on new quasi-dense point features that are resampled subpixel points from a disparity map. The quasi-dense approach gives more robust and accurate geometry estimations than the standard sparse approach. The robustness is measured as the success rate of full automatic geometry estimation with all involved parameters fixed. The accuracy is measured by a fast gauge-free uncertainty estimation algorithm. The quasi-dense approach also works for more largely separated images than the sparse approach, therefore, it requires fewer images for modeling. More importantly, the quasi-dense approach delivers a high density of reconstructed 3D points on which a surface representation can be reconstructed. This fills the gap of insufficiency of the sparse approach for surface reconstruction, essential for modeling and visualization applications. Second, surface reconstruction methods from the given quasi-dense geometry are also developed. The algorithm optimizes new unified functionals integrating both 3D quasi-dense points and 2D image information, including silhouettes. Combining both 3D data and 2D images is more robust than the existing methods using only 2D information or only 3D data. An efficient bounded regularization method is proposed to implement the surface evolution by level-set methods. Its properties are discussed and proven for some cases. As a whole, a complete automatic and practical system of 3D modeling from raw images captured by hand-held cameras to surface representation is proposed. Extensive experiments demonstrate the superior performance of the quasi-dense approach with respect to the standard sparse approach in robustness, accuracy, and applicability.     

Automatic reconstruction of 3D human arm motion from a monocular image sequence

A model-based approach to reconstruction of 3D human arm motion from a monocular image sequence taken under orthographic projection is presented. The reconstruction is divided into two stages. First, a 2D shape model is used to track the arm silhouettes and second-order curves are used to model the arm based on an iteratively reweighted least square method. As a result, 2D stick figures are extracted. In the second stage, the stick figures are backprojected into the scene. 3D postures are reconstructed using the constraints of a 3D kinematic model of the human arm. The motion of the arm is then derived as a transition between the arm postures. Applications of these results are foreseen in the analysis of human motion patterns. Received: 26 January 1996 / Accepted: 17 July 1997     

Silhouette smoothing for real-time rendering of mesh surfaces

Coarse piecewise linear approximation of surfaces causes undesirable polygonal appearance of silhouettes. We present an efficient method for smoothing the silhouettes of coarse triangle meshes using efficient 3D curve reconstruction and simple local re-meshing. It does not assume the availability of a fine mesh and generates only moderate amount of additional data at run time. Furthermore, polygonal feature edges are also smoothed in a unified framework. Our method is based on a novel interpolation scheme over silhouette triangles and this ensures that smooth silhouettes are faithfully reconstructed and always change continuously with respect to continuous movement of the view point or objects. We speed up computation with GPU assistance to achieve real-time rendering of coarse meshes with the smoothed silhouettes. Experiments show that this method outperforms previous methods for silhouette smoothing.     

基于截面数据的C1插值融合曲面重建

为了避免NURBS曲面重建需要进行节点矢量相容的问题，提出了一种双方向融合插值的[C1]参数曲面重建方法，该方法先后分段插值截面上连续的数据点、截面曲线以构造样条曲线和曲面片，并引入融合算法进行曲线、曲面拼接，从而得到光滑的待建曲面。该方法不会产生由节点插入所带来的大量的数据冗余以及复杂的计算过程，同时采用了融合的思想来处理曲线、曲面的拼接，改良了传统参数曲线、曲面拼接方法需要满足边界条件的缺陷。     

任意摆放的二次曲面体的三维重建

本文采用特征点匹配的方法,先由二维视力获取三维空间的点和曲线,然后经过一系列线性运算得出三维曲面。本算法可对任意二次曲面进行三维重建,不仅对曲面的空间摆放沿有限制,而且对曲面自身的对称性也无要求。因此,可将重建对象扩展为圆柱、椭圆柱、球、椭球、双曲面、抛物面等。本文给出的算例表明了算法的有效性。     

Error estimation in reconstruction of quadratic curves in 3D space

Many natural and man-made objects have planar and curvilinear surfaces. The images of such curves do not usually have sufficient distinctive features to apply conventional feature-based reconstruction algorithms. In this paper, we describe a method for the reconstruction of various kinds of quadratic curves in 3D space as an intersection of two cones containing the respective projected digitized curve images in the presence of Gaussian noise. The advantage of this method is that it overcomes the correspondence problem that occurs in pairs of projections of the curve. Using nonlinear least-squares curve fitting, the parameters of a curve in 2D digitized image planes are determined. From this we reconstruct the 3D quadratic curve. Relevant mathematical formulations and analytical solutions for obtaining the equation of the reconstructed curve are given. Simulation studies have been conducted to observe the effect of noise on errors in the process of reconstruction. Results for various types of quadratic curves are presented using simulation studies. These are the main contributions of this work. The angle between the reconstructed and the original quadratic curves in 3D space has been used as the criterion for the measurement of the error. The results of this study are useful for the design of a stereo-based imaging system (such as the LBW decision in cricket, the path of a missile, robotic vision, path planning, etc.) and for the best reconstruction with minimum error.     

基于单张影像的月面三维仿真

根据阴影恢复形状原理,提出一种基于单张影像的快速月面三维建模和光照模拟方法。利用单张2D影像中残留的3D线索——灰度信息,通过SFS方法对月面三维形貌进行快速重建,采用改进的Hapke光照模型对重建后的三维形貌进行渲染。实验结果证明,在精度允许范围内,该方法能快速地实现对月面三维形貌的提取和仿真。     

基于三视图的三维形体重建技术

基于工程图纸的三维形体的重建技术是根据形体的二维视图中的几何信息和拓扑信息,生成相应的三维形体,这是一个从爸维到高维的构造过程,文中提出了一个三维形体重建算法,该算法首先根据三视图生成形体的线框图,然后应用左邻边搜索策略求出线框图内的所有面及相应的极小环,最后利用Ｍｏｅｂｉｕｓ规则及工程图的性质,删除重建过程中生成的非法元素,该方法利用二几何基元和三维几何基元的几何性质及生成关系,减少了重建过程中     

3D reconstruction for featureless scenes with curvature hints

We present a novel interactive framework for improving 3D reconstruction starting from incomplete or noisy results obtained through image-based reconstruction algorithms. The core idea is to enable the user to provide localized hints on the curvature of the surface, which are turned into constraints during an energy minimization reconstruction. To make this task simple, we propose two algorithms. The first is a multi-view segmentation algorithm that allows the user to propagate the foreground selection of one or more images both to all the images of the input set and to the 3D points, to accurately select the part of the scene to be reconstructed. The second is a fast GPU-based algorithm for the reconstruction of smooth surfaces from multiple views, which incorporates the hints provided by the user. We show that our framework can turn a poor-quality reconstruction produced with state of the art image-based reconstruction methods into a high- quality one.     

带自由曲面的三视图重建算法研究

针对蕴含自由曲面的三视图,提出了通过恢复边界线和轮廓线进而根据这些恢复 的空间曲线重建自由曲面的算法。首先分析并证明了自由曲面在三视图中的投影性质,从而提 出边界投影和轮廓投影的匹配算法。针对视图中存在的被打断的样条曲线,提出了分段样条曲 线的爬坡算法来解决此类曲线不能匹配的问题。然后基于投影匹配序列重建出自由曲面的边界 线,再由边界投影上的点和轮廓投影端点的对应关系重建出空间轮廓线。由轮廓线等参采样构 造截面线并和边界线一起蒙皮生成最终自由曲面。本文提出的算法扩展了工程图的重建域。     

基于光线的全局优化多视图三维重建方法

利用光线跟踪原理,提出一种全局优化的多视图三维重建方法。根据图像轮廓得到物体的包围盒,采用体素离散物体所在的几何空间。从相机中心向图像上每个像素发射一条光线,为确定光线达到的体素,使用归一化互相关（NCC）值度量光线一体素的一致性,并估计采样空间中面片的法向信息,以提高NCC值的可信度。设计基于因子图的全局优化模型得到物体体素,针对光线因子的特殊性,设计一种高效的置信度传播算法,使重建方法的时间复杂度从指数阶降为线性阶。实验结果表明,与基于马尔可夫场的重建方法相比,该方法的鲁棒性较好,可提高重建模型的准确度和完整性。     

Shape estimation using polarization and shading from two views

This paper presents a novel method for 3D surface reconstruction that uses polarization and shading information from two views. The method relies on polarization data acquired using a standard digital camera and a linear polarizer. Fresnel theory is used to process the raw images and to obtain initial estimates of surface normals, assuming that the reflection type is diffuse. Based on this idea, the paper presents two novel contributions to the problem of surface reconstruction. The first is a technique to enhance the surface normal estimates by incorporating shading information into the method. This is done using robust statistics to estimate how the measured pixel brightnesses depend on the surface orientation. This gives an estimate of the object material reflectance function, which is used to refine the estimates of the surface normals. The second contribution is to use the refined estimates to establish correspondence between two views of an object. To do this, a set of patches are extracted from each view and are aligned by minimizing an energy functional based on the surface normal estimates and local topographic properties. The optimum alignment parameters for different patch pairs are then used to establish stereo correspondence. This process results in an unambiguous field of surface normals, which can be integrated to recover the surface depth. Our technique is most suited to smooth, non-metallic surfaces. It complements existing stereo algorithms since it does not require salient surface features to obtain correspondences. An extensive set of experiments, yielding reconstructed objects and reflectance functions, are presented and compared to ground truth.     

一种基于投影的散乱数据表面增量重建算法

针对3维散乱数据场提出了一种表面重建算法.根据空间曲面的局平特性和平面三角化的基本原则,在参考点的切平面上对邻域点按角度排序,应用可见性准则删除不可见点后,相邻邻域点和参考点形成三角网格.将平面上的网格关系对应到空间,以增量方式重建反映散乱数据场拓扑关系的空间曲面.设定角度阈值优化网格,判断空间曲面的边界和孔洞.对多个数据场进行重建并对结果进行分析.对多个数据场进行重建并对结果进行分析表明,算法具有原理简单,重建速度快,重建效果好的特点.     

从二维系列摄影图片提取剪影重构三维实体的光线跟踪算法

给出了利用二维系列摄影图片提取剪影重构三维实体的一种新算法,即围绕一个三维实体从多个角度拍摄照片,然后从照片中提取出实体的边界,通过基于光线跟踪的、对二维影像的合成得到原物体近似表面的三维点坐标,与传统的基于体像素的算法相比,该算法节省存储空间,近似精度与三维分辨率无直接关系,而且速度有所提高,特别当实体体积较大时,效果明显。     

联合LBS和Snake的3D人体外形和运动跟踪方法

为了解决基于多目视频轮廓信息的3D人体外形和运动跟踪问题,提出一种联合线性混合蒙皮和Snake变形模型的算法框架.首先建立人物对象的蒙皮模型,以每一帧多目同步视频的轮廓作为输入,采用一种基于剪影轮廓的可视外壳重建算法,使得作为3D特征的可视外壳保持了局部细节且更加光滑;并使用关节型迭代最近点算法进行匹配以捕获出每一帧骨架子空间下的人物3D外形及运动;再一次使用当前帧的多目轮廓信息,让Snake内外力共同作用于人物网格模型上的顶点,使之自由地趋近于目标对象.使用带ground-truth的合成数据进行对比实验的结果表明,该方法因同时使用3D误差约束和2D误差约束,提高了跟踪精度.     

The visual hull concept for silhouette-based image understanding

Many algorithms for both identifying and reconstructing a 3-D object are based on the 2-D silhouettes of the object. In general, identifying a nonconvex object using a silhouette-based approach implies neglecting some features of its surface as identification clues. The same features cannot be reconstructed by volume intersection techniques using multiple silhouettes of the object. This paper addresses the problem of finding which parts of a nonconvex object are relevant for silhouette-based image understanding. For this purpose, the geometric concept of visual hull of a 3-D object is introduced. This is the closest approximation of object S that can be obtained with the volume intersection approach; it is the maximal object silhouette-equivalent to S, i.e., which can be substituted for S without affecting any silhouette. Only the parts of the surface of S that also lie on the surface of the visual hull can be reconstructed or identified using silhouette-based algorithms. The visual hull depends not only on the object but also on the region allowed to the viewpoint. Two main viewing regions result in the external and internal visual hull. In the former case the viewing region is related to the convex hull of S, in the latter it is bounded by S. The internal visual hull also admits an interpretation not related to silhouettes. Algorithms for computing visual hulls are presented and their complexity analyzed. In general, the visual hull of a 3-D planar face object turns out to be bounded by planar and curved patches     

基于两视图的曲面重建方法

提出了一种利用曲面物体的两幅图像进行曲面重建的方法。从物体的正面和侧面拍摄两幅图像，而后利用用户交互获得这两幅图像中的侧影轮廓线。并通过最小化以这些侧影轮廓线作为约束二次目标函数，重建出最光滑的三维曲面。另外，在二次目标函数中增加一个系数，实现了曲面折痕。利用真实图像进行实验，验证了该方法的可行性和正确性。     

Exploration of local surface geometry with minimum number of contact points and surface normal information

In this paper, we propose a method of exploring the surface geometry of an unknown object by touch. The method is based on the idea that a three-dimensional surface geometry can be reconstructed from two principal curvatures of the object which are estimated from three concurrent curves. First, the process to minimize the number of contact points is addressed for the approximation of an arbitrary curve, which uses normal vectors at the contact points. Then, an algorithm for reconstructing a three-dimensional local surface from four contact points, two of which can be used to compute a normal curvature, is presented. Lastly, our method is applied to cylindrical, spherical and planar objects in simulation and experiments for validation.