A multiview 3D modeling system based on stereo vision techniques

This paper introduces a stereo vision system to automatically generate 3D models of real objects. 3D model generation is based on the merging of multiview range images obtained from a digital stereo camera. Stereo images obtained from the camera are rectified, and a correlation-based stereo matching technique reconstructs range images from them. A turntable stage is also employed to obtain multiple range images of the objects. To register range images into a common coordinate system automatically, we introduce and calibrate a turntable coordinate system with respect to the camera coordinate system. After the registration of multiview range images, a 3D model is reconstructed using a volumetric integration technique. Error analysis on turntable calibration and 3D model reconstruction shows the accuracy of our 3D modeling system.Received: 2 August 2003, Accepted: 20 September 2004, Published online: 25 February 2005 Correspondence to: S.Y. Park     

A robust hybrid image-based modeling system

This paper presents a new robust image-based modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods, we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a low-distortion mesh parameterization suitable for objects of arbitrary genus. A high-quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based inpainting method which exploits appearance space attributes to improve patch search, and blends patches using Poisson-guided interpolation. We analyzed the effect of different algorithm parameters, and compared our system with a laser scanning-based reconstruction and existing commercial systems. Our results indicate that our system is robust, superior to other image-based modeling techniques, and can achieve a reconstruction quality visually not discernible from that of a laser scanner.     

3D object model reconstruction from image sequence based on photometric consistency in volume space

This paper presents a system that can reconstruct a photorealistic 3D object model from an image sequence captured at arbitrary viewpoints. The whole system consists of four steps: camera calibration, volumetric modeling, polygonal model formation and texture mapping. We adopt the shape-from-silhouette approach for volumetric modeling. There are two common types of object surface that are difficult to reconstruct—textureless surface and concave surface. To tackle the problems, we propose to perform the volumetric modeling based on the constraints of viewpoint proximity and photometric consistency in the volume space. The volumetric model is converted to the mesh model for efficient manipulation. Finally, the texture map is generated from the image sequence to give the 3D model a photorealistic appearance. Some reconstructed object models are presented to demonstrate the superior performance of our system as compared with the conventional modeling technique based on the photo-consistency in the image space.     

基于序列图像的三维重构

提出了一种基于序列图像的三维重构方法。在相机标定的基础上,利用两幅图像间的匹配关系得到一个初始的三维模型,然后利用剩余图像与初始模型间的匹配关系,采用反投影映射的方法计算出剩余图像的外方位元素。最后经过稠密匹配、三角剖分和纹理渲染得到物体的三维模型。实验结果表明,相机标定精度较高的情况下,采用该算法能够得到较低失真的三维模型。     

3D Scene Reconstruction from Multiple Spherical Stereo Pairs

We propose a 3D environment modelling method using multiple pairs of high-resolution spherical images. Spherical images of a scene are captured using a rotating line scan camera. Reconstruction is based on stereo image pairs with a vertical displacement between camera views. A 3D mesh model for each pair of spherical images is reconstructed by stereo matching. For accurate surface reconstruction, we propose a PDE-based disparity estimation method which produces continuous depth fields with sharp depth discontinuities even in occluded and highly textured regions. A full environment model is constructed by fusion of partial reconstruction from spherical stereo pairs at multiple widely spaced locations. To avoid camera calibration steps for all camera locations, we calculate 3D rigid transforms between capture points using feature matching and register all meshes into a unified coordinate system. Finally a complete 3D model of the environment is generated by selecting the most reliable observations among overlapped surface measurements considering surface visibility, orientation and distance from the camera. We analyse the characteristics and behaviour of errors for spherical stereo imaging. Performance of the proposed algorithm is evaluated against ground-truth from the Middlebury stereo test bed and LIDAR scans. Results are also compared with conventional structure-from-motion algorithms. The final composite model is rendered from a wide range of viewpoints with high quality textures.     

New algorithm for 3D facial model reconstruction and its application in virtual reality

3D human face model reconstruction is essential to the generation of facial animations that is widely used in the field of virtual reality (VR). The main issues of 3D facial model reconstruction based on images by vision technologies are in twofold: one is to select and match the corresponding features of face from two images with minimal interaction and the other is to generate the realistic-looking human face model. In this paper, a new algorithm for realistic-looking face reconstruction is presented based on stereo vision. Firstly, a pattern is printed and attached to a planar surface for camera calibration, and corners generation and corners matching between two images are performed by integrating modified image pyramid Lucas-Kanade (PLK) algorithm and local adjustment algorithm, and then 3D coordinates of corners are obtained by 3D reconstruction. Individual face model is generated by the deformation of general 3D model and interpolation of the features. Finally, realistic-looking human face model     

An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery

Modeling the energy performance of existing buildings enables quick identification and reporting of potential areas for building retrofit. However, current modeling practices of using energy simulation tools do not model the energy performance of buildings at their element level. As a result, potential retrofit candidates caused by construction defects and degradations are not represented. Furthermore, due to manual modeling and calibration processes, their application is often time-consuming. Current application of 2D thermography for building diagnostics is also facing several challenges due to a large number of unordered and non-geo-tagged images. To address these limitations, this paper presents a new computer vision-based method for automated 3D energy performance modeling of existing buildings using thermal and digital imagery captured by a single thermal camera. First, using a new image-based 3D reconstruction pipeline which consists of Graphic Processing Unit (GPU)-based Structure-from-Motion (SfM) and Multi-View Stereo (MVS) algorithms, the geometrical conditions of an existing building is reconstructed in 3D. Next, a 3D thermal point cloud model of the building is generated by using a new 3D thermal modeling algorithm. This algorithm involves a one-time thermal camera calibration, deriving the relative transformation by forming the Epipolar geometry between thermal and digital images, and the MVS algorithm for dense reconstruction. By automatically superimposing the 3D building and thermal point cloud models, 3D spatio-thermal models are formed, which enable the users to visualize, query, and analyze temperatures at the level of 3D points. The underlying algorithms for generating and visualizing the 3D spatio-thermal models and the 3D-registered digital and thermal images are presented in detail. The proposed method is validated for several interior and exterior locations of a typical residential building and an instructional facility. The experimental results show that inexpensive digital and thermal imagery can be converted into ubiquitous reporters of the actual energy performance of existing buildings. The proposed method expedites the modeling process and has the potential to be used as a rapid and robust building diagnostic tool.     

摄像机简化模型对三维重构的影响--分析与实验

讨论了摄像机简化模型对三维重构的影响．主要结论有：当摄像机在两幅图像间的运动为纯平移运动时,从理论上证明了使用摄像机简化模型重构空间点与实际空间点之间满足仿射变换;当摄像机在两幅图像间的运动为一般刚体运动时,使用简化模型的重构只有在一定条件下才能较好地保持原物体的形状;在简化模型下,基于Kruppa方程的方法所估计的焦距精度不能满足三维重构的要求．实验结果表明：在三维重构中不能盲目地使用简化模型,必须对摄像机内参数进行全面标定．     

Camera calibration from surfaces of revolution

This paper addresses the problem of calibrating a pinhole camera from images of a surface of revolution. Camera calibration is the process of determining the intrinsic or internal parameters (i.e., aspect ratio, focal length, and principal point) of a camera, and it is important for both motion estimation and metric reconstruction of 3D models. In this paper, a novel and simple calibration technique is introduced, which is based on exploiting the symmetry of images of surfaces of revolution. Traditional techniques for camera calibration involve taking images of some precisely machined calibration pattern (such as a calibration grid). The use of surfaces of revolution, which are commonly found in daily life (e.g., bowls and vases), makes the process easier as a result of the reduced cost and increased accessibility of the calibration objects. In this paper, it is shown that two images of a surface of revolution will provide enough information for determining the aspect ratio, focal length, and principal point of a camera with fixed intrinsic parameters. The algorithms presented in this paper have been implemented and tested with both synthetic and real data. Experimental results show that the camera calibration method presented is both practical and accurate.     

基于图割及动态片结构的3维人脸多视图体重建

针对3维人脸重建问题提出了一种新颖的多视图体重建方法,以解决目前3维人脸重建方法只适用于小样本集合,大范围推广时精度难以保证的弱点。该方法创新之处在于将基于特征点匹配的重建方法与立体重建方法结合引入到图割优化框架,并应用于3维人脸重建。本文两个重要改进工作是设计动态片结构描述来进行颜色一致性估计以及设计新的动态图结构以去除半个体素尺寸的重建误差。实验中分别采用8张、16张和30张存在亮度变化的人脸多视角图像验证算法。实验结果逼真,同时避免了传统重建方法结果受限于样本集分布的问题。     

Implicit Reconstruction by Zooming

This paper presents a new method for inferring 3D information using a static camera equipped with a zoom lens. The modeling algorithm does not require any explicit calibration model and the calculations involved are straightforward. This approach uses several images of accurate regular grids placed on a micrometric table, as the calibration process. The basic idea is to compute a local transformation that allows the establishment of a relationship between a distorted grid detected on the CCD matrix and the real one located in front of the camera. This relationship takes automatically into account all distortion phenomena and allows one to obtain reconstruction results much more accurate than those of previous works in the same field. A complete experiment on real data is provided and shows that it is possible to compute 3D information from a zooming image set even if data are close to the optical axis.     

基于图像运动的三维重建及虚拟化应用

提出了一种稳定、快速地获取摄像机视频运动图像的三维重建方法,并对该运动图像做适当的虚拟化处理以展示重建效果。采用基于尺度不变特征点匹配的摄像机标定进行三维重建。尺度不变特征对于视频图像中的特征具有优秀敏锐的匹配能力,极大地放宽了摄像机标定对于设备上的限制,拓宽了实时三维重建的适用范围。通过对系统的一系列优化,不但提升了三维重建的精度,减少了错误匹配对摄像机标定的影响,而且进一步提升了处理速度。通过在三维重建的基础之上进行虚拟化处理,展示了本系统的三维重建效果。实验结果表明,该系统适用范围广,处理速度较快,重建精度高,实现了基于视频运动图像的三维重建。     

基于网格变形的从图像重建三维人脸

从图像重建高质量三维人脸一直是计算机视觉和图形学的一个重要研究问题.不同于传统的基于立体匹配的窄基线多视几何和数据驱动的人脸形变方法,提出一种结合网格变形技术和立体视觉原理的、从图像重建高质量三维人脸模型方法.给定从不同视角拍摄的几幅人脸图像,基于健壮图像特征获得可靠的相机外部参数和稀疏三维点;在此基础上,提出一种结合几何细节保持和图像一致性约束的三维人脸变形算法重建三维人脸,通过对人脸模板的网格变形,使得变形人脸在多幅图像中的可见投影具有一致性的图像颜色强度.基于模板的人脸变形可以有效地解决三维模型成像中的遮挡问题,采用健壮估计法消除噪声、离群点和光照对目标函数收敛性的影响,对目标函数的多次非线性优化求解进一步改进了人脸重建的质量.采用合成人脸图像和真实人脸图像重建三维人脸的实验结果表明,文中算法可以从几幅宽基线图像重建高质量的三维人脸模型.     

Three-dimensional laser scanning under the pinhole camera with lens distortion

This paper proposes a 3D laser scanning technology for the pinhole camera with lens distortion. On the basis of the pinhole camera mathematical model, we improve the extraction algorithm of Harris corner points, and introduce the professional 2D RAC calibration, 3D reconstruction, and 3D volume calculation. With the help of stepper motor, and the camera, we can capture a series of images and obtain the object 3D surface information. Furthermore, we use the line laser with Gaussian distribution as the measurement illuminant, and select the samples at the peak of intensity distribution. This brings both high accuracy and speed of measurement.     

3D reconstruction from 2D images with hierarchical continuous simplices

This paper presents an effective framework for the reconstruction of volumetric data from a sequence of 2D images. The 2D images are first aligned to generate an initial 3D volume, followed by the creation of a tetrahedral domain using the Carver algorithm. The resulting tetrahedralization preserves both the geometry and topology of the original dataset. Then a solid model is reconstructed using simplex splines with fitting and faring procedures. The reconstructed heterogenous volumetric model can be quantitatively analyzed and easily visualized. Our experiments demonstrated that our approach can achieve high accuracy in the data reconstruction. The novel techniques and algorithms proposed in this paper can be applied to reconstruct a heterogeneous solid model with complex geometry and topology from other visual data.     

三维重构中数码相机参数标定方法研究

随着数码技术的迅速发展,在许多场合人们已逐渐采用数码相机作为主要的图像采集设备.数码相机的标定被认为是实现三维重构基本而又关键的一步,可以为后继的立体图像匹配与三维重构奠定良好的基础.在介绍数码相机成像理论模型和已知相机内部参数的基础上,提出了一种新的利用直角三点来实现数码相机外部参数标定的方法.与现有的标定方法相比,简化了标定需要的特征点的数量和标定装置的复杂程度.文章最后通过实例说明了利用该标定方法实现三维重构的具体步骤和方法,并且验证了此方法的可行性及标定的精度.     

一种基于激光散斑去相关技术的三维重建方法

为了解决测量范围有限和摄像机标定复杂的问题,本文在传统的数字散斑相关计量方法基础上,提出一种基于数字散斑去相关测量的三维重建方法。首先通过激光照射散射体的不同面元发生干涉,在物体表面叠加形成散斑图像;其次利用不同空间距离处参考散斑图像的去相关特性来估计物体的位置;最后通过三角测量原理求出物体散斑图像和参考图像局部偏移量来确定物体的三维图。仿真实验结果表明,该系统能够在误差允许范围内得到基本完整的物体深度图像,并且与物体自身特性无关,具有良好的鲁棒性。     

一种基于TOF相机与CCD相机的联合标定算法研究

针对基于Time-of-Flight(TOF)相机的彩色目标三维重建需标定CCD相机与TOF相机联合系统的几何参数,在研究现有的基于彩色图像和TOF深度图像标定算法的基础上,提出了一种基于平面棋盘模板的标定方法。拍摄了固定在平面标定模板上的彩色棋盘图案在不同角度下的彩色图像和振幅图像,改进了Harris角点提取,根据棋盘格上角点与虚拟像点的共轭关系,建立了相机标定系统模型,利用Levenberg-Marquardt算法求解,进行了标定实验。获取了TOF与CCD相机内参数,并利用像平面之间的位姿关系估计两相机坐标系的相对姿态,最后进行联合优化,获取了相机之间的旋转矩阵与平移向量。实验结果表明,提出的算法优化了求解过程,提高了标定效率,能够获得较高的精度。     

视觉图像三维重构计算一般性框架研究

三维实体数字图像重构建模是当前计算机视觉研究领域的热点课题,涉及到相机标定、对应点匹配和三维重构计算等关键技术.本文回顾了三维重构建模的研究现状,分析了现有理论和方法在实际应用中存在的问题;在此基础上进行了相关研究,总结出了三维重构计算的一般性框架,提出了一些研究观点和方法,并探讨了三维重构技术的难点和发展趋势.     

Parametric Surface Representation with Bump Image for Dense 3D Modeling Using an RBG-D Camera

When constructing a dense 3D model of an indoor static scene from a sequence of RGB-D images, the choice of the 3D representation (e.g. 3D mesh, cloud of points or implicit function) is of crucial importance. In the last few years, the volumetric truncated signed distance function (TSDF) and its extensions have become popular in the community and largely used for the task of dense 3D modelling using RGB-D sensors. However, as this representation is voxel based, it offers few possibilities for manipulating and/or editing the constructed 3D model, which limits its applicability. In particular, the amount of data required to maintain the volumetric TSDF rapidly becomes huge which limits possibilities for portability. Moreover, simplifications (such as mesh extraction and surface simplification) significantly reduce the accuracy of the 3D model (especially in the color space), and editing the 3D model is difficult. We propose a novel compact, flexible and accurate 3D surface representation based on parametric surface patches augmented by geometric and color texture images. Simple parametric shapes such as planes are roughly fitted to the input depth images, and the deviations of the 3D measurements to the fitted parametric surfaces are fused into a geometric texture image (called the Bump image). A confidence and color texture image are also built. Our 3D scene representation is accurate yet memory efficient. Moreover, updating or editing the 3D model becomes trivial since it is reduced to manipulating 2D images. Our experimental results demonstrate the advantages of our proposed 3D representation through a concrete indoor scene reconstruction application.