基于平面的建筑物表面模型重建算法的研究

针对建筑物模型的规则性，提出了一种基于平面的建筑物模型重建算法，可以从单幅透视图像恢复建筑物的表面模型．该算法主要分为相机定标、基平面的提取、平面位置和方向的计算等几个子过程．相机定标主要用于求解相机的焦距，是一个非常重要的部分．该算法以建筑物场景中的几何结构作为约束条件，从单幅图像中求解相机的焦距；然后计算基平面位置和法向；最后通过交互式操作指明场景中各平面之间的相互关系，递归求解各平面的位置和法向，达到根据图像重建建筑物场景表面模型的目的．     

基于场景几何约束未标定两视图的三维模型重建

提出了一种从两幅未标定图象重建场景三维模型的方法 .这种方法充分利用了人造结构场景中大量存在的平行性和正交性几何约束 ,即利用每幅视图中三组互相垂直的平行线 ,计算出 3个影灭点 ,从而对每幅视图进行标定 .对两幅未标定图象 ,从基本矩阵只能得到射影重构 ,如果每幅图象都已标定 ,则可将基本矩阵转化为本质矩阵 .三维重构过程有两个步骤 :先是恢复相机的位置和运动 ;后是用三角测量法计算出点的三维坐标 .对多平面组成的场景进行三维重构实验 ,所得三维模型产生新的视点图象 ,与所观察的场景一致 ,重构的两个平面夹角与实际值相近 ,实验结果表明 ,该算法是行之有效的     

基于平面的Warping技术

提出了一种基于平面的逆向Warping算法,用于根据多幅参考图像生成任意视点下的新图像.首先通过参考图像的深度信息来重建三维平面,然后寻找这些重建平面间的对应关系,并比较它们对空间平面同一部分的采样密度,以获得最好的采样结果.在生成新视点图像时,首先对那些采样密度最好的重建平面进行可见性判断,然后将其投影到新视点下,在此基础上求得目标图像上各点的深度,最后将目标图像上的点逆向Warping到相应的参考图像中,以获取它们的颜色值.同时,对于参考图像中不能被重建成平面的像素点,用正向Warping的方法对其进     

一种基于深度和颜色的3D表面重建方法

为了快速、有效地对三维物体的表面进行重建，提出了一种基于深度和颜色的3D表面重建方法。该方法利用同一物体在空间稀疏分布的不同视点下的多幅深度图像，根据3D物体表面同一点在不同深度图像上颜色和纹理的一致性，通过逆投影变换，可将图像中像素点映射到三维空间中的正确位置。然后在以物体为中心的单个坐标系下，进行数据融合，可得到三维物体表面各点的三维数据。实验结果表明该方法具有复杂度恒定、获取图像真实感强等优点，适用于形状、结构草杂的物体重建。     

三维动态脚型测量方法

基于多视几何原理来恢复和重建随时间变化的三维柔性物体是当前计算机视觉和三维扫描领域的研究热点之一,本文利用主动式标记点方法,提出并实现了在三维模型指导下鲁棒地从多视点视频中恢复和重建三维脚型的方法.它首先基于传统立体视觉方法重建出参考帧中的三维脚模型,然后在此基础上,通过构建三维模型顶点运动投影的速度矢量场来建立相邻帧...     

基于水平集的3D左心室表面重建

提出了基于分层2D MRI图像重建3D左心室表面的算法．分层2D MRI图像能够跟踪左心室边界在成像平面内的变形,但由于缺少层问数据,层间边界形变的信息难以获取．为了重建左心室的形状,使用基于形变模型的方法：首先建立变形曲面的动力学方程;再将成像坐标系中的图像平面映射到重建坐标系,根据图像数据构造外力,使用平均曲率构造变形曲面的弹性力,然后用水平集数值解法求解动力学方程以重建左心室的表面．实验结果证明了算法的有效性．     

面向未知物体自动测量和重建的视点规划方法

针对未知三维物体自动测量和重建问题,提出一种基于视觉系统极限可视区域的新颖视点规划方法.首先确定单目线激光视觉测量系统的最佳可视测量区域,结合初始视点下所获模型的边界区域信息构造未知空间的极限可视表面模型,并以此表面作为未知物体的预测曲面模型;然后确定下一视点预测曲线的可视性,将左右规划进程中能获取最大可视曲线长度的位置定义为下一最优视点的参考位置,通过比较左右参考位置下所获可视面积的大小,取大者所在位置作为下一视点的最终位置.最后通过对实体模型重建,验证了文中方法的可行性及有效性.     

相位矩不变量用于评价基于图像的三维重建结果

基于图像的三维重建的评价一般要在三维扫描仪获取的精确三维模型上进行,成本高、效率低,且不适用于大型的物体或者大规模的场景.针对该问题,提出一种基于图像相位信息不变性的评价方法.首先获取物体的三维重建模型在某视点下的二维投影图;然后对该投影图以及真实物体在相近视点下的图像进行边缘提取;再对2幅边缘图像计算文中提出的相位矩不变量,以构造特征向量;最后通过比较特征向量的距离来衡量重建结果的质量.实验结果表明了该方法对评价基于图像重建的三维面片模型的有效性和适用性.     

基于稀疏点云的多平面场景稠密重建

多平面场景是生活中常见的一种场景,然而由于该类场景中常常存在物体表面纹理缺乏和纹理重复的现象,导致从多视图像重建获得的三维点云数据中存在点云过于稀疏甚至孔洞等问题,进而导致以微面片拟合三维点云所得到的重建表面出现平面颠簸现象.针对这些问题,本文提出了一种基于稀疏点云的分段平面场景重建方法.首先,利用分层抽样代替随机抽样,改进了J-Linkage多模型估计算法;然后,利用该方法对稀疏点云进行多平面拟合,来获得场景的多平面模型;最后,将多平面模型和无监督的图像分割相结合,提取并重建场景中的平面区域.场景中的非平面部分用CMVS/PMVS(Clustering views for multi-view stereo/patch-based multi-view stereo)算法重建.多平面模型估计的实验表明,改进的J-Linkage算法提高了模型估计的准确度.三维重建的实验证实,提出的重建方法在有效地克服孔洞和平面颠簸问题的同时,还能重建出完整平面区域.     

单图三维重构方法

在场景深度信息未知的情况下,由多视点成像产生的焦散失真是无法消除的。针对该问题,利用多视点成像中光束斜交的特性,提取失真图像中结构化图形中包含的场景深度信息,实现对单幅多视点图像的三维几何结构重建。重构适用范围包括所有已知函数形式的曲线,尤其为直线、圆等特殊几何图形的三维重构提供了专门的数学方法。     

基于网格变形的从图像重建三维人脸

从图像重建高质量三维人脸一直是计算机视觉和图形学的一个重要研究问题.不同于传统的基于立体匹配的窄基线多视几何和数据驱动的人脸形变方法,提出一种结合网格变形技术和立体视觉原理的、从图像重建高质量三维人脸模型方法.给定从不同视角拍摄的几幅人脸图像,基于健壮图像特征获得可靠的相机外部参数和稀疏三维点;在此基础上,提出一种结合几何细节保持和图像一致性约束的三维人脸变形算法重建三维人脸,通过对人脸模板的网格变形,使得变形人脸在多幅图像中的可见投影具有一致性的图像颜色强度.基于模板的人脸变形可以有效地解决三维模型成像中的遮挡问题,采用健壮估计法消除噪声、离群点和光照对目标函数收敛性的影响,对目标函数的多次非线性优化求解进一步改进了人脸重建的质量.采用合成人脸图像和真实人脸图像重建三维人脸的实验结果表明,文中算法可以从几幅宽基线图像重建高质量的三维人脸模型.     

Textured mesh surface reconstruction of large buildings with multi-view stereo

There are three main approaches for reconstructing 3D models of buildings. Laser scanning is accurate but expensive and limited by the laser’s range. Structure-from-motion (SfM) and multi-view stereo (MVS) recover 3D point clouds from multiple views of a building. MVS methods, especially patch-based MVS, can achieve higher density than do SfM methods. Sophisticated algorithms need to be applied to the point clouds to construct mesh surfaces. The recovered point clouds can be sparse in areas that lack features for accurate reconstruction, making recovery of complete surfaces difficult. Moreover, segmentation of the building’s surfaces from surrounding surfaces almost always requires some form of manual inputs, diminishing the ease of practical application of automatic 3D reconstruction algorithms. This paper presents an alternative approach for reconstructing textured mesh surfaces from point cloud recovered by patch-based MVS method. To a good first approximation, a building’s surfaces can be modeled by planes or curve surfaces which are fitted to the point cloud. 3D points are resampled on the fitted surfaces in an orderly pattern, whose colors are obtained from the input images. This approach is simple, inexpensive, and effective for reconstructing textured mesh surfaces of large buildings. Test results show that the reconstructed 3D models are sufficiently accurate and realistic for 3D visualization in various applications.     

Multi-view Performance Capture of Surface Details

This paper presents a novel approach to recover true fine surface detail of deforming meshes reconstructed from multi-view video. Template-based methods for performance capture usually produce a coarse-to-medium scale detail 4D surface reconstruction which does not contain the real high-frequency geometric detail present in the original video footage. Fine scale deformation is often incorporated in a second pass by using stereo constraints, features, or shading-based refinement. In this paper, we propose an alternative solution to this second stage by formulating dense dynamic surface reconstruction as a global optimization problem of the densely deforming surface. Our main contribution is an implicit representation of a deformable mesh that uses a set of Gaussian functions on the surface to represent the initial coarse mesh, and a set of Gaussians for the images to represent the original captured multi-view images. We effectively find the fine scale deformations for all mesh vertices, which maximize photo-temporal-consistency, by densely optimizing our model-to-image consistency energy on all vertex positions. Our formulation yields a smooth closed form energy with implicit occlusion handling and analytic derivatives. Furthermore, it does not require error-prone correspondence finding or discrete sampling of surface displacement values. We demonstrate our approach on a variety of datasets of human subjects wearing loose clothing and performing different motions. We qualitatively and quantitatively demonstrate that our technique successfully reproduces finer detail than the input baseline geometry.     

3D reconstruction for featureless scenes with curvature hints

We present a novel interactive framework for improving 3D reconstruction starting from incomplete or noisy results obtained through image-based reconstruction algorithms. The core idea is to enable the user to provide localized hints on the curvature of the surface, which are turned into constraints during an energy minimization reconstruction. To make this task simple, we propose two algorithms. The first is a multi-view segmentation algorithm that allows the user to propagate the foreground selection of one or more images both to all the images of the input set and to the 3D points, to accurately select the part of the scene to be reconstructed. The second is a fast GPU-based algorithm for the reconstruction of smooth surfaces from multiple views, which incorporates the hints provided by the user. We show that our framework can turn a poor-quality reconstruction produced with state of the art image-based reconstruction methods into a high- quality one.     

Estimation of the fundamental matrix from uncalibrated stereo hand images for 3D hand gesture recognition

All 3D hand models employed for hand gesture recognition so far use kinematic models of the hand. We propose to use computer vision models of the hand, and recover hand gestures using 3D reconstruction techniques. In this paper, we present a new method to estimate the epipolar geometry between two uncalibrated cameras from stereo hand images. We first segmented hand images using the RCE neural network based color segmentation algorithm and extracted edge points of fingers as points of interest, then match them based on the topological features of the hand. The fundamental matrix is estimated using a combination of techniques such as input data normalization, rank-2 constraint, linear criterion, nonlinear criterion as well as M-estimator. This method has been tested with real calibrated and uncalibrated images. The experimental comparison demonstrates the effectiveness and robustness of the method.     

Multi-view stereo in the Deep Learning Era: A comprehensive review

Multi-view stereo infers the 3D geometry from a set of images captured from several known positions and viewpoints. It is one of the most important components of 3D reconstruction. Recently, deep learning has been increasingly used to solve several 3D vision problems due to the predominating performance, including the multi-view stereo problem. This paper presents a comprehensive review, covering recent deep learning methods for multi-view stereo. These methods are mainly categorized into depth map based and volumetric based methods according to the 3D representation form, and representative methods are reviewed in detail. Specifically, the plane sweep based methods leveraging depth maps are presented following the stage of approaches, i.e. feature extraction, cost volume construction, cost volume regularization, depth map regression and post-processing. This review also summarizes several widely used datasets and their corresponding metrics for evaluation. Finally, several insightful observations and challenges are put forward enlightening future research directions.     

Automatic segmentation of point clouds from multi-view reconstruction using graph-cut

In multi-view reconstruction systems, the recovered point cloud often consists of numerous unwanted background points. We propose a graph-cut based method for automatically segmenting point clouds from multi-view reconstruction. Based on the observation that the object of interest is likely to be central to the intended multi-view images, our method requires no user interaction except two roughly estimated parameters of objects covering in the central area of images. The proposed segmentation process is carried out in two steps: first, we build a weighted graph whose nodes represent points and edges that connect each point to its k-nearest neighbors. The potentials of each point being object and background are estimated according to distances between its projections in images and the corresponding image centers. The pairwise potentials between each point and its neighbors are computed according to their positions, colors and normals. Graph-cut optimization is then used to find the initial binary segmentation of object and background points. Second, to refine the initial segmentation, Gaussian mixture models (GMMs) are created from the color and density features of points in object and background classes, respectively. The potentials of each point being object and background are re-calculated based on the learned GMMs. The graph is updated and the segmentation of point clouds is improved by graph-cut optimization. The second step is iterated until convergence. Our method requires no manual labeling points and employs available information of point clouds from multi-view systems. We test the approach on real-world data generated by multi-view reconstruction systems.     

基于移动平台的三维虚拟试发型系统实现及应用

发型对一个人的形象气质有非常重要的影响,由此所产生的消费也日益增多。消费 者更换发型不适合后无法恢复,因此在换发型和发色之前预览非常必要。已有的发型图册和基于 二维图像的试发应用都存在较为明显的缺陷,不能提供真实的效果。为此,结合虚拟现实技术和 基于单张照片的三维人脸重建算法,实现了一个适用于移动端的三维虚拟试发系统,试发效果更 真实,并可进行多角度观察、更换发色和配饰等。系统在移动平台上进行搭建,通过合理的框架 设计和资源管理进行优化,用户体验和实用性更佳,满足移动用户的需求。     

激光三维扫描数据的表面重建

对激光三维扫描系统获得的没有任何附加信息的轮廓线点云数据进行处理,首先采用求最大连通域的方法删除噪声点,利用设定相邻点连线夹角正切阈值的方法精简数据,然后采用基于局部切平面簇的方法对数据点云进行切平面的估算、法向量的调整和计算距离函数,用改进的MC方法输出三维网格,并且应用基于顶点的网格删除算法对三维网格进行简化,在估算切平面的时候采用新的估算原则,提高了重建速度,改善了重建效果,所表述的重建流程,成功地解决了激光扫描系统所得轮廓数据点的表面重建问题。     

Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video

3D video [IEEE Multimedia (1997) 18] is the ultimate image media recording dynamic visual events in the real world as is; it records time varying 3D object shape with high fidelity surface properties (i.e., color and texture). Its applications cover wide varieties of personal and social human activities: entertainment (e.g., 3D game and 3D TV), education (e.g., 3D animal picture books), sports (e.g., sport performance analysis), medicine (e.g., 3D surgery monitoring), culture (e.g., 3D archive of traditional dances), and so on. In this paper, we propose: (1) a PC cluster system for real-time reconstruction of dynamic 3D object action from multi-view video images, (2) a deformable 3D mesh model for reconstructing the accurate dynamic 3D object shape, and (3) an algorithm of rendering natural-looking texture on the 3D object surface from the multi-view video images. Experimental results with quantitative performance evaluations demonstrate the effectiveness of these methods in generating high fidelity 3D video from multi-view video images.