Continuous Global Optimization in Multiview 3D Reconstruction

In this article, we introduce a new global optimization method to the field of multiview 3D reconstruction. While global minimization has been proposed in a discrete formulation in form of the maxflow-mincut framework, we suggest the use of a continuous convex relaxation scheme. Specifically, we propose to cast the problem of 3D shape reconstruction as one of minimizing a spatially continuous convex functional. In qualitative and quantitative evaluation we demonstrate several advantages of the proposed continuous formulation over the discrete graph cut solution. Firstly, geometric properties such as weighted boundary length and surface area are represented in a numerically consistent manner: The continuous convex relaxation assures that the algorithm does not suffer from metrication errors in the sense that the reconstruction converges to the continuous solution as the spatial resolution is increased. Moreover, memory requirements are reduced, allowing for globally optimal reconstructions at higher resolutions. We study three different energy models for multiview reconstruction, which are based on a common variational template unifying regional volumetric terms and on-surface photoconsistency. The three models use data measurements at increasing levels of sophistication. While the first two approaches are based on a classical silhouette-based volume subdivision, the third one relies on stereo information to define regional costs. Furthermore, this scheme is exploited to compute a precise photoconsistency measure as opposed to the classical estimation. All three models are compared on standard data sets demonstrating their advantages and shortcomings. For the third one, which gives the most accurate results, a more exhaustive qualitative and quantitative evaluation is presented.     

A Surface Reconstruction Method Using Global Graph Cut Optimization

Surface reconstruction from multiple calibrated images has been mainly approached using local methods, either as a continuous optimization problem driven by level sets, or by discrete volumetric methods such as space carving. We propose a direct surface reconstruction approach which starts from a continuous geometric functional that is minimized up to a discretization by a global graph-cut algorithm operating on a 3D embedded graph. The method is related to the stereo disparity computation based on graph-cut formulation, but fundamentally different in two aspects. First, existing stereo disparity methods are only interested in obtaining layers of constant disparity, while we focus on high resolution surface geometry. Second, most of the existing graph-cut algorithms only reach approximate solutions, while we guarantee a global minimum. The whole procedure is consistently incorporated into a voxel representation that handles both occlusions and discontinuities. We demonstrate our algorithm on real sequences, yielding remarkably detailed surface geometry up to 1/10th of a pixel. Author has worked on this project during his Ph. D. at ARTIS     

A surface based approach to recognition of geometric features for quality freeform surface machining

The paper presents a surface-based approach for geometric feature recognition for the purpose of automating the process planning of freeform surface machining. The proposed approach consists of the following four steps for recognition of the geometric features: conversion and preprocessing of the surface geometry data, subdivision of NURBS surface, reconstruction of surface orientation areas, and recognition of geometric features. The proposed scheme assumes that the input geometry data form is based on an IGES CAD model and the surface model can be represented in the form of trimmed NURBS surfaces. The connectivity relations of the product surfaces are analyzed and each surface is subdivided into orientation regions based on the surface normal vector over a certain point density grid, and then all the connected regions with the same orientation type are grouped into surface orientation areas. After that, the geometric feature will be recognized through the analysis of area connectivity and relationship. The paper describes the developed algorithms on surface orientation region subdivision, surface orientation area reconstruction, and geometric feature recognition. The verified feasibility study of the developed method is also presented.     

Efficient surface reconstruction using generalized coulomb potentials

We propose a novel, geometrically adaptive method for surface reconstruction from noisy and sparse point clouds, without orientation information. The method employs a fast convection algorithm to attract the evolving surface towards the data points. The force field in which the surface is convected is based on generalized Coulomb potentials evaluated on an adaptive grid (i.e., an octree) using a fast, hierarchical algorithm. Formulating reconstruction as a convection problem in a velocity field generated by Coulomb potentials offers a number of advantages. Unlike methods which compute the distance from the data set to the implicit surface, which are sensitive to noise due to the very reliance on the distance transform, our method is highly resilient to shot noise since global, generalized Coulomb potentials can be used to disregard the presence of outliers due to noise. Coulomb potentials represent long-range interactions that consider all data points at once, and thus they convey global information which is crucial in the fitting process. Both the spatial and temporal complexities of our spatially-adaptive method are proportional to the size of the reconstructed object, which makes our method compare favorably with respect to previous approaches in terms of speed and flexibility. Experiments with sparse as well as noisy data sets show that the method is capable of delivering crisp and detailed yet smooth surfaces.     

Skeleton-based Variational Mesh Deformations

In this paper, a new free-form shape deformation approach is proposed. We combine a skeleton-based mesh deformation technique with discrete differential coordinates in order to create natural-looking global shape deformations. Given a triangle mesh, we first extract a skeletal mesh, a two-sided Voronoibased approximation of the medial axis. Next the skeletal mesh is modified by free-form deformations. Then a desired global shape deformation is obtained by reconstructing the shape corresponding to the deformed skeletal mesh. The reconstruction is based on using discrete differential coordinates. Our method preserves fine geometric details and original shape thickness because of using discrete differential coordinates and skeleton-based deformations. We also develop a new mesh evolution technique which allow us to eliminate possible global and local self-intersections of the deformed mesh while preserving fine geometric details. Finally, we present a multi-resolution version of our approach in order to simplify and accelerate the deformation process. In addition, interesting links between the proposed free-form shape deformation technique and classical and modern results in the differential geometry of sphere congruences are established and discussed.     

一种用于指纹方向场估计的网格插值模型

指纹方向场的估计是指纹识别预处理算法中的重要环节,对算法识别效率起到关键作用。本文提出了一种网格插值模型,该模型以指纹奇异点为中心,将指纹平面做网格划分,利用插值算法建立了方向场与指纹奇异点之间的非线性关系。模型中利用了指纹的全局信息来调整网格点的值,使得它与传统的基于局部信息的方向场算法有本质的区别。在FVC2002 和FVC2004 指纹数据库上的实验结果表明,该模型比传统算法具有更高的准确性和鲁棒性,同时对于低质量的指纹图像,仍然能够给出很好的方向场估计。     

Eulerian shape design sensitivity analysis and optimization with a fixed grid

Conventional shape optimization based on the finite element method uses Lagrangian representation in which the finite element mesh moves according to shape change, while modern topology optimization uses Eulerian representation. In this paper, an approach to shape optimization using Eulerian representation such that the mesh distortion problem in the conventional approach can be resolved is proposed. A continuum geometric model is defined on the fixed grid of finite elements. An active set of finite elements that defines the discrete domain is determined using a procedure similar to topology optimization, in which each element has a unique shape density. The shape design parameter that is defined on the geometric model is transformed into the corresponding shape density variation of the boundary elements. Using this transformation, it has been shown that the shape design problem can be treated as a parameter design problem, which is a much easier method than the former. A detailed derivation of how the shape design velocity field can be converted into the shape density variation is presented along with sensitivity calculation. Very efficient sensitivity coefficients are calculated by integrating only those elements that belong to the structural boundary. The accuracy of the sensitivity information is compared with that derived by the finite difference method with excellent agreement. Two design optimization problems are presented to show the feasibility of the proposed design approach.     

一种基于AKAZE算法的多视图几何三维重建方法

针对增量式运动恢复结构算法在多视图几何三维重建算法中运行效率低的问题,提出了一种基于AKAZE算法的多视图几何三维重建方法。首先对利用摄像机获得的目标图像使用AKAZE算法检测特征并匹配,并使用随机抽取一致性算法和三视图约束剔除弱匹配图像。然后根据匹配图间的相对位姿参数,通过最小二乘法解算全局旋转参数,并利用三视图约束关系求解全局位移参数。最后进行一次光束法平差优化。实验结果表明,该算法在改善重建效果的基础上提高了处理效率,能够满足快速处理的需求。     

二阶多面体网格中关键特征控制的表面重建技术

针对基于二阶多节点多面体网格的表面重建过程中存在的准确拓扑及绘制、传输代价等问题,提出了一种基于关键特征控制的表面重建技术.研究并分析了二阶多节点多面体单元等参插值函数的性质特征,在网格单元棱边插值计算曲面轮廓点,在网格表面及体内提取曲面的几何特征关键点;根据3类插值关键点间的逻辑关系制定了令拓扑准确唯一的面片三角化规则及修复策略,设计了基于关键点的三角面片压缩索引结构.实验结果表明,该方法可准确计算并描述基于二阶多节点多面体网格单元的曲面几何拓扑结构,反映网格单元内部面片的真实凹凸性质,克服了拓扑二义性,具备对不同精度要求的适应性,并有效降低了绘制与传输代价.     

Multi-view Scene Flow Estimation: A View Centered Variational Approach

We present a novel method for recovering the 3D structure and scene flow from calibrated multi-view sequences. We propose a 3D point cloud parametrization of the 3D structure and scene flow that allows us to directly estimate the desired unknowns. A unified global energy functional is proposed to incorporate the information from the available sequences and simultaneously recover both depth and scene flow. The functional enforces multi-view geometric consistency and imposes brightness constancy and piecewise smoothness assumptions directly on the 3D unknowns. It inherently handles the challenges of discontinuities, occlusions, and large displacements. The main contribution of this work is the fusion of a 3D representation and an advanced variational framework that directly uses the available multi-view information. This formulation allows us to advantageously bind the 3D unknowns in time and space. Different from optical flow and disparity, the proposed method results in a nonlinear mapping between the images’ coordinates, thus giving rise to additional challenges in the optimization process. Our experiments on real and synthetic data demonstrate that the proposed method successfully recovers the 3D structure and scene flow despite the complicated nonconvex optimization problem.     

Combinatorial manifold mesh reconstruction and optimization from unorganized points with arbitrary topology

In this paper, a novel approach is proposed to reliably reconstruct the geometric shape of a physically existing object based on unorganized point cloud sampled from its boundary surface. The proposed approach is composed of two steps. In the first step, triangle mesh structure is reconstructed as a continuous manifold surface by imposing explicit relationship among the discrete data points. For efficient reconstruction, a growing procedure is employed to build the 2-manifold directly without intermediate 3D representation. Local and global topological operations with ensured completeness and soundness are defined to incrementally construct the 2-manifold with arbitrary topology. In addition, a novel criterion is proposed to control the growing process for ensured geometric integrity and automatic boundary detection with a non-metric threshold. The reconstructed manifold surface captures the object topology with the built-in combinatorial structure and approximates the object geometry to the first order. In the second step, new methods are proposed to efficiently obtain reliable curvature estimation for both the object surface and the reconstructed mesh surface. The combinatorial structure of the triangle mesh is then optimized by changing its local topology to minimize the curvature difference between the two surfaces. The optimized triangle mesh achieves second order approximation to the object geometry and can serve as a basis for many applications including virtual reality, computer vision, and reverse engineering.     

A comparison of global optimization algorithms applied to a ride comfort optimization problem

This study presents a comparison of global optimization algorithms applied to an industrial engineering optimization problem. Three global stochastic optimization algorithms using continuous variables, i.e. the domain elimination method, the zooming method and controlled random search, have been applied to a previously studied ride comfort optimization problem. Each algorithm is executed three times and the total number of objective function evaluations needed to locate a global optimum is averaged and used as a measure of efficiency. The results show that the zooming method, with a proposed modification, is most efficient in terms of number of objective function evaluations and ability to locate the global optimum. Each design variable is thereafter given a set of discrete values and two optimization algorithms using discrete variables, i.e. a genetic algorithm and simulated annealing, are applied to the discrete ride comfort optimization problem. The results show that the genetic algorithm is more efficient than the simulated annealing algorithm for this particular optimization problem.     

Linear Surface Reconstruction from Discrete Fundamental Forms on Triangle Meshes

We present a linear algorithm to reconstruct the vertex coordinates for a surface mesh given its edge lengths and dihedral angles, unique up to rotation and translation. A local integrability condition for the existence of an immersion of the mesh in 3D Euclidean space is provided, mirroring the fundamental theorem of surfaces in the continuous setting (i.e. Gauss's equation and the Mainardi–Codazzi equations) if we regard edge lengths as the discrete first fundamental form and dihedral angles as the discrete second fundamental form. The resulting sparse linear system to solve for the immersion is derived from the convex optimization of a quadratic energy based on a lift from the immersion in the 3D Euclidean space to the 6D rigid motion space. This discrete representation and linear reconstruction can benefit a wide range of geometry processing tasks such as surface deformation and shape analysis. A rotation‐invariant surface deformation through point and orientation constraints is demonstrated as well.     

Multi-view Performance Capture of Surface Details

This paper presents a novel approach to recover true fine surface detail of deforming meshes reconstructed from multi-view video. Template-based methods for performance capture usually produce a coarse-to-medium scale detail 4D surface reconstruction which does not contain the real high-frequency geometric detail present in the original video footage. Fine scale deformation is often incorporated in a second pass by using stereo constraints, features, or shading-based refinement. In this paper, we propose an alternative solution to this second stage by formulating dense dynamic surface reconstruction as a global optimization problem of the densely deforming surface. Our main contribution is an implicit representation of a deformable mesh that uses a set of Gaussian functions on the surface to represent the initial coarse mesh, and a set of Gaussians for the images to represent the original captured multi-view images. We effectively find the fine scale deformations for all mesh vertices, which maximize photo-temporal-consistency, by densely optimizing our model-to-image consistency energy on all vertex positions. Our formulation yields a smooth closed form energy with implicit occlusion handling and analytic derivatives. Furthermore, it does not require error-prone correspondence finding or discrete sampling of surface displacement values. We demonstrate our approach on a variety of datasets of human subjects wearing loose clothing and performing different motions. We qualitatively and quantitatively demonstrate that our technique successfully reproduces finer detail than the input baseline geometry.     

多视三维重构中连续能量模型的凸优化

利用融合了轮廓线及体视的序列图像信息,提出了一个面向多视三维重构的稳健能量模型;为了适配于可视性约束,提出一种针对该能量模型的连续全局优化方法;为了保证栅格连通性选择的一致性及独立性,实施了全局连续优化的超松弛离散化。实例证明,该方法的实用性好,极大地减少了算法处理的内存开销,实现了在更高分辨率上有效的多视重构。     

Quasiconvex optimization for robust geometric reconstruction

Geometric reconstruction problems in computer vision are often solved by minimizing a cost function that combines the reprojection errors in the 2D images. In this paper, we show that, for various geometric reconstruction problems, their reprojection error functions share a common and quasiconvex formulation. Based on the quasiconvexity, we present a novel quasiconvex optimization framework in which the geometric reconstruction problems are formulated as a small number of small-scale convex programs that are ready to solve. Our final reconstruction algorithm is simple and has intuitive geometric interpretation. In contrast to existing local minimization approaches, our algorithm is deterministic and guarantees a predefined accuracy of the minimization result.The quasiconvexity also provides an intuitive method to handle directional uncertainties and outliers in measurements. When there are outliers in the measurements, our method provides a mechanism to locate the global minimum of a robust error function. For large scale problems and when computational resources are constrained, we provide an efficient approximation that gives a good upper bound (but not global minimum) on the reconstruction error. We demonstrate the effectiveness of our algorithm by experiments on both synthetic and real data.     

叶片破损区域边界的自动提取算法研究

为了提高叶片的维修效率,提出一种叶片破损区域边界的自动提取算法．该算法主要包括两部分：区域分割和边界重构．在区域分割算法中,通过曲率估算、种子栅格选取和区域生长等步骤将破损区域的数据点从原始点云中分离出来;在边界重构算法中,通过多边形细分和离散光顺算法重构破损区域边界．应用实例表明,文中算法效率高、稳定性好,能够从散乱的点云数据中直接获取高质量的区域边界信息。     

Reconstruction of 3D Object Meshes from Silhouette Images

In this work we propose a method for computing mesh representations of 3D objects reconstructed from a set of silhouette images. Our method is based on the polygonization of volumetric reconstructions by using a modified version of the dual contouring method. In order to apply dual contouring on volumetric reconstruction from silhouettes we devised a method that is able to determine the discrete topology of the surface in relation to the octree cells. We also developed a new scheme for computing hermitian data representing the intersections of conic volumes with the octree cells and their corresponding normals with subpixel accuracy. Due to the discrete and extremely noisy nature of the data used in the reconstruction we had to devise a different criterion for mesh simplification that applies topological consistency tests only when the geometric error measure is beyond a given tolerance. We present results of the application of the proposed method in the extraction of a mesh corresponding to the surface of objects of a real scene.     

Seamless Montage for Texturing Models

We present an automatic method to recover high‐resolution texture over an object by mapping detailed photographs onto its surface. Such high‐resolution detail often reveals inaccuracies in geometry and registration, as well as lighting variations and surface reflections. Simple image projection results in visible seams on the surface. We minimize such seams using a global optimization that assigns compatible texture to adjacent triangles. The key idea is to search not only combinatorially over the source images, but also over a set of local image transformations that compensate for geometric misalignment. This broad search space is traversed using a discrete labeling algorithm, aided by a coarse‐to‐fine strategy. Our approach significantly improves resilience to acquisition errors, thereby allowing simple and easy creation of textured models for use in computer graphics.     

基于点云配准的人脸三维尺寸测量

接触式人脸三维尺寸测量易损坏表面特征,依赖于特征点标定,常含冗余信息.针对该问题,提出一种基于结构光与多视图图像点云配准的非接触式人脸三维尺寸测量方法.首先利用改进的迭代最近点算法建立转换函数,求出尺度因子、旋转矩阵和平移向量;然后基于模糊C均值算法对人脸面部进行聚类分割以获得候选区域;针对人脸表面离散点云不平整问题,...