A unified method for appearance and geometry completion of point set surfaces

This paper presents a novel approach for appearance and geometry completion over point-sampled geometry. Based on the result of surface clustering and a given texture sample, we define a global texture energy function on the point set surface for direct texture synthesis. The color texture completion is performed by minimizing a constrained global energy using the existing surface texture on the surface as the input texture sample. We convert the problem of context-based geometry completion into a task of texture completion on the surface. The geometric detail is then peeled and converted into a piece of signed gray-scale texture on the base surface of the point set surface. We fill the holes on the base surface by smoothed extrapolation and the geometric details over these patches are reconstructed by a process of gray-scale texture completion. Experiments show that our method is flexible, efficient and easy to implement. It provides a practical texture synthesis and geometry completion tool for 3D point set surfaces.     

Geometry Images of Arbitrary Genus in the Spherical Domain

While existing spherical parameterization algorithms are limited to genus‐0 geometrical models, we believe a wide class of models of arbitrary genus can also benefit from the spherical domain. We present a complete and robust pipeline that can generate spherical geometry images from arbitrary genus surfaces where the holes are explicitly represented. The geometrical model, represented as a triangle mesh, is first made topologically equivalent to a sphere by cutting each hole along its generators, thus performing genus reduction. The resulting genus‐0 model is then parameterized on the sphere, where it is resampled in a way to preserve connectivity between holes and to reduce the visual impact of seams due to these holes. Knowing the location of each pair of boundary components in parametric space, our novel sampling scheme can automatically choose to scale down or completely eliminate the associated hole, depending on geometry image resolution, thus lowering the genus of the reconstructed model. We found our method to scale better than other geometry image algorithms for higher genus models. We illustrate our approach on remeshing, level‐of‐detail rendering, normal mapping and topology editing.     

一类周期多孔固体材料热传导性能计算的孔洞填充方法

对于一类周期多孔固体材料,提出了一种孔洞填充方法,用一种极低热导率的材料填充孔洞,将原本几何复杂的单相多孔区域的热传导边值问题转化为几何简单的多相无孔区域上的问题,借助于延拓定理给出了填充前后材料的热传导问题解和均匀化热导率的误差估计,对结果的分析表明可以用填充后材料热传导问题的双尺度解近似原孔洞问题的解,在最后的数值算例中,讨论了具有对称和非对称单胞构造的周期多孔固体的热传导边值问题,分别比较了多孔固体材料填充前后的均匀化热导率、温度和温度梯度解,结果表明孔洞填充方法的确可行。     

Pose analysis using spectral geometry

We propose a novel method to analyze a set of poses of 3D models that are represented with triangle meshes and unregistered. Different shapes of poses are transformed from the 3D spatial domain to a geometry spectrum domain that is defined by Laplace–Beltrami operator. During this space-spectrum transform, all near-isometric deformations, mesh triangulations and Euclidean transformations are filtered away. The different spatial poses from a 3D model are represented with near-isometric deformations; therefore, they have similar behaviors in the spectral domain. Semantic parts of that model are then determined based on the computed geometric properties of all the mapped vertices in the geometry spectrum domain. Semantic skeleton can be automatically built with joints detected as well. The Laplace–Beltrami operator is proved to be invariant to isometric deformations and Euclidean transformations such as translation and rotation. It also can be invariant to scaling with normalization. The discrete implementation also makes the Laplace–Beltrami operator straightforward to be applied on triangle meshes despite triangulations. Our method turns a rather difficult spatial problem into a spectral problem that is much easier to solve. The applications show that our 3D pose analysis method leads to a registration-free pose analysis and a high-level semantic part understanding of 3D shapes.     

A geometric approach to shape from defocus

We introduce a novel approach to shape from defocus, i.e., the problem of inferring the three-dimensional (3D) geometry of a scene from a collection of defocused images. Typically, in shape from defocus, the task of extracting geometry also requires deblurring the given images. A common approach to bypass this task relies on approximating the scene locally by a plane parallel to the image (the so-called equifocal assumption). We show that this approximation is indeed not necessary, as one can estimate 3D geometry while avoiding deblurring without strong assumptions on the scene. Solving the problem of shape from defocus requires modeling how light interacts with the optics before reaching the imaging surface. This interaction is described by the so-called point spread function (PSF). When the form of the PSF is known, we propose an optimal method to infer 3D geometry from defocused images that involves computing orthogonal operators which are regularized via functional singular value decomposition. When the form of the PSF is unknown, we propose a simple and efficient method that first learns a set of projection operators from blurred images and then uses these operators to estimate the 3D geometry of the scene from novel blurred images. Our experiments on both real and synthetic images show that the performance of the algorithm is relatively insensitive to the form of the PSF Our general approach is to minimize the Euclidean norm of the difference between the estimated images and the observed images. The method is geometric in that we reduce the minimization to performing projections onto linear subspaces, by using inner product structures on both infinite and finite-dimensional Hilbert spaces. Both proposed algorithms involve only simple matrix-vector multiplications which can be implemented in real-time.     

基于多视角的高精度三维人脸重建

提出了一种多阶段优化的方法来解决基于多视角图片在未知姿态、表情以及光照条件下的高精度三维人脸重建问题.首先,通过重新渲染合成的方法将参数化模型拟合到输入的多视角图片,然后在纹理域上求解一个光流问题来获取不同视角之间的对应关系.通过对应关系可以恢复出人脸的点云,并利用基于明暗恢复几何的方法来恢复人脸细节.在真实数据以及合成数据下的实验结果表明,文中方法能够恢复出带有几何细节的高精度的三维人脸模型,并且提高了现有方法的重建精度.     

Capture and Synthesis of 3D Surface Texture

We present and compare five approaches for capturing, synthesising and relighting real 3D surface textures. Unlike 2D texture synthesis techniques they allow the captured textures to be relit using illumination conditions that differ from those of the original. We adapted a texture quilting method due to Efros and combined this with five different relighting representations, comprising: a set of three photometric images; surface gradient and albedo maps; polynomial texture maps; and two eigen based representations using 3 and 6 base images.We used twelve real textures to perform quantitative tests on the relighting methods in isolation. We developed a qualitative test for the assessment of the complete synthesis systems. Ten observers were asked to rank the images obtained from the five methods using five real textures. Statistical tests were applied to the rankings.The six-base-image eigen method produced the best quantitative relighting results and in particular was better able to cope with specular surfaces. However, in the qualitative tests there were no significant performance differences detected between it and the other two top performers. Our conclusion is therefore that the cheaper gradient and three-base-image eigen methods should be used in preference, especially where the surfaces are Lambertian or near Lambertian.     

三维图像中边界曲面的抽取与半透明可视化

已有算法在重构及可视化梯度值从高到低变化且包含弱边界的边界曲面时无法分辨弱边界与噪声碎片。针对上述问题,提出边界曲面半透明显示方法。采用半透明显示技术显示边界曲面及附着在其上的小碎片,利用人的智能及知识在观察可视化结果时对小碎片及弱边界进行有效的区分。结果证明,该方法有助于正确理解及完整显示三维图像中梯度值从高到低变化的边界曲面,为进一步的交互式操作去除小碎片提供依据。     

A 3D model perceptual feature metric based on global height field

Human visual attention system tends to be attracted to perceptual feature points on 3D model surfaces. However, purely geometric-based feature metrics may be insufficient to extract perceptual features, because they tend to detect local structure details. Intuitively, the perceptual importance degree of vertex is associated with the height of its geometry position between original model and a datum plane. So, we propose a novel and straightforward method to extract perceptually important points based on global height field. Firstly, we construct spectral domain using Laplace–Beltrami operator, and we perform spectral synthesis to reconstruct a rough approximation of the original model by adopting low-frequency coefficients, and make it as the 3D datum plane. Then, to build global height field, we calculate the Euclidean distance between vertex geometry position on original surface and the one on 3D datum plane. Finally, we set a threshold to extract perceptual feature vertices. We implement our technique on several 3D mesh models and compare our algorithm to six state-of-the-art interest points detection approaches. Experimental results demonstrate that our algorithm can accurately capture perceptually important points on arbitrary topology 3D model.     

Data‐Driven Sparse Priors of 3D Shapes

We present a sparse optimization framework for extracting sparse shape priors from a collection of 3D models. Shape priors are defined as point‐set neighborhoods sampled from shape surfaces which convey important information encompassing normals and local shape characterization. A 3D shape model can be considered to be formed with a set of 3D local shape priors, while most of them are likely to have similar geometry. Our key observation is that the local priors extracted from a family of 3D shapes lie in a very low‐dimensional manifold. Consequently, a compact and informative subset of priors can be learned to efficiently encode all shapes of the same family. A comprehensive library of local shape priors is first built with the given collection of 3D models of the same family. We then formulate a global, sparse optimization problem which enforces selecting representative priors while minimizing the reconstruction error. To solve the optimization problem, we design an efficient solver based on the Augmented Lagrangian Multipliers method (ALM). Extensive experiments exhibit the power of our data‐driven sparse priors in elegantly solving several high‐level shape analysis applications and geometry processing tasks, such as shape retrieval, style analysis and symmetry detection.     

Edge structure preserving 3D image denoising by local surface approximation

In various applications, including magnetic resonance imaging (MRI) and functional MRI (fMRI), 3D images are becoming increasingly popular. To improve the reliability of subsequent image analyses, 3D image denoising is often a necessary preprocessing step, which is the focus of the current paper. In the literature, most existing image denoising procedures are for 2D images. Their direct extensions to 3D cases generally cannot handle 3D images efficiently because the structure of a typical 3D image is substantially more complicated than that of a typical 2D image. For instance, edge locations are surfaces in 3D cases which would be much more challenging to handle compared to edge curves in 2D cases. We propose a novel 3D image denoising procedure in this paper, based on local approximation of the edge surfaces using a set of surface templates. An important property of this method is that it can preserve edges and major edge structures (e.g., intersections of two edge surfaces and pointed corners). Numerical studies show that it works well in various applications.     

Adjoint sensitivity related to geometric parameters for mid-high frequency range vibroacoustics

Shape optimization issues under vibroacoustic criteria are considered in present paper. The adjoint-based gradient method was developed to minimize the energy density inside a cavity by changing its geometry parameters. The adjoint equations give the sensitivity information, which is subsequently used in a gradient-based minimization of a prescribed cost functional that models the energy density via Simplified Energy Method. Objective function is obtained by mapping 3D cavity surface on a 2D domain with the help of transformation function. Thus, the remeshing of the geometry is avoided and smoothness of the solution is reached. The optimization process is based on adjoint calculation of the gradient that leads to analytical expressions of the directional derivatives without additional computational cost. The proposed method is validated for the case of rectangular cavity modeled with patches of Bezier surfaces.     

Scene-consistent detection of feature points in video sequences

Detection of feature points in images is an important preprocessing stage for many algorithms in Computer Vision. We address the problem of detection of feature points in video sequences of 3D scenes, which could be mainly used for obtaining scene correspondence. The main feature we use is the zero crossing of the intensity gradient argument. We analytically show that this local feature corresponds to specific constraints on the local 3D geometry of the scene, thus ensuring that the detected points are based on real 3D features. We present a robust algorithm that tracks the detected points along a video sequence, and suggest some criteria for quantitative evaluation of such algorithms. These criteria serve in a comparison of the suggested operator with four other feature trackers. The suggested criteria are generic and could serve other researchers as well for performance evaluation of stable point detectors.     

Normal improvement for point rendering

Models created from 3D scanners are becoming more prevalent as the demand for realistic geometry grows and scanners become more common. Unfortunately, scanned models are invariably noisy. This noise corrupts both samples' positions and normals. Our proposed method for improving normals is derived from a feature-preserving geometry filter. Many such filters are available, most operating on models represented as triangle meshes. We argue that for point rendering, removing noise from normals is more important than removing noise from geometry, because normals have a greater impact on the model's perceived quality. Two approaches for smoothing point models have been proposed. Point set surfaces estimate smoothed normals and geometry by least-squares fitting to locally weighted neighborhoods. The spectral processing method creates a local height field, which is then filtered and resampled. The former method is not feature preserving, while the latter requires resampling to a regular grid, which can degrade features. Our method is novel in that it preserves features and doesn't require resampling.     

Peek‐in‐the‐Pic: Flying Through Architectural Scenes From a Single Image*

Many casually taken ‘tourist’ photographs comprise of architectural objects like houses, buildings, etc. Reconstructing such 3D scenes captured in a single photograph is a very challenging problem. We propose a novel approach to reconstruct such architectural scenes with minimal and simple user interaction, with the goal of providing 3D navigational capability to an image rather than acquiring accurate geometric detail. Our system, Peek‐in‐the‐Pic, is based on a sketch‐based geometry reconstruction paradigm. Given an image, the user simply traces out objects from it. Our system regards these as perspective line drawings, automatically completes them and reconstructs geometry from them. We make basic assumptions about the structure of traced objects and provide simple gestures for placing additional constraints. We also provide a simple sketching tool to progressively complete parts of the reconstructed buildings that are not visible in the image and cannot be automatically completed. Finally, we fill holes created in the original image when reconstructed buildings are removed from it, by automatic texture synthesis. Users can spend more time using interactive texture synthesis for further refining the image. Thus, instead of looking at flat images, a user can fly through them after some simple processing. Minimal manual work, ease of use and interactivity are the salient features of our approach.     

Capture and fusion of 3d surface texture

Image fusion is a process that multiple images of a scene are combined to form a single image. The aim of image fusion is to preserve the full content and retain important features of each original image. In this paper, we propose a novel approach based on wavelet transform to capture and fusion of real-world rough surface textures, which are commonly used in multimedia applications and referred to as3D surface texture. These textures are different from 2D textures as their appearances can vary dramatically with different illumination conditions due to complex surface geometry and reflectance properties. In our approach, we first extract gradient/height and albedo maps from sample 3D surface texture images as their representation. Then we measure saliency of wavelet coefficients of these 3D surface texture representations. The saliency values reflect the meaningful content of the wavelet coefficients and are consistent with human visual perception. Finally we fuse the gradient/height and albedo maps based on the measured saliency values. This novel scheme aims to preserve the original texture patterns together with geometry and reflectance characteristics from input images. Experimental results show that the proposed approach can not only capture and fuse 3D surface texture under arbitrary illumination directions, but also has the ability to retain the surface geometry properties and preserve perceptual features in the original images.     

Local Shape from Mirror Reflections

We study the problem of recovering the 3D shape of an unknown smooth specular surface from a single image. The surface reflects a calibrated pattern onto the image plane of a calibrated camera. The pattern is such that points are available in the image where position, orientations, and local scale may be measured (e.g. checkerboard). We first explore the differential relationship between the local geometry of the surface around the point of reflection and the local geometry in the image.We then study the inverse problem and give necessary and sufficient conditions for recovering surface position and shape.We prove that surface position and shape up to third order can be derived as a function of local position, orientation and local scale measurements in the image when two orientations are available at the same point (e.g. a corner). Information equivalent to scale and orientation measurements can be also extracted from the reflection of a planar scene patch of arbitrary geometry, provided that the reflections of (at least) 3 distinctive points may be identified.We validate our theoretical results with both numerical simulations and experiments with real surfaces.