Real‐time Bas‐Relief Generation from Depth‐and‐Normal Maps on GPU

To design a bas‐relief from a 3D scene is an inherently interactive task in many scenarios. The user normally needs to get instant feedback to select a proper viewpoint. However, current methods are too slow to facilitate this interaction. This paper proposes a two‐scale bas‐relief modeling method, which is computationally efficient and easy to produce different styles of bas‐reliefs. The input 3D scene is first rendered into two textures, one recording the depth information and the other recording the normal information. The depth map is then compressed to produce a base surface with level‐of‐depth, and the normal map is used to extract local details with two different schemes. One scheme provides certain freedom to design bas‐reliefs with different visual appearances, and the other provides a control over the level of detail. Finally, the local feature details are added into the base surface to produce the final result. Our approach allows for real‐time computation due to its implementation on graphics hardware. Experiments with a wide range of 3D models and scenes show that our approach can effectively generate digital bas‐reliefs in real time.     

Computer‐assisted Relief Modelling: A Comprehensive Survey

As an art form between drawing and sculpture, relief has been widely used in a variety of media for signs, narratives, decorations and other purposes. Traditional relief creation relies on both professional skills and artistic expertise, which is extremely time‐consuming. Recently, automatic or semi‐automatic relief modelling from a 3D object or a 2D image has been a subject of interest in computer graphics. Various methods have been proposed to generate reliefs with few user interactions or minor human efforts, while preserving or enhancing the appearance of the input. This survey provides a comprehensive review of the advances in computer‐assisted relief modelling during the past decade. First, we provide an overview of relief types and their art characteristics. Then, we introduce the key techniques of object‐space methods and image‐space methods respectively. Advantages and limitations of each category are discussed in details. We conclude the report by discussing directions for possible future research.     

Reconstructing 3D Shape,Albedo and Illumination from a Single Face Image

The morphable model has been employed to efficiently describe 3D face shape and the associated albedo with a reduced set of basis vectors. The spherical harmonics (SH) model provides a compact basis to well approximate the image appearance of a Lambertian object under different illumination conditions. Recently, the SH and morphable models have been integrated for 3D face shape reconstruction. However, the reconstructed 3D shape is either inconsistent with the SH bases or obtained just from landmarks only. In this work, we propose a geometrically consistent algorithm to reconstruct the 3D face shape and the associated albedo from a single face image iteratively by combining the morphable model and the SH model. The reconstructed 3D face geometry can uniquely determine the SH bases, therefore the optimal 3D face model can be obtained by minimizing the error between the input face image and a linear combination of the associated SH bases. In this way, we are able to preserve the consistency between the 3D geometry and the SH model, thus refining the 3D shape reconstruction recursively. Furthermore, we present a novel approach to recover the illumination condition from the estimated weighting vector for the SH bases in a constrained optimization formulation independent of the 3D geometry. Experimental results show the effectiveness and accuracy of the proposed face reconstruction and illumination estimation algorithm under different face poses and multiple‐light‐source illumination conditions.     

基于3维模型的数字浮雕生成技术

目的 浮雕是雕塑艺术的一种,根据其空间结构和用途的不同分为高浮雕、浅浮雕和凹浮雕3类。随着数字化技术和3D打印技术的发展,数字化浮雕的生成技术已经成为近年来计算机图形学领域的研究热点之一,从3维模型生成浮雕以其真实自然的效果成为浮雕生成的主要方法之一。为了使即将进入该领域的学者尽快了解该方法的现状和发展趋势,本文对3种类型的浮雕生成技术进行了系统的综述。方法 介绍了3种类型的浮雕生成技术,着重比较分析了基于3维网格模型的数字浅浮雕生成过程中的关键技术,存在问题及解决方案。针对复杂3维网格模型在生成数字凹浮雕过程中存在的部分细节信息丢失、特征线类型体现形式不完善、线条与形体间的过渡尚未解决、生成浮雕效果不自然等具体问题,提出了适用于3维复杂网格模型生成数字凹浮雕的研究方案。同时,从角色动画序列出发,对最优浮雕的生成技术进行了探讨,探讨结合信息熵理论计算选择最佳动作及观察视角的场景,还原艺术家的创作过程,为适用于面向3维打印的用户浮雕产品定制服务提供了可行的解决方案。结果 基于3维模型的浮雕生成方法是生成数字浮雕的一种重要方法,如何通过压缩和细节保持相关算法得到效果自然的浮雕模型一直是研究者们研究的热点问题。结论 虽然由3维模型生成数字浮雕是一种行之有效的方法,但是仍存在细节信息丢失、线条过渡不自然、特征线类型不完善等几个值得继续研究的问题,另外一个值得研究的问题就是如何智能地从3维动画序列生成浮雕。     

实时交互的带浮雕纹理的三维模型构建方法

为了快速生成带浮雕纹理的三维模型,提出一种实时交互的浮雕纹理模型构建方法。方法分两步：第一步,将生成浮雕的源模型或图像转换为初始深度图,并进一步转换为梯度图,再通过梯度域的压缩、过滤,求解线性方程重建出整体连续的浮雕深度图;第二步,借助基于网格求交的浮雕纹理映射算法将浮雕深度图贴在目标模型表面,并通过移动、旋转、缩放等操作实时在目标模型三维空间上修改浮雕效果,最终重建目标模型网格,生成浮雕纹理模型。实验表明,所提方法可快速实现在一个目标模型上生成凹浮雕、凸浮雕、多浮雕等效果,所得模型无需经过其他处理,可直接应用于3D打印,打印效果较好。     

Provably Good 2D Shape Reconstruction from Unorganized Cross‐Sections

This paper deals with the reconstruction of 2‐dimensional geometric shapes from unorganized 1‐dimensional cross‐sections. We study the problem in its full generality following the approach of Boissonnat and Memari [ [BM07] ] for the analogous 3D problem. We propose a new variant of this method and provide sampling conditions to guarantee that the output of the algorithm has the same topology as the original object and is close to it (for the Hausdorff distance).     

High Reliefs from 3D Scenes

We present a method for synthesizing high reliefs, a sculpting technique that attaches 3D objects onto a 2D surface within a limited depth range. The main challenges are the preservation of distinct scene parts by preserving depth discontinuities, the fine details of the shape, and the overall continuity of the scene. Bas relief depth compression methods such as gradient compression and depth range compression are not applicable for high relief production. Instead, our method is based on differential coordinates to bring scene elements to the relief plane while preserving depth discontinuities and surface details of the scene. We select a user‐defined number of attenuation points within the scene, attenuate these points towards the relief plane and recompute the positions of all scene elements by preserving the differential coordinates. Finally, if the desired depth range is not achieved we apply a range compression. High relief synthesis is semi‐automatic and can be controlled by user‐defined parameters to adjust the depth range, as well as the placement of the scene elements with respect to the relief plane.     

Object Completion using k‐Sparse Optimization

We present a new method for the completion of partial globally‐symmetric 3D objects, based on the detection of partial and approximate symmetries in the incomplete input dataset. In our approach, symmetry detection is formulated as a constrained sparsity maximization problem, which is solved efficiently using a robust RANSAC‐based optimizer. The detected partial symmetries are then reused iteratively, in order to complete the missing parts of the object. A global error relaxation method minimizes the accumulated alignment errors and a non‐rigid registration approach applies local deformations in order to properly handle approximate symmetry. Unlike previous approaches, our method does not rely on the computation of features, it uniformly handles translational, rotational and reflectional symmetries and can provide plausible object completion results, even on challenging cases, where more than half of the target object is missing. We demonstrate our algorithm in the completion of 3D scans with varying levels of partiality and we show the applicability of our approach in the repair and completion of heavily eroded or incomplete cultural heritage objects.     

Sketching and Composing Widgets for 3D Manipulation

We present an interface for 3D object manipulation in which standard transformation tools are replaced with transient 3D widgets invoked by sketching context‐dependent strokes. The widgets are automatically aligned to axes and planes determined by the user's stroke. Sketched pivot‐points further expand the interaction vocabulary. Using gestural commands, these basic elements can be assembled into dynamic, user‐constructed 3D transformation systems. We supplement precise widget interaction with techniques for coarse object positioning and snapping. Our approach, which is implemented within a broader sketch‐based modeling system, also integrates an underlying “widget history” to enable the fluid transfer of widgets between objects. An evaluation indicates that users familiar with 3D manipulation concepts can be taught how to efficiently use our system in under an hour.     

Computer‐Suggested Facial Makeup

Finding the best makeup for a given human face is an art in its own right. Experienced makeup artists train for years to be skilled enough to propose a best‐fit makeup for an individual. In this work we propose a system that automates this task. We acquired the appearance of 56 human faces, both without and with professional makeup. To this end, we use a controlled‐light setup, which allows to capture detailed facial appearance information, such as diffuse reflectance, normals, subsurface‐scattering, specularity, or glossiness. A 3D morphable face model is used to obtain 3D positional information and to register all faces into a common parameterization. We then define makeup to be the change of facial appearance and use the acquired database to find a mapping from the space of human facial appearance to makeup. Our main application is to use this mapping to suggest the best‐fit makeup for novel faces that are not in the database. Further applications are makeup transfer, automatic rating of makeup, makeup‐training, or makeup‐exaggeration. As our makeup representation captures a change in reflectance and scattering, it allows us to synthesize faces with makeup in novel 3D views and novel lighting with high realism. The effectiveness of our approach is further validated in a user‐study.     

Reconstructing Animated Meshes from Time‐Varying Point Clouds

In this paper, we describe a novel approach for the reconstruction of animated meshes from a series of time‐deforming point clouds. Given a set of unordered point clouds that have been captured by a fast 3‐D scanner, our algorithm is able to compute coherent meshes which approximate the input data at arbitrary time instances. Our method is based on the computation of an implicit function in ?⁴ that approximates the time‐space surface of the time‐varying point cloud. We then use the four‐dimensional implicit function to reconstruct a polygonal model for the first time‐step. By sliding this template mesh along the time‐space surface in an as‐rigid‐as‐possible manner, we obtain reconstructions for further time‐steps which have the same connectivity as the previously extracted mesh while recovering rigid motion exactly. The resulting animated meshes allow accurate motion tracking of arbitrary points and are well suited for animation compression. We demonstrate the qualities of the proposed method by applying it to several data sets acquired by real‐time 3‐D scanners.     

A Local/Global Approach to Mesh Parameterization

We present a novel approach to parameterize a mesh with disk topology to the plane in a shape‐preserving manner. Our key contribution is a local/global algorithm, which combines a local mapping of each 3D triangle to the plane, using transformations taken from a restricted set, with a global “stitch” operation of all triangles, involving a sparse linear system. The local transformations can be taken from a variety of families, e.g. similarities or rotations, generating different types of parameterizations. In the first case, the parameterization tries to force each 2D triangle to be an as‐similar‐as‐possible version of its 3D counterpart. This is shown to yield results identical to those of the LSCM algorithm. In the second case, the parameterization tries to force each 2D triangle to be an as‐rigid‐as‐possible version of its 3D counterpart. This approach preserves shape as much as possible. It is simple, effective, and fast, due to pre‐factoring of the linear system involved in the global phase. Experimental results show that our approach provides almost isometric parameterizations and obtains more shape‐preserving results than other state‐of‐the‐art approaches. We present also a more general “hybrid” parameterization model which provides a continuous spectrum of possibilities, controlled by a single parameter. The two cases described above lie at the two ends of the spectrum. We generalize our local/global algorithm to compute these parameterizations. The local phase may also be accelerated by parallelizing the independent computations per triangle.     

Hallucinating Stereoscopy from a Single Image

We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre‐segmented image. Rather than attempting full 3D reconstruction or accurate depth map recovery, we hallucinate a rough approximation of the scene's 3D model using a number of simple depth and occlusion cues and shape priors. We begin by depth‐sorting the segments, each of which is assumed to represent a separate object in the scene, resulting in a collection of depth layers. The shapes and textures of the partially occluded segments are then completed using symmetry and convexity priors. Next, each completed segment is converted to a union of generalized cylinders yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state‐of‐the‐art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi‐interactive and automatic single image depth recovery methods.     

Learning Style Compatibility Between Objects in a Real‐World 3D Asset Database

Large 3D asset databases are critical for designing virtual worlds, and using them effectively requires techniques for efficient querying and navigation. One important form of query is search by style compatibility: given a query object, find others that would be visually compatible if used in the same scene. In this paper, we present a scalable, learning‐based approach for solving this problem which is designed for use with real‐world 3D asset databases; we conduct experiments on 121 3D asset packages containing around 4000 3D objects from the Unity Asset Store. By leveraging the structure of the object packages, we introduce a technique to synthesize training labels for metric learning that work as well as human labels. These labels can grow exponentially with the number of objects, allowing our approach to scale to large real‐world 3D asset databases without the need for expensive human training labels. We use these synthetic training labels in a metric learning model that analyzes the in‐engine rendered appearance of an object—combining geometry, material, and texture—whereas prior work considers only object geometry, or disjoint geometry and texture features. Through an ablation experiment, we find that using this representation yields better results than using renders which lack texture, materiality, or both.     

Adaptive Space Deformations Based on Rigid Cells

We propose a new adaptive space deformation method for interactive shape modeling. A novel energy formulation based on elastically coupled volumetric cells yields intuitive detail preservation even under large deformations. By enforcing rigidity of the cells, we obtain an extremely robust numerical solver for the resulting nonlinear optimization problem. Scalability is achieved using an adaptive spatial discretization that is decoupled from the resolution of the embedded object. Our approach is versatile and easy to implement, supports thin-shell and solid deformations of 2D and 3D objects, and is applicable to arbitrary sample-based representations, such as meshes, triangle soups, or point clouds.     

On‐Site Example‐Based Material Appearance Acquisition

We present a novel example‐based material appearance modeling method suitable for rapid digital content creation. Our method only requires a single HDR photograph of a homogeneous isotropic dielectric exemplar object under known natural illumination. While conventional methods for appearance modeling require prior knowledge on the object shape, our method does not, nor does it recover the shape explicitly, greatly simplifying on‐site appearance acquisition to a lightweight photography process suited for non‐expert users. As our central contribution, we propose a shape‐agnostic BRDF estimation procedure based on binary RGB profile matching. We also model the appearance of materials exhibiting a regular or stationary texture‐like appearance, by synthesizing appropriate mesostructure from the same input HDR photograph and a mesostructure exemplar with (roughly) similar features. We believe our lightweight method for on‐site shape‐agnostic appearance acquisition presents a suitable alternative for a variety of applications that require plausible “rapid‐appearance‐modeling”.     

Symmetry‐Aware Template Deformation and Fitting

In this paper, we propose a new method for reconstructing 3D models from a noisy and incomplete 3D scan and a coarse template model. The main idea is to maintain characteristic high‐level features of the template that remain unchanged for different variants of the same type of object. As invariants, we chose the partial symmetry structure of the template model under Euclidian transformations, i.e. we maintain the algebraic structure of all reflections, rotations and translations that map the object partially to itself. We propose an optimization scheme that maintains continuous and discrete symmetry properties of this kind while registering a template against scan data using a deformable iterative closest points (ICP) framework with thin‐plate‐spline regularization. We apply our new deformation approach to a large number of example data sets and demonstrate that symmetry‐guided template matching often yields much more plausible reconstructions than previous variants of ICP.     

Retargeting 3D Objects and Scenes with a General Framework

In this paper, we introduce an interactive method suitable for retargeting both 3D objects and scenes. Initially, the input object or scene is decomposed into a collection of constituent components enclosed by corresponding control bounding volumes which capture the intra‐structures of the object or semantic grouping of objects in the 3D scene. The overall retargeting is accomplished through a constrained optimization by manipulating the control bounding volumes. Without inferring the intricate dependencies between the components, we define a minimal set of constraints that maintain the spatial arrangement and connectivity between the components to regularize the valid retargeting results. The default retargeting behavior can then be easily altered by additional semantic constraints imposed by users. This strategy makes the proposed method highly flexible to process a wide variety of 3D objects and scenes under an unified framework. In addition, the proposed method achieved more general structure‐preserving pattern synthesis in both object and scene levels. We demonstrate the effectiveness of our method by applying it to several complicated 3D objects and scenes.     

Mobile Surface Reflectometry

We present two novel mobile reflectometry approaches for acquiring detailed spatially varying isotropic surface reflectance and mesostructure of a planar material sample using commodity mobile devices. The first approach relies on the integrated camera and flash pair present on typical mobile devices to support free‐form handheld acquisition of spatially varying rough specular material samples. The second approach, suited for highly specular samples, uses the LCD panel to illuminate the sample with polarized second‐order gradient illumination. To address the limited overlap of the front facing camera's view and the LCD illumination (and thus limited sample size), we propose a novel appearance transfer method that combines controlled reflectance measurement of a small exemplar section with uncontrolled reflectance measurements of the full sample under natural lighting. Finally, we introduce a novel surface detail enhancement method that adds fine scale surface mesostructure from close‐up observations under uncontrolled natural lighting. We demonstrate the accuracy and versatility of the proposed mobile reflectometry methods on a wide variety of spatially varying materials.