A general framework for three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features

In this paper a novel framework for three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features is presented. Based on the analysis of at least two pixel-synchronous images of the scene under different illumination conditions, this framework combines a shape from shading approach for estimating surface gradients and altitude variations on small scales with a shadow analysis method that allows for the determination of the large-scale properties of the surface. As a first step, the result of shadow analysis is used for selecting a consistent solution of the shape from shading reconstruction algorithm. As a second step, an additional error term derived from the fine-structure of the shadow is incorporated into the reconstruction algorithm. This approach is extended to the analysis of two or more shadows under different illumination conditions leading to an appropriate initialization of the shape from shading algorithm. The framework is applied to the astrogeological task of three-dimensional reconstruction of regions on the lunar surface using ground-based CCD images and to the machine vision task of industrial quality inspection.     

一种快速精确的人脸三维形状重构新方法

针对传统 SFS(Shap from Shading)的不足 ,提出了一种新的基于 BP神经网络的明暗恢复形状的方法 ,该方法是基于兰伯特 (L am bertian)反射模型的改进算法 ,利用了 BP神经网络强的非线性映射能力 ,将 L ambertian表面反射模型与光滑表面模型相结合 ,然后再利用一些已知条件 ,构成 SFS问题的正则化模型 ;变换不同的照明条件 ,将模型平移或旋转获得多幅图象 ,以增加约束条件 ;计算出误差补偿参数去修正邻域内的三维误差 .由于考虑了邻域的平均值 ,使算法的稳定性和精确性都得到了加强 .实例表明 ,该算法较传统的算法更快和更精确     

Joint Estimation of Shape and Reflectance using Multiple Images with Known Illumination Conditions

We propose a generative model based method for recovering both the shape and the reflectance of the surface(s) of a scene from multiple images, assuming that illumination conditions and cameras calibration are known in advance. Based on a variational framework and via gradient descents, the algorithm minimizes simultaneously and consistently a global cost functional with respect to both shape and reflectance. The motivations for our approach are threefold. (1) Contrary to previous works which mainly consider specific individual scenarios, our method applies indiscriminately to a number of classical scenarios; in particular it works for classical stereovision, multiview photometric stereo and multiview shape from shading. It works with changing as well as static illumination. (2) Our approach naturally combines stereo, silhouette and shading cues in a single framework. (3) Moreover, unlike most previous methods dealing with only Lambertian surfaces, the proposed method considers general dichromatic surfaces. We verify the method using various synthetic and real data sets.     

Local shape approximation from shading

Shading can be used as an independent cue for exact shape recovery, or it can be used as a supplementary cue for shape interpolation between features whose depths are known from other cues. Exact shape cannot be inferred from a local analysis of shading. However, for shape interpolation a crude local approximation may be sufficient. This paper explores the limits of such local approximations that are easy to compute. In particular, the shape of shading is used to approximate the surface in areas of monotonic change of intensity. This analysis is accompanied by a method for computing the direction of a single-point light source from the shading on occluding contours. A qualitative classification of shape near shading singularities is also discussed.This work was performed at the Massachusetts Institute of Technology, Center for Biological Information Processing.     

The measurement of highlights in color images

In this paper, we present an approach to color image understanding that accounts for color variations due to highlights and shading. We demonstrate that the reflected light from every point on a dielectric object, such as plastic, can be described as a linear combination of the object color and the highlight color. The colors of all light rays reflected from one object then form a planar cluster in the color space. The shape of this cluster is determined by the object and highlight colors and by the object shape and illumination geometry. We present a method that exploits the difference between object color and highlight color to separate the color of every pixel into a matte component and a highlight component. This generates two intrinsic images, one showing the scene without highlights, and the other one showing only the highlights. The intrinsic images may be a useful tool for a variety of algorithms in computer vision, such as stereo vision, motion analysis, shape from shading, and shape from highlights. Our method combines the analysis of matte and highlight reflection with a sensor model that accounts for camera limitations. This enables us to successfully run our algorithm on real images taken in a laboratory setting. We show and discuss the results.This material is based upon work supported by the National Science Foundation under Grant DCR-8419990 and by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 4976, monitored by the Air Force Avionics Laboratory under contract F33615-84-K-1520. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation, the Defense Advanced Research Projects Agency, or the US Government.     

基于分形的从明暗恢复形状算法研究

从明暗恢复形状是计算机视觉领域中的经典病态问题，传统方法是通过引入光滑约束等条件来获得问题的解，但传统方法存在因过平滑而失真的缺点。针对传统方法恢复结果的局限性，提出了一种基于分形约束的从明暗恢复形状的新算法，该方法首先给出分形约束条件，之后结合反射图线性化与最小能量法来计算出曲面高度。该方法不仅克服了传统算法因基于光滑假设所造成的恢复结果过分平滑而失真的缺点，且不需要可积性的约束条件，也不需要对边界条件的假设，实验结果表明，该方法用于自然景物的三维表面重构，可获得比传统方法更好的恢复效果。     

Estimation of surface orientation from directional derivatives of shading image

A new estimation approach is proposed to reconstruct the surface shape from shading images. The reflectance property of the object surface material is the uniform lambertian property; the light source produces a uniform parallel beam, and image projection is orthographic. The illuminating direction is varied slightly to obtain directional derivatives of image density. The new approach shows that the local surface orientation is reconstructed from the directional derivatives of the image density. Differently from other approaches, all components of the illuminating direction vector or the proportional constant are not necessary for this reconstruction. Computer simulations show the evaluation of errors in approximating and quantizing process. The effectiveness of this approach is demonstrated in the shape reconstruction simulations for the curved objects     

Dense Reconstruction of Transparent Objects by Altering Incident Light Paths Through Refraction

This paper addresses the problem of reconstructing the surface shape of transparent objects. The difficulty of this problem originates from the viewpoint dependent appearance of a transparent object, which quickly makes reconstruction methods tailored for diffuse surfaces fail disgracefully. In this paper, we introduce a fixed viewpoint approach to dense surface reconstruction of transparent objects based on refraction of light. We present a simple setup that allows us to alter the incident light paths before light rays enter the object by immersing the object partially in a liquid, and develop a method for recovering the object surface through reconstructing and triangulating such incident light paths. Our proposed approach does not need to model the complex interactions of light as it travels through the object, neither does it assume any parametric form for the object shape nor the exact number of refractions and reflections taken place along the light paths. It can therefore handle transparent objects with a relatively complex shape and structure, with unknown and inhomogeneous refractive index. We also show that for thin transparent objects, our proposed acquisition setup can be further simplified by adopting a single refraction approximation. Experimental results on both synthetic and real data demonstrate the feasibility and accuracy of our proposed approach.     

Variable-Source Shading Analysis

The shading on curved surfaces is a cue to shape. Current computer vision methods for analyzing shading use physically unrealistic models, have serious mathematical problems, cannot exploit geometric information if it is available, and are not reliable in practice. We introduce a novel method of accounting for variations in irradiance resulting from interreflections, complex sources and the like. Our approach uses a spatially varying source model with a local shading model. Fast spatial variation in the source is penalised, consistent with the rendering community’s insight that interreflections are spatially slow. This yields a physically plausible shading model. Because modern cameras can make accurate reports of observed radiance, our method compels the reconstructed surface to have shading exactly consistent with that of the image. For inference, we use a variational formulation, with a selection of regularization terms which guarantee that a solution exists. Our method is evaluated on physically accurate renderings of virtual objects, and on images of real scenes, for a variety of different kinds of boundary condition. Reconstructions for single sources compare well with photometric stereo reconstructions and with ground truth.     

Lit-Sphere extension for artistic rendering

The Lit-Sphere model proposed by Sloan et al. (Proceedings of Graphics Interface 2001, pp. 143–150, 2001) is a method for emulating expressive artistic shading styles for 3D scenes. Assuming that artistic shading styles are described by view space normals, this model produces a variety of stylized shading scenes beyond traditional 3D lighting control. However, it is limited to the static lighting case: the shading effect is only dependent on the camera view. In addition, it cannot support small-scale brush stroke styles. In this paper, we propose a scheme to extend the Lit-Sphere model based on light space normals rather than view space normals. Owing to the light space representation, our shading model addresses the issues of the original Lit-Sphere approach, and allows artists to use a light source to obtain dynamic diffuse and specular shading. Then the shading appearance can be refined using stylization effects including highlight shape control, sub-lighting effects, and brush stroke styles. Our algorithms are easy to implement on GPU, so that our system allows interactive shading design.     

Linear-nonlinear neuronal model for shape from shading

The goal of shape from shading (SFS) is to recover a relative depth map from the variations of image intensity associated to changes in surface shape. There have been very few attempts at developing biologically plausible solutions to this problem, and a sound neurophysiological basis is still missing. Here we present a biologically inspired approach to SFS, formulated in terms of the well-known linear-nonlinear model of neuronal responses. Without resorting to the image irradiance equation, which is at the heart of the traditional SFS algorithms, we submit the input image to a linear filter followed by nonlinear transformations modelled on the tuning curves of the disparity-selective binocular neurons. This yields plausible shape estimates, without requiring information regarding surface reflectance or illumination.     

A Practical and Hierarchical Yarn-based Shading Model for Cloth

Realistic cloth rendering is a longstanding challenge in computer graphics due to the intricate geometry and hierarchical structure of cloth: Fibers form plies which in turn are combined into yarns which then are woven or knitted into fabrics. Previous fiber-based models have achieved high-quality close-up rendering, but they suffer from high computational cost, which limits their practicality. In this paper, we propose a novel hierarchical model that analytically aggregates light simulation on the fiber level by building on dual-scattering theory. Based on this, we can perform an efficient simulation of ply and yarn shading. Compared to previous methods, our approach is faster and uses less memory while preserving a similar accuracy. We demonstrate both through comparison with existing fiber-based shading models. Our yarn shading model can be applied to curves or surfaces, making it highly versatile for cloth shading. This duality paired with its simplicity and flexibility makes the model particularly useful for film and games production.     

Adaptive Image‐Space Sampling for Gaze‐Contingent Real‐time Rendering

With ever‐increasing display resolution for wide field‐of‐view displays—such as head‐mounted displays or 8k projectors—shading has become the major computational cost in rasterization. To reduce computational effort, we propose an algorithm that only shades visible features of the image while cost‐effectively interpolating the remaining features without affecting perceived quality. In contrast to previous approaches we do not only simulate acuity falloff but also introduce a sampling scheme that incorporates multiple aspects of the human visual system: acuity, eye motion, contrast (stemming from geometry, material or lighting properties), and brightness adaptation. Our sampling scheme is incorporated into a deferred shading pipeline to shade the image's perceptually relevant fragments while a pull‐push algorithm interpolates the radiance for the rest of the image. Our approach does not impose any restrictions on the performed shading. We conduct a number of psycho‐visual experiments to validate scene‐ and task‐independence of our approach. The number of fragments that need to be shaded is reduced by 50 % to 80 %. Our algorithm scales favorably with increasing resolution and field‐of‐view, rendering it well‐suited for head‐mounted displays and wide‐field‐of‐view projection.     

Multiple Light Source Estimation in a Single Image

Many high‐level image processing tasks require an estimate of the positions, directions and relative intensities of the light sources that illuminated the depicted scene. In image‐based rendering, augmented reality and computer vision, such tasks include matching image contents based on illumination, inserting rendered synthetic objects into a natural image, intrinsic images, shape from shading and image relighting. Yet, accurate and robust illumination estimation, particularly from a single image, is a highly ill‐posed problem. In this paper, we present a new method to estimate the illumination in a single image as a combination of achromatic lights with their 3D directions and relative intensities. In contrast to previous methods, we base our azimuth angle estimation on curve fitting and recursive refinement of the number of light sources. Similarly, we present a novel surface normal approximation using an osculating arc for the estimation of zenith angles. By means of a new data set of ground‐truth data and images, we demonstrate that our approach produces more robust and accurate results, and show its versatility through novel applications such as image compositing and analysis.     

A Green's function approach to shape from shading

We present a new algorithm for shape from shading, inspired on the recently introduced disparity-based approach to photometric stereo (DBPS). Assuming that the single input image will be matched to a second image through a uniform disparity field, we construct an irradiance conservation equation and solve it for the matching image, via Green's function. When a linear expansion of the reflectance map is considered, this leads to a closed-form approximate expression for the surface function, whose parameters can be estimated via a structure-from-motion approach already used for DBPS.     

Building a color classification system for textured and hue homogeneous surfaces: system calibration and algorithm

We present a system for classifying the color aspect of textured surfaces having a nearly constant hue (such as wooden boards, textiles, wallpaper, etc.). The system is designed to compensate for small fluctuations (over time) of the light source and for inhomogeneous illumination conditions (shading correction). This is an important feature because even in industrial environments where the lighting conditions are controlled, a constant and homogeneous illumination cannot be guaranteed. Together with an appropriate camera calibration (which includes a periodic update), our approach offers a robust system which is able to “distinguish” (i.e., classify correctly) between surface classes which exhibit visually barely perceptible color variations. In particular, our approach is based on relative (not absolute) color measurements. In this paper, we outline the classification algorithm while focusing in detail on the camera calibration and a method for compensating for fluctuations of the light source. Received: 1 September 1998 / Accepted: 16 March 2000     

Computational Creation of Hollow Mask Type Illusionary Solids with Shading Effect

"Hollow mask illusion" is an optical illusion and appears due to an error in the process of reconstructing the three-dimensional objects from our two-dimensional retinal image. In this paper, we present a computational method to create "Hollow mask" type new illusionary solid by calculating the hollow structure and its shading. The straight line Voronoi diagram for a given shape obtains the three-dimensional vertices of the hollow structure and the shading effect on each surface is calculated under the assumption that each surface has diffusely reflecting surface(Lambertian reflectance). We also show two examples of our new illusionary solid works.     

Impossible and ambiguous shading patterns

A smooth object depicted in a monochrome image will often exhibit brightness variation, or shading. A problem much studied in computer vision has been that of how object shape may be recovered from image shading. When the imaging conditions are such that an overhead point-source illuminates a smooth Lambertian surface, the problem may be formulated as that of finding a solution to an eikonal equation. This article will focus on the existence and uniqueness of such solutions, reporting recent results obtained. With regard to existence, shading patterns are exhibited for which there is no corresponding object shape. Specifically, a necessary and sufficient condition is presented for a circularly symmetric eikonal equation to admit exclusively unbounded solutions; additionally, a sufficient condition is given for an eikonal equation to have no solution whatsoever. In connection with uniqueness, we consider eikonal equations, defined over a disc, such that the Euclidean norm of the gradient of any solution is circularly symmetric, vanishes exactly at the disc center, and diverges to infinity as the circumference of the disc is approached. Contrary to earlier influential work, a class of such eikonal equations is shown to possess simultaneously circularly symmetric and noncircularly symmetric bounded smooth solutions.     

Direct Differential Photometric Stereo Shape Recovery of Diffuse and Specular Surfaces

Recovering the 3D shape of an object from shading is a challenging problem due to the complexity of modeling light propagation and surface reflections. Photometric Stereo (PS) is broadly considered a suitable approach for high-resolution shape recovery, but its functionality is restricted to a limited set of object surfaces and controlled lighting setup. In particular, PS models generally consider reflection from objects as purely diffuse, with specularities being regarded as a nuisance that breaks down shape reconstruction. This is a serious drawback for implementing PS approaches, since most common materials have prominent specular components. In this paper, we propose a PS model that solves the problem for both diffuse and specular components aimed at shape recovery of generic objects with the approach being independent of the albedo values thanks to the image ratio formulation used. Notably, we show that by including specularities, it is possible to solve the PS problem for a minimal number of three images using a setup with three calibrated lights and a standard industrial camera. Even if an initial separation of diffuse and specular components is still required for each input image, experimental results on synthetic and real objects demonstrate the feasibility of our approach for shape reconstruction of complex geometries.     

Implicit and Nonparametric Shape Reconstruction from Unorganized Data Using a Variational Level Set Method

In this paper we consider a fundamental visualization problem: shape reconstruction from an unorganized data set. A new minimal-surface-like model and its variational and partial differential equation (PDE) formulation are introduced. In our formulation only distance to the data set is used as our input. Moreover, the distance is computed with optimal speed using a new numerical PDE algorithm. The data set can include points, curves, and surface patches. Our model has a natural scaling in the nonlinear regularization that allows flexibility close to the data set while it also minimizes oscillations between data points. To find the final shape, we continuously deform an initial surface following the gradient flow of our energy functional. An offset (an exterior contour) of the distance function to the data set is used as our initial surface. We have developed a new and efficient algorithm to find this initial surface. We use the level set method in our numerical computation in order to capture the deformation of the initial surface and to find an implicit representation (using the signed distance function) of the final shape on a fixed rectangular grid. Our variational/PDE approach using the level set method allows us to handle complicated topologies and noisy or highly nonuniform data sets quite easily. The constructed shape is smoother than any piecewise linear reconstruction. Moreover, our approach is easily scalable for different resolutions and works in any number of space dimensions.