Light source position and reflectance estimation from a single view without the distant illumination assumption

Several techniques have been developed for recovering reflectance properties of real surfaces under unknown illumination. However, in most cases, those techniques assume that the light sources are located at infinity, which cannot be applied safely to, for example, reflectance modeling of indoor environments. In this paper, we propose two types of methods to estimate the surface reflectance property of an object, as well as the position of a light source from a single view without the distant illumination assumption, thus relaxing the conditions in the previous methods. Given a real image and a 3D geometric model of an object with specular reflection as inputs, the first method estimates the light source position by fitting to the Lambertian diffuse component, while separating the specular and diffuse components by using an iterative relaxation scheme. Our second method extends that first method by using as input a specular component image, which is acquired by analyzing multiple polarization images taken from a single view, thus removing its constraints on the diffuse reflectance property. This method simultaneously recovers the reflectance properties and the light source positions by optimizing the linearity of a log-transformed Torrance-Sparrow model. By estimating the object's reflectance property and the light source position, we can freely generate synthetic images of the target object under arbitrary lighting conditions with not only source direction modification but also source-surface distance modification. Experimental results show the accuracy of our estimation framework.     

Stratified Sampling of Projected Spherical Caps

We present a method for uniformly sampling points inside the projection of a spherical cap onto a plane through the sphere's center. To achieve this, we devise two novel area‐preserving mappings from the unit square to this projection, which is often an ellipse but generally has a more complex shape. Our maps allow for low‐variance rendering of direct illumination from finite and infinite (e.g. sun‐like) spherical light sources by sampling their projected solid angle in a stratified manner. We discuss the practical implementation of our maps and show significant quality improvement over traditional uniform spherical cap sampling in a production renderer.     

利用线性插值合成图象

室外场景的光照模拟与图象合成的难点在于场景与光照的复杂性。提出了一种利用现有基图象的线性组合来生成不同天候条件下同一场景图象的方法。它根据光照函数的线性关系,估计不同太阳位置和不同天气状况基图象的系数,从而最终通过基图象的组合来产生目标图象。实验结果表明本文的方法用于固定场景下的图象合成是可行的。     

基于标志点注册及探测球的光源追踪算法

增强现实系统中,解决虚拟物体表面光照情况与真实环境匹配问题关键的一步是进行光源追踪。基于标志点注册及具有漫反射特性的探测球提出一种光源追踪算法。该算法仅对一幅被单一光源照射的标志立方体和探测球图像进行分析,利用图中标志点确定探测球相对照相机的位置、姿态关系,利用探测球表面的亮度信息推算光源向量。该探测球图像要进行一系列图像处理程序,其中等亮度线的提取及拟合是关键步骤。实验结果表明,本文算法能够达到预期效果,较好地实现了光源追踪,适用于各种位置的单一光源照射情况及基于标志点注册的增强现实系统。     

Interactive Lighting Design with Hierarchical Light Representation

Lighting design plays a crucial role in indoor lighting design, computer cinematograph and many other applications. Computer‐assisted lighting design aims to find a lighting configuration that best approximates the illumination effect specified by designers. In this paper, we present an automatic approach for lighting design, in which discrete and continuous optimization of the lighting configuration, including the number, intensity, and position of lights, are achieved. Our lighting design algorithm consists of two major steps. The first step estimates an initial lighting configuration by light sampling and clustering. The initial light clusters are then recursively merged to form a light hierarchy. The second step optimizes the lighting configuration by alternatively selecting a light cut on the light hierarchy to determine the number of representative lights and optimizing the lighting parameters using the simplex method. To speed up the optimization computation, only illumination at scene vertices that are important to rendering are sampled and taken into account in the optimization. Using the proposed approach, we develop a lighting design system that can compute appropriate lighting configurations to generate the illumination effects iteratively painted and modified by a designer interactively.     

Interactive virtual relighting of real scenes

Computer augmented reality (CAR) is a rapidly emerging field which enables users to mix real and virtual worlds. Our goal is to provide interactive tools to perform common illumination, i.e., light interactions between real and virtual objects, including shadows and relighting (real and virtual light source modification). In particular, we concentrate on virtually modifying real light source intensities and inserting virtual lights and objects into a real scene; such changes can be very useful for virtual lighting design and prototyping. To achieve this, we present a three-step method. We first reconstruct a simplified representation of real scene geometry using semiautomatic vision-based techniques. With the simplified geometry, and by adapting recent hierarchical radiosity algorithms, we construct an approximation of real scene light exchanges. We next perform a preprocessing step, based on the radiosity system, to create unoccluded illumination textures. These replace the original scene textures which contained real light effects such as shadows from real lights. This texture is then modulated by a ratio of the radiosity (which can be changed) over a display factor which corresponds to the radiosity for which occlusion has been ignored. Since our goal is to achieve a convincing relighting effect, rather than an accurate solution, we present a heuristic correction process which results in visually plausible renderings. Finally, we perform an interactive process to compute new illumination with modified real and virtual light intensities     

Determining surface orientations of specular surfaces by using the photometric stereo method

The orientation of patches on the surface of an object can be determined from multiple images taken with different illumination, but from the same viewing position. The method, referred to as photometric stereo, can be implemented using table lookup based on numerical inversion of reflectance maps. Here we concentrate on objects with specularly reflecting surfaces, since these are of importance in industrial applications. Previous methods, intended for diffusely reflecting surfaces, employed point source illumination, which is quite unsuitable in this case. Instead, we use a distributed light source obtained by uneven illumination of a diffusely reflecting planar surface. Experimental results are shown to verify analytic expressions obtained for a method employing three light source distributions.     

Real‐Time Indirect Illumination and Soft Shadows in Dynamic Scenes Using Spherical Lights

We present a method for rendering approximate soft shadows and diffuse indirect illumination in dynamic scenes. The proposed method approximates the original scene geometry with a set of tightly fitting spheres. In previous work, such spheres have been used to dynamically evaluate the visibility function to render soft shadows. In this paper, each sphere also acts as a low‐frequency secondary light source, thereby providing diffuse one‐bounce indirect illumination. The method is completely dynamic and proceeds in two passes: In a first pass, the light intensity distribution on each sphere is updated based on sample points on the corresponding object surface and converted into the spherical harmonics basis. In a second pass, this radiance information and the visibility are accumulated to shade final image pixels. The sphere approximation allows us to compute visibility and diffuse reflections of an object at interactive frame rates of over 20 fps for moderately complex scenes.     

What Do the Sun and the Sky Tell Us About the Camera?

As the main observed illuminant outdoors, the sky is a rich source of information about the scene. However, it is yet to be fully explored in computer vision because its appearance in an image depends on the sun position, weather conditions, photometric and geometric parameters of the camera, and the location of capture. In this paper, we analyze two sources of information available within the visible portion of the sky region: the sun position, and the sky appearance. By fitting a model of the predicted sun position to an image sequence, we show how to extract camera parameters such as the focal length, and the zenith and azimuth angles. Similarly, we show how we can extract the same parameters by fitting a physically-based sky model to the sky appearance. In short, the sun and the sky serve as geometric calibration targets, which can be used to annotate a large database of image sequences. We test our methods on a high-quality image sequence with known camera parameters, and obtain errors of less that 1% for the focal length, 1° for azimuth angle and 3° for zenith angle. We also use our methods to calibrate 22 real, low-quality webcam sequences scattered throughout the continental US, and show deviations below 4% for focal length, and 3° for the zenith and azimuth angles. Finally, we demonstrate that by combining the information available within the sun position and the sky appearance, we can also estimate the camera geolocation, as well as its geometric parameters. Our method achieves a mean localization error of 110 km on real, low-quality Internet webcams. The estimated viewing and illumination geometry of the scene can be useful for a variety of vision and graphics tasks such as relighting, appearance analysis and scene recovery.     

Interactive Global Photon Mapping

We present a photon mapping technique capable of computing high quality global illumination at interactive frame rates. By extending the concept of photon differentials to efficiently handle diffuse reflections, we generate footprints at all photon hit points. These enable illumination reconstruction by density estimation with variable kernel bandwidths without having to locate the k nearest photon hits first. Adapting an efficient BVH construction process for ray tracing acceleration, we build photon maps that enable the fast retrieval of all hits relevant to a shading point. We present a heuristic that automatically tunes the BVH build's termination criterion to the scene and illumination conditions. As all stages of the algorithm are highly parallelizable, we demonstrate an implementation using NVidia's CUDA manycore architecture running at interactive rates on a single GPU. Both light source and camera may be freely moved with global illumination fully recalculated in each frame.     

Estimating the Natural Illumination Conditions from a Single Outdoor Image

Given a single outdoor image, we present a method for estimating the likely illumination conditions of the scene. In particular, we compute the probability distribution over the sun position and visibility. The method relies on a combination of weak cues that can be extracted from different portions of the image: the sky, the vertical surfaces, the ground, and the convex objects in the image. While no single cue can reliably estimate illumination by itself, each one can reinforce the others to yield a more robust estimate. This is combined with a data-driven prior computed over a dataset of 6 million photos. We present quantitative results on a webcam dataset with annotated sun positions, as well as quantitative and qualitative results on consumer-grade photographs downloaded from Internet. Based on the estimated illumination, we show how to realistically insert synthetic 3-D objects into the scene, and how to transfer appearance across images while keeping the illumination consistent.     

A method to generate soft shadows using a layered depth image and warping

We present an image-based method for propagating area light illumination through a layered depth image (LDI) to generate soft shadows from opaque and nonrefractive transparent objects. In our approach, using the depth peeling technique, we render an LDI from a reference light sample on a planar light source. Light illumination of all pixels in an LDI is then determined for all the other sample points via warping, an image-based rendering technique, which approximates ray tracing in our method. We use an image-warping equation and McMillan's warp ordering algorithm to find the intersections between rays and polygons and to find the order of intersections. Experiments for opaque and nonrefractive transparent objects are presented. Results indicate our approach generates soft shadows fast and effectively. Advantages and disadvantages of the proposed method are also discussed.     

光学观测中太阳夹角的分析与计算

对影响光学观测效果的太阳夹角计算方法进行了研究。首先给出了太阳夹角的定义及计算公式。然后重点研究了太阳视位置的计算方法,比较了几种赤纬角计算方法,给出了一种改进的时角计算方法,实现了高精度的太阳视位置计算,并对其计算精度进行了分析。最后提出了在实际光学观测应用中的太阳夹角计算算法。实验表明,太阳视位置计算精度比传统数值近似方法高一个数量级,能够满足光学观测中太阳光照分析的需要。     

Illumination-Guided Furniture Layout Optimization

Lighting plays a very important role in interior design. However, in the specific problem of furniture layout recommendation, illumination has been either neglected or addressed with empirical or very simplified solutions. The effectiveness of a particular layout in its expected task performance can be greatly affected by daylighting and artificial illumination in a non-trivial manner. In this paper, we introduce a robust method for furniture layout optimization guided by illumination constraints. The method takes into account all dominant light sources, such as sun light, skylighting and fixtures, while also being able to handle movable light emitters. For this task, the method introduces multiple generic illumination constraints and physically-based light transport estimators, operating alongside typical geometric design guidelines, in a unified manner. We demonstrate how to produce furniture arrangements that comply with important safety, comfort and efficiency illumination criteria, such as glare suppression, under complex light-environment interactions, which are very hard to handle using empirical or simplified models.     

Robust Estimation of Albedo for Illumination-Invariant Matching and Shape Recovery

We present a nonstationary stochastic filtering framework for the task of albedo estimation from a single image. There are several approaches in the literature for albedo estimation, but few include the errors in estimates of surface normals and light source direction to improve the albedo estimate. The proposed approach effectively utilizes the error statistics of surface normals and illumination direction for robust estimation of albedo, for images illuminated by single and multiple light sources. The albedo estimate obtained is subsequently used to generate albedo-free normalized images for recovering the shape of an object. Traditional Shape-from-Shading (SFS) approaches often assume constant/piecewise constant albedo and known light source direction to recover the underlying shape. Using the estimated albedo, the general problem of estimating the shape of an object with varying albedo map and unknown illumination source is reduced to one that can be handled by traditional SFS approaches. Experimental results are provided to show the effectiveness of the approach and its application to illumination-invariant matching and shape recovery. The estimated albedo maps are compared with the ground truth. The maps are used as illumination-invariant signatures for the task of face recognition across illumination variations. The recognition results obtained compare well with the current state-of-the-art approaches. Impressive shape recovery results are obtained using images downloaded from the Web with little control over imaging conditions. The recovered shapes are also used to synthesize novel views under novel illumination conditions.     

基于自适应可见性滤波的近似软影绘制

针对全局光照下的物理正确软影绘制较难满足交互性的难题,提出体现遮挡对象 空间位置远近关系的可变半影近似绘制算法。首先,以光源中心点为参照通过基于光线跟踪的 遮挡测试方法生成二值光源可见性图;并提出每个可视场景点对应自适应可见性空间平滑滤波 器宽度的确定方法;然后执行带掩模计算的自适应可见性滤波来获得从可见区到非可见区平滑 过渡的可见性因子;最后在光线跟踪流程中使用可见性因子动态调制相应可视场景点不考虑遮 挡的直接光照值,再加上间接光照得到高真实感软影。实验结果表明：该算法效果与物理正确 阴影在柔和度方面非常接近,容易绘制镜面反射间接光照,且测试场景的帧率在 30 帧/秒以上, 满足交互性要求。     

Tensors,Differential Geometry and Statistical Shading Analysis

We develop a linear algebraic framework for the shape-from-shading problem, because tensors arise when scalar (e.g., image) and vector (e.g., surface normal) fields are differentiated multiple times. Using this framework, we first investigate when image derivatives exhibit invariance to changing illumination by calculating the statistics of image derivatives under general distributions on the light source. Second, we apply that framework to develop Taylor-like expansions and build a boot-strapping algorithm to find the polynomial surface solutions (under any light source) consistent with a given patch to arbitrary order. A generic constraint on the light source restricts these solutions to a 2-D subspace, plus an unknown rotation matrix. It is this unknown matrix that encapsulates the ambiguity in the problem. Finally, we use the framework to computationally validate the hypothesis that image orientations (derivatives) provide increased invariance to illumination by showing (for a Lambertian model) that a shape-from-shading algorithm matching gradients instead of intensities provides more accurate reconstructions when illumination is incorrectly estimated under a flatness prior.     

An Improved Virtual Spherical Lights Approach for Glossy Illumination

Based on a virtual spherical light source method, this paper presents an improved virtual spherical lights approach for glossy illumination by modifying a reflection render equation. First, in order to keep primitive spherical lights, it uses non-zero solid angle integration instead of a geometric item of traditional Instant Radiosity arithmetic. Second, the formula mode for the light energy allocation of VSLs is improved by the energy distribution, in which the radius of sphere is replaced by the radius of inscribed circle generated by the tangent to the non-zero solid angle of cone. Third, the similar function of judging two point visibility takes place of a cosine term in the approximate equations of the VSLs. Experiments show that blurring or disappearing phenomenon that appear in part of right-angle surface scene is avoided.     

Design and enhancement of painting interface for room lights

We propose a novel painting interface that enables users to design an illumination distribution for a real room using an array of computer-controlled lights. Users specify which area of the room is to be well-lit and which is to be dark by painting a target illumination distribution on a tablet device displaying the image obtained by a camera mounted in the room. The painting result is overlaid on the camera image as contour lines of the target illumination intensity. The system then runs an optimization to calculate light parameters to deliver the requested illumination condition. We implemented a GPU-based parallel search to achieve real-time processing. In our system, we used actuated lights that can change the lighting direction to generate the requested illumination condition more faithfully than static lights. We built a miniature-scale experimental environment and ran a user study to compare our method with a standard direct manipulation method using sliders. The results showed that the users preferred our method for informal light control.     

Coarse view synthesis using shape-from-shading

This paper investigates the use of shape-from-shading for coarse view synthesis. The aim of our study is to determine whether needle-maps delivered by a new shape-from-shading (SFS) algorithm can be used as a compact object-representation for the purposes of efficiently generating appearance manifolds. Specifically, we aim to show that the needle-maps can be used to generate novel object views under changing light source and viewer directions. To this end we conduct two sets of experiments. Firstly, we use the recovered needle-maps to re-illuminate objects under varying lighting directions. Here we show that a single input image can be used to construct relatively faithful re-illuminations under radical illumination changes. Secondly, we investigate how the needle-map can be used to generate new object poses. Here we show that needle-maps can be used for both view interpolation and view extrapolation.