Light source position and reflectance estimation from a single view without the distant illumination assumption

Several techniques have been developed for recovering reflectance properties of real surfaces under unknown illumination. However, in most cases, those techniques assume that the light sources are located at infinity, which cannot be applied safely to, for example, reflectance modeling of indoor environments. In this paper, we propose two types of methods to estimate the surface reflectance property of an object, as well as the position of a light source from a single view without the distant illumination assumption, thus relaxing the conditions in the previous methods. Given a real image and a 3D geometric model of an object with specular reflection as inputs, the first method estimates the light source position by fitting to the Lambertian diffuse component, while separating the specular and diffuse components by using an iterative relaxation scheme. Our second method extends that first method by using as input a specular component image, which is acquired by analyzing multiple polarization images taken from a single view, thus removing its constraints on the diffuse reflectance property. This method simultaneously recovers the reflectance properties and the light source positions by optimizing the linearity of a log-transformed Torrance-Sparrow model. By estimating the object's reflectance property and the light source position, we can freely generate synthetic images of the target object under arbitrary lighting conditions with not only source direction modification but also source-surface distance modification. Experimental results show the accuracy of our estimation framework.     

Efficient,complete radiosity ray tracing using a shadow-coherence method

Most two-pass rendering methods calculate a radiosity shading for each patch or element in a scene in the first pass. This shading contains two components: one for the light received directly from the main light sources and one representing the intensity of the light received indirectly by means of diffuse and specular interreflection between patches. However, it is very difficult to achieve accurate representation of the distribution of this radiosity shading over the patch, particularly where clearly visible shadow boundaries exist. A better approach is to store only the indirect reflection component in the form of radiosity shading, and to calculate the direct reflection component during the second pass by casting shadow rays. This approach normally requires that many shadow rays must be cast. However, the number of rays for shadow testing can be kept low by selecting only those light sources that substantially contribute to the shading of a patch and applying an adaptive image refinement technique in combination with a shadow coherence method.     

Situation normal [Gourand and Phong shading]

Two of the most famous graphics techniques, derived from shading algorithms by Henri Gouraud and Bui Tuong Phong, have been implemented in hardware and software worldwide. Both methods use various hacks to smooth out the shading of a polygonal surface. So rather than looking like an assembly of flat slabs, a smooth-shaded model seems smoother. The sharp creases between polygons are gone, replaced by a continuous change in tone or color. But if the original polygons aren't being rendered directly, then the shading doesn't correspond to the original model. What model does it correspond to? In other words, what is the smooth surface which, when rendered accurately, has the same appearance as a Phong shaded polygonal model? Phong normal interpolation is the process of computing a point's surface normal by linear interpolation of the components of two normals at either end of a line containing that point. This normal may then be used as part of a shading equation that takes into account specular highlights in an empirical manner, Phong illumination. Throughout this article, I deal only with perfectly diffuse surfaces lit by a single light source, so Phong illumination isn't part of the discussion. We assume that the light source is conveniently located at infinity. Gourand shading means the process of interpolating a color component to find intermediate color values across a polygon. Phong shading means interpolating surface normals to find intermediate normals that we can then evaluate with respect to the light source to find a color for that point     

Separation of Reflection Components Using Color and Polarization

Specular reflections and interreflections produce strong highlights in brightness images. These highlights can cause vision algorithms for segmentation, shape from shading, binocular stereo, and motion estimation to produce erroneous results. A technique is developed for separating the specular and diffuse components of reflection from images. The approach is to use color and polarization information, simultaneously, to obtain constraints on the reflection components at each image point. Polarization yields local and independent estimates of the color of specular reflection. The result is a linear subspace in color space in which the local diffuse component must lie. This subspace constraint is applied to neighboring image points to determine the diffuse component. In contrast to previous separation algorithms, the proposed method can handle highlights on surfaces with substantial texture, smoothly varying diffuse reflectance, and varying material properties. The separation algorithm is applied to several complex scenes with textured objects and strong interreflections. The separation results are then used to solve three problems pertinent to visual perception; determining illumination color, estimating illumination direction, and shape recovery.     

Environment Mapping and Other Applications of World Projections

Various techniques have been developed that employ projections of the world as seen from a particular viewpoint. Blinn and Newell introduced reflection mapping for simulating mirror reflections on curved surfaces. Miller and Hoffman have presented a general illumination model based on environment mapping. World projections have also been used to model distant objects and to produce pictures with the fish-eye distortion required for Omnimax frames. This article proposes a uniform framework for representing and using world projections and argues that the best general-purpose representation is the is projection onto a cube. Surface shading and texture filtering are discussed in the context of environment mapping, and methods are presented for obtaining diffuse and specular surface illumination from prefiltered environment maps. Comparisons are made with ray tracing, noting that two problems with ray tracing?obtaining diffuse reflection and antialiasing specular reflection?can be handled effectively by environment mapping.     

Mixture of spherical distributions for single-view relighting

We present a method for simultaneously estimating the illumination of a scene and the reflectance property of an object from single view images - a single image or a small number of images taken from the same viewpoint. We assume that the illumination consists of multiple point light sources and the shape of the object is known. First, we represent the illumination on the surface of a unit sphere as a finite mixture of von Mises-Fisher distributions based on a novel spherical specular reflection model that well approximates the Torrance-Sparrow reflection model. Next, we estimate the parameters of this mixture model including the number of its component distributions and the standard deviation of them, which correspond to the number of light sources and the surface roughness, respectively. Finally, using these results as the initial estimates, we iteratively refine the estimates based on the original Torrance-Sparrow reflection model. The final estimates can be used to relight single-view images such as altering the intensities and directions of the individual light sources. The proposed method provides a unified framework based on directional statistics for simultaneously estimating the intensities and directions of an unknown number of light sources as well as the specular reflection parameter of the object in the scene.     

一种基于物理反射模型颜色不变性的阈值分割算法

提出了一种适用于视频监控场景的基于物理反射模型的阈值分割算法,该算法主要解决背景颜色识别受光强非均匀分布、高光效应影响的问题.算法步骤主要包括:首先基于Phong反射模型推导出漫反射分量颜色不变性并根据这一判定条件计算得到漫反射分量系数;其次,利用微分法则实现对模型镜面反射分量系数和镜面反射强度指数的估计;最后,根据建立的物理反射模型实现背景阈值分割.大量实验分析结果表明,文中提出的算法利用视频监控的物理反射模型和大量统计信息,能够更好地解决受光强非均匀分布和高光效应影响的颜色识别问题.     

Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images

The image of an opaque object is created by observing the reflection of the light incident on its surface. The dichromatic reflection model describes the surface reflection as the sum of two components, diffuse and specular terms. The specular reflection component is usually strong in its intensity and polarized significantly compared to the diffuse components. On the other hand, the intensity of the diffuse component is weak and it tends to be unpolarized except near occluding contours. Thus, the observation of an object through a rotating polarizer approximately yields images containing constant diffuse component and specular component of different intensity. In this paper, we show that diffuse and specular components of surface reflection can be separated as two independent components when we apply independent component analysis to the images observed through a polarizer of different orientations. We give a separation simulation of artificial data and also give some separation results of real scenes.     

Separating reflection components based on chromaticity and noise analysis

Many algorithms in computer vision assume diffuse only reflections and deem specular reflections to be outliers. However, in the real world, the presence of specular reflections is inevitable since there are many dielectric inhomogeneous objects which have both diffuse and specular reflections. To resolve this problem, we present a method to separate the two reflection components. The method is principally based on the distribution of specular and diffuse points in a two-dimensional maximum chromaticity-intensity space. We found that, by utilizing the space and known illumination color, the problem of reflection component separation can be simplified into the problem of identifying diffuse maximum chromaticity. To be able to identity the diffuse maximum chromaticity correctly, an analysis of the noise is required since most real images suffer from it. Unlike existing methods, the proposed method can separate the reflection components robustly for any kind of surface roughness and light direction.     

基于生物组织表面特性的体绘制光照模型

传统的光照模型侧重于模拟光能在微面元上的分布,它将表面反射分为镜面反射和漫反射,并采用表面法向与入射光夹角的余弦来模拟漫反射。由于生物组织表面富含水份,因此具有确定性介质和随机介质的双重特性。从物组织表面的漫反射光,就是由于光线射入介质表面产在介质内部被散射,而后射出表面的光能。因此,淡了逼真地表面生物组织的表面特性,必须同时考虑生物介质的表面反射和介质内部体散射双重特性。文中根据生物组织的特点,     

物体表面明暗处理算法的研究

基于现在普遍应用的Phone光照模型而对曲面体表面离散成小多边形后进行明暗处理的方法进行了简单的陈述和比较,提出了改进的Phone明暗处理方法,采用Visual C＋＋实现程序,对Phone光照模型中的漫反射项中的L·N和镜面反射项中的N·H直接进行双线性插值,既保证了着色效果,又使Phone明暗处理的效率提高了近1/3。     

Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation

We present a computational model and algorithm for detecting diffuse and specular interface reflections and some inter-reflections. Our color reflection model is based on the dichromatic model for dielectric materials and on a color space, called S space, formed with three orthogonal basis functions. We transform color pixels measured in RGB into the S space and analyze color variations on objects in terms of brightness, hue and saturation which are defined in the S space. When transforming the original RGB data into the S space, we discount the scene illumination color that is estimated using a white reference plate as an active probe. As a result, the color image appears as if the scene illumination is white. Under the whitened illumination, the interface reflection clusters in the S space are all aligned with the brightness direction. The brightness, hue and saturation values exhibit a more direct correspondence to body colors and to diffuse and specular interface reflections, shading, shadows and inter-reflections than the RGB coordinates. We exploit these relationships to segment the color image, and to separate specular and diffuse interface reflections and some inter-reflections from body reflections. The proposed algorithm is effications for uniformly colored dielectric surfaces under singly colored scene illumination. Experimental results conform to our model and algorithm within the liminations discussed.     

A Light Hierarchy for Fast Rendering of Scenes with Many Lights

We introduce a new data structure in the form of a light hierarchy for efficiently ray-tracing scenes with many light sources. An octree is constructed with the point light sources in a scene. Each node represents all the light sources it contains by means of a virtual light source. We determine bounds on the error committed with this approximation to shade a point, both for the cases of diffuse and specular reflections. These bounds are then used to guide a hierarchical shading algorithm. If the current level of the light hierarchy provides shading of sufficient quality, the approximation is used, thus avoiding the cost of shading for all the light sources contained below this level. Otherwise the descent into the light hierarchy continues.
Our approach has been implemented for scenes without occlusion. The results show important acceleration compared to standard ray-tracing (up to 90 times faster) and an important improvement compared to Ward's adaptive shadow testing.     

Direct Differential Photometric Stereo Shape Recovery of Diffuse and Specular Surfaces

Recovering the 3D shape of an object from shading is a challenging problem due to the complexity of modeling light propagation and surface reflections. Photometric Stereo (PS) is broadly considered a suitable approach for high-resolution shape recovery, but its functionality is restricted to a limited set of object surfaces and controlled lighting setup. In particular, PS models generally consider reflection from objects as purely diffuse, with specularities being regarded as a nuisance that breaks down shape reconstruction. This is a serious drawback for implementing PS approaches, since most common materials have prominent specular components. In this paper, we propose a PS model that solves the problem for both diffuse and specular components aimed at shape recovery of generic objects with the approach being independent of the albedo values thanks to the image ratio formulation used. Notably, we show that by including specularities, it is possible to solve the PS problem for a minimal number of three images using a setup with three calibrated lights and a standard industrial camera. Even if an initial separation of diffuse and specular components is still required for each input image, experimental results on synthetic and real objects demonstrate the feasibility of our approach for shape reconstruction of complex geometries.     

基于Lambert-Phong模型的图像盲取证方法

由于现有的图像盲取证方法中所使用的光照模型不能有效地表征物体表面的实际光照效果,提出Lambert-Phong光照模型。该模型同时考虑光照的漫反射和镜面反射,利用该光照模型对无限光源模式下的图像进行蓄意修改检测。实验结果表明,Lambert-Phong光照模型能较准确地计算出图像中不同目标的光照方向,有效地判别出图像是否经过蓄意修改。     

物体三维形状恢复的遗传算法方法

提出了一种新的物体三维形状恢复的遗传算法方法。用固定位置的摄像机在不同位置的球扩展光源下获取图象序列,并用遗传算法对物体表面每一点的法线矢量进行快速搜索。实验结果表明,此方法能有效地恢复扩展光源下物体的三维形状。不仅放宽了对光源和物体表面的限制,而且精度及鲁棒性均有很大提高。     

A Framework for the Construction of Reflectance Maps for Machine Vision

A reflectance map is the transfer function from surface orientation and illumination geometry to the surface normal, and in machine vision it plays a fundamental role in the reconstruction of surface by shape-from-shading and photometric stereo algorithms. While reflectance maps for Lambertain and specular surfaces are well understood, maps for real-world diffusely reflecting surfaces are scant. In this paper, the fundamental mechanisms of reflection from such surfaces are reviewed. Based on this, it is proposed that for point light source illumination, the diffuse component of the reflectance map has three terms: a forescatter term, a normal term, and a backscatter term. The physical origin of the three terms is discussed in detail and useful mathematical expressions are obtained for them. The range of applicability of the proposed reflectance maps is established, and an example of their use in photometric stereo is provided. The mathematical form of the reflectance map obtained from physical theories is amenable to generalization and such a generalization is called the m-lobed reflectance map is proposed.     

Accurately estimating reflectance parameters for color and gloss reproduction

A new method is described to estimate diffuse and specular reflectance parameters using spectral images, which overcomes the dynamic range limitation of imaging devices. After eliminating the influences of illumination and camera on spectral images, reflection values are initially assumed as diffuse-only reflection components, and subjected to the least squares method to estimate diffuse reflectance parameters at each wavelength on each single surface particle. Based on the dichromatic reflection model, specular reflection components are obtained, and then subjected to the least squares method to estimate specular reflectance parameters for gloss intensity and surface roughness. Experiments were carried out using both simulation data and measured spectral images. Our results demonstrate that this method is capable of estimating diffuse and specular reflectance parameters precisely for color and gloss reproduction, without requiring preprocesses such as image segmentation and synthesis of high dynamic range images.     

Wavelet-Texture Method: Appearance Compression by Polarization, Parametric Reflection Model, and Daubechies Wavelet

In order to create a photorealistic Virtual Reality model, we have to record the appearance of the object from different directions under different illuminations. In this paper, we propose a method that renders photorealistic images from a small amount of data. First, we separate the images of the object into a diffuse reflection component and a specular reflection component by using linear polarizers. Then, we estimate the parameters of the reflection model for each component. Finally, we compress the difference between the input images and the rendered images by using wavelet transform. At the rendering stage, we first calculate the diffuse and specular reflection images from the reflection parameters, then add the difference decompressed by inverse wavelet transform into the calculated reflection images, and finally obtain the photorealistic image of the object.     

基于混合反射模型的SFS有限元方法的研究

提出了基于混合反射模型的由明暗恢复物体三维形状的有限元算法。用正方形面元逼近光滑曲面,把曲面表示为所有节点基函数的线性组合;基于既含有漫反射成分又有镜面反射成分的混合模型,结合节点基函数,将反射图线性化。考虑数字图像的特点,直接使用离散形式的SFS问题的亮度约束形式,用最小化方法得到高度满足的线性方程;使用Kaczmarz算法计算出表面三维形状。使用合成图像和实际图像验证该文算法的有效性,探讨了该算法的性能。