Mixture of spherical distributions for single-view relighting

We present a method for simultaneously estimating the illumination of a scene and the reflectance property of an object from single view images - a single image or a small number of images taken from the same viewpoint. We assume that the illumination consists of multiple point light sources and the shape of the object is known. First, we represent the illumination on the surface of a unit sphere as a finite mixture of von Mises-Fisher distributions based on a novel spherical specular reflection model that well approximates the Torrance-Sparrow reflection model. Next, we estimate the parameters of this mixture model including the number of its component distributions and the standard deviation of them, which correspond to the number of light sources and the surface roughness, respectively. Finally, using these results as the initial estimates, we iteratively refine the estimates based on the original Torrance-Sparrow reflection model. The final estimates can be used to relight single-view images such as altering the intensities and directions of the individual light sources. The proposed method provides a unified framework based on directional statistics for simultaneously estimating the intensities and directions of an unknown number of light sources as well as the specular reflection parameter of the object in the scene.     

纹理物体表面反射参数的恢复

物体表面反射参数的恢复是新兴的逆向绘制技术的核心问题，提出一种恢复物体表面BRDF模型参数的方法，假定物体表面的BRDF模型可以近似分解为漫射纹理和整体不变的镜面反射分量，则每一点的反射参数可以用Phong模型来近似计算，首先选取物体表面的一片区域，利用该区域在不同视点光照条件下采样的几幅图像，计算得到该区域的平均镜面反射分量，作为整个物体表面的镜面反射分量，然后据此从物体表面各点的采样中分离出漫射分量，计算各点的漫反射参数，得到近似的BRDF模型，实验数据表明算法是有效的。     

A Framework for Automatically Recovering Object Shape,Reflectance and Light Sources from Calibrated Images

In this paper, we present a complete framework for recovering an object shape, estimating its reflectance properties and light sources from a set of images. The whole process is performed automatically. We use the shape from silhouette approach proposed by R. Szeliski (1993) combined with image pixels for reconstructing a triangular mesh according to the marching cubes algorithm. A classification process identifies regions of the object having the same appearance. For each region, a single point or directional light source is detected. Therefore, we use specular lobes, lambertian regions of the surface or specular highlights seen on images. An identification method jointly (i) decides what light sources are actually significant and (ii) estimates diffuse and specular coefficients for a surface represented by the modified Phong model (Lewis, 1994). In order to validate our algorithm efficiency, we present a case study with various objects, light sources and surface properties. As shown in the results, our system proves accurate even for real objects images obtained with an inexpensive acquisition system.     

Shape and View Independent Reflectance Map from Multiple Views

We consider the problem of estimating the 3D shape and reflectance properties of an object made of a single material from a set of calibrated views. To model the reflectance, we propose to use the View Independent Reflectance Map (VIRM), which is a representation of the joint effect of the diffuse+specular Bidirectional Reflectance Distribution Function (BRDF) and the environment illumination. The object shape is parameterized using a triangular mesh. We pose the estimation problem as minimizing the cost of matching input images, and the images synthesized using the shape and VIRM estimates. We show that by enforcing a constant value of VIRM as a global constraint, we can minimize the cost function by iterating between the VIRM and shape estimation. Experimental results on both synthetic and real objects show that our algorithm can recover both the 3D shape and the diffuse/specular reflectance information. Our algorithm does not require the light sources to be known or calibrated. The estimated VIRM can be used to predict the appearances of objects with the same material from novel viewpoints and under transformed illumination. The support of National Science Foundation under grant ECS 02-25523 is gratefully acknowledged. Tianli Yu was supported in part by a Beckman Institute Graduate Fellowship.     

Retrieving shape information from multiple images of a specularsurface

In many remote sensing and machine vision applications, the shape of a specular surface such as water, glass, or polished metal must be determined instantaneously and under natural lighting conditions. Most image analysis techniques, however, assume surface reflectance properties or lighting conditions that are incompatible with these situations. To retrieve the shape of smooth specular surfaces, a technique known as specular surface stereo was developed. The method analyzes multiple images of a surface and finds a surface shape that results in a set of synthetic images that match the observed ones. An image synthesis model is used to predict image irradiance values as a function of the shape and reflectance properties of the surface, camera geometry, and radiance distribution of the illumination. The specular surface stereo technique was tested by processing four numerical simulations-a water surface illuminated by a low- and high-contrast extended light source, and a mirrored surface illuminated by a low- and high-contrast extended light source. Under these controlled circumstances, the recovered surface shape showed good agreement with the known input     

Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images

The image of an opaque object is created by observing the reflection of the light incident on its surface. The dichromatic reflection model describes the surface reflection as the sum of two components, diffuse and specular terms. The specular reflection component is usually strong in its intensity and polarized significantly compared to the diffuse components. On the other hand, the intensity of the diffuse component is weak and it tends to be unpolarized except near occluding contours. Thus, the observation of an object through a rotating polarizer approximately yields images containing constant diffuse component and specular component of different intensity. In this paper, we show that diffuse and specular components of surface reflection can be separated as two independent components when we apply independent component analysis to the images observed through a polarizer of different orientations. We give a separation simulation of artificial data and also give some separation results of real scenes.     

Accurately estimating reflectance parameters for color and gloss reproduction

A new method is described to estimate diffuse and specular reflectance parameters using spectral images, which overcomes the dynamic range limitation of imaging devices. After eliminating the influences of illumination and camera on spectral images, reflection values are initially assumed as diffuse-only reflection components, and subjected to the least squares method to estimate diffuse reflectance parameters at each wavelength on each single surface particle. Based on the dichromatic reflection model, specular reflection components are obtained, and then subjected to the least squares method to estimate specular reflectance parameters for gloss intensity and surface roughness. Experiments were carried out using both simulation data and measured spectral images. Our results demonstrate that this method is capable of estimating diffuse and specular reflectance parameters precisely for color and gloss reproduction, without requiring preprocesses such as image segmentation and synthesis of high dynamic range images.     

双边滤波的改进高光去除

目的 由于非均匀光照条件下,物体表面通常出现块状的强反射区域,传统的去高光方法在还原图像时容易造成颜色失真或者边缘的丢失。针对这些缺点,提出一种改进的基于双边滤波的去高光方法。方法 首先通过双色反射模型变换得到镜面反射分量与最大漫反射色度之间的转换关系,然后利用阈值将图像的像素点分为两类,将仅含漫反射分量的像素点与含有镜面反射分量的像素点分离开来,对两类像素点的最大漫反射色度分别做估计,接着以估计的最大漫反射色度的相似度作为双边滤波器的值域,同时以图像的最大色度图作为双边滤波的引导图保边去噪,进而达到去除镜面反射分量的目的。结果 以经典的高光图像作为处理对象,对含有镜面反射和仅含漫反射的像素点分别做最大漫反射色度估计,再以该估计图作为双边滤波的引导图,不仅能去除镜面反射分量还能有效的保留图像的边缘信息,最大程度的还原图像细节颜色,并且解决了原始算法处理结果中R、G、B三通道相似的像素点所出现的颜色退化问题。用改进的双边滤波去高光算法对50幅含高光的图像做处理,并将该算法与Yang方法和Shen方法分别作对比,结果图的峰值信噪比（PSNR）也分别平均提高4.17%和8.40%,所提算法的处理效果更符合人眼视觉,图像质量更好。结论 实验结果表明针对含镜面反射的图像,本文方法能够更有效去除图像的多区域局部高光,完成对图像的复原,可为室内外光照不匀情况下所采集图像的复原提供有效理论基础。     

Shading with Curve Light Sources1

A new shading model for curve light sources is presented. It accounts for both diffuse reflection and specular reflection of the illuminated surface. By regarding a linear light source as a directional quadrilateral light source with very small width, a simple formula is derived first for calculating the diffuse reflection component due to the illumination of the linear segment. The diffuse reflection of the surface by direct illumination of a curve light source is then evaluated by approximating the curve light with a series of linear segments. The specular reflection component due to a curve light source is represented by an integration taking Phong's specular model as the kernel and evaluated by summing the contributions from the linear segments. Finally, an efficient shadow detection algorithm for curve light sources is proposed. The images rendered with the shading model are very photo-realistic.     

Specular Highlight Removal for Real‐world Images

Removing specular highlight in an image is a fundamental research problem in computer vision and computer graphics. While various methods have been proposed, they typically do not work well for real‐world images due to the presence of rich textures, complex materials, hard shadows, occlusions and color illumination, etc. In this paper, we present a novel specular highlight removal method for real‐world images. Our approach is based on two observations of the real‐world images: (i) the specular highlight is often small in size and sparse in distribution; (ii) the remaining diffuse image can be represented by linear combination of a small number of basis colors with the sparse encoding coefficients. Based on the two observations, we design an optimization framework for simultaneously estimating the diffuse and specular highlight images from a single image. Specifically, we recover the diffuse components of those regions with specular highlight by encouraging the encoding coefficients sparseness using L₀ norm. Moreover, the encoding coefficients and specular highlight are also subject to the non‐negativity according to the additive color mixing theory and the illumination definition, respectively. Extensive experiments have been performed on a variety of images to validate the effectiveness of the proposed method and its superiority over the previous methods.     

Separation of Reflection Components Using Color and Polarization

Specular reflections and interreflections produce strong highlights in brightness images. These highlights can cause vision algorithms for segmentation, shape from shading, binocular stereo, and motion estimation to produce erroneous results. A technique is developed for separating the specular and diffuse components of reflection from images. The approach is to use color and polarization information, simultaneously, to obtain constraints on the reflection components at each image point. Polarization yields local and independent estimates of the color of specular reflection. The result is a linear subspace in color space in which the local diffuse component must lie. This subspace constraint is applied to neighboring image points to determine the diffuse component. In contrast to previous separation algorithms, the proposed method can handle highlights on surfaces with substantial texture, smoothly varying diffuse reflectance, and varying material properties. The separation algorithm is applied to several complex scenes with textured objects and strong interreflections. The separation results are then used to solve three problems pertinent to visual perception; determining illumination color, estimating illumination direction, and shape recovery.     

Separating reflection components based on chromaticity and noise analysis

Many algorithms in computer vision assume diffuse only reflections and deem specular reflections to be outliers. However, in the real world, the presence of specular reflections is inevitable since there are many dielectric inhomogeneous objects which have both diffuse and specular reflections. To resolve this problem, we present a method to separate the two reflection components. The method is principally based on the distribution of specular and diffuse points in a two-dimensional maximum chromaticity-intensity space. We found that, by utilizing the space and known illumination color, the problem of reflection component separation can be simplified into the problem of identifying diffuse maximum chromaticity. To be able to identity the diffuse maximum chromaticity correctly, an analysis of the noise is required since most real images suffer from it. Unlike existing methods, the proposed method can separate the reflection components robustly for any kind of surface roughness and light direction.     

Adaptive Acquisition of Lumigraphs from Synthetic Scenes

Light fields and Lumigraphs are capable of rendering scenes of arbitrary geometrical or illumination complexity in real time. They are thus interesting ways of interacting with both recorded real-world and high-quality synthetic scenes.
Unfortunately, both light fields and Lumigraph rely on a dense sampling of the illumination to provide a good rendering quality. This induces high costs both in terms of storage requirements and computational resources for the image acquisition. Techniques for acquiring adaptive light field and Lumigraph representations are thus mandatory for practical applications.
In this paper we present a method for the adaptive acquisition of images for Lumigraphs from synthetic scenes. Using image warping to predict the potential improvement in image quality when adding a certain view, we decide which new views of the scene should be rendered and added to the light field. This a-priori error estimator accounts for both visibility problems and illumination effects such as specular highlights.     

Determining reflectance properties of an object using range andbrightness images

The authors discuss a method of recovering reflectance properties of a surface from a range image given by a range finder and a brightness image given by a standard TV camera. The Torrance-Sparrow model is used for the reflectance model. The model consists of the Lambertian and specular components: its reflectance properties consist of the relative strength between the Lambertian and specular components and specular sharpness as well as light source direction. An iterative least square fitting method is used to obtain these parameters based on the range and brightness images. An input image is segmented into four different parts using the parameters: Lambertian reflection, specular reflection, interreflection, and shadow part. The authors also reconstruct ideal images that consist of only Lambertian or specular reflection     

Surface identification using the dichromatic reflection model

The author describes a method based on the dichromatic reflection model for identifying object surfaces. The surface spectral reflectance function of an inhomogeneous object is described as the sum of a constant interface (specular) reflectance and a body (diffuse) reflectance under all illumination geometries. The interface component is used to estimate the spectral power distribution of the illuminant without using a reference white standard, whereas the body component is used as the principal indication of the surface identity. The body reflectance function of each surface is recovered. A method to classify the observed reflectances is developed, and an algorithm to estimate a body reflectance function, unique to each surface, from the classified reflectances is proposed. The author shows the reliability of the surface classification method and the accuracy of estimated body reflectance function     

Illumination normalization with time-dependent intrinsic images for video surveillance

Variation in illumination conditions caused by weather, time of day, etc., makes the task difficult when building video surveillance systems of real world scenes. Especially, cast shadows produce troublesome effects, typically for object tracking from a fixed viewpoint, since it yields appearance variations of objects depending on whether they are inside or outside the shadow. In this paper, we handle such appearance variations by removing shadows in the image sequence. This can be considered as a preprocessing stage which leads to robust video surveillance. To achieve this, we propose a framework based on the idea of intrinsic images. Unlike previous methods of deriving intrinsic images, we derive time-varying reflectance images and corresponding illumination images from a sequence of images instead of assuming a single reflectance image. Using obtained illumination images, we normalize the input image sequence in terms of incident lighting distribution to eliminate shadowing effects. We also propose an illumination normalization scheme which can potentially run in real time, utilizing the illumination eigenspace, which captures the illumination variation due to weather, time of day, etc., and a shadow interpolation method based on shadow hulls. This paper describes the theory of the framework with simulation results and shows its effectiveness with object tracking results on real scene data sets.     

物体三维形状恢复的遗传算法方法

提出了一种新的物体三维形状恢复的遗传算法方法。用固定位置的摄像机在不同位置的球扩展光源下获取图象序列,并用遗传算法对物体表面每一点的法线矢量进行快速搜索。实验结果表明,此方法能有效地恢复扩展光源下物体的三维形状。不仅放宽了对光源和物体表面的限制,而且精度及鲁棒性均有很大提高。     

Wavelet-Texture Method: Appearance Compression by Polarization, Parametric Reflection Model, and Daubechies Wavelet

In order to create a photorealistic Virtual Reality model, we have to record the appearance of the object from different directions under different illuminations. In this paper, we propose a method that renders photorealistic images from a small amount of data. First, we separate the images of the object into a diffuse reflection component and a specular reflection component by using linear polarizers. Then, we estimate the parameters of the reflection model for each component. Finally, we compress the difference between the input images and the rendered images by using wavelet transform. At the rendering stage, we first calculate the diffuse and specular reflection images from the reflection parameters, then add the difference decompressed by inverse wavelet transform into the calculated reflection images, and finally obtain the photorealistic image of the object.     

基于图像分解的光照估计算法

增强现实技术的目的在于将计算机生成的虚拟物体叠加到真实场景中.实现良好的虚实融合需要对场景光照进行估算,针对高光场景,利用场景中的不同反射光信息对场景进行有效的光照估计,首先通过基于像素聚类方法的图像分解对图像进行反射光的分解,得到漫反射图和镜面反射图,对漫反射图进行进一步的本征图像分解,得到反照率图和阴影图;之后结合...     

Polarization multiplexing and demultiplexing for appearance-based modeling

Polarization has been used in numerous prior studies for separating diffuse and specular reflectance components, but in this work we show that it also can be used to separate surface reflectance contributions from individual light sources. Our approach is called polarization multiplexing and it has a significant impact in appearance modeling where the image as a function of illumination direction is needed. Multiple unknown light sources can illuminate the scene simultaneously, and the individual contributions to the overall surface reflectance are estimated. Polarization multiplexing relies on the relationship between the light source direction and the intensity modulation. Inverting this transformation enables the individual intensity contributions to be estimated. In addition to polarization multiplexing, we show that phase histograms from the intensity modulations can be used to estimate scene properties including the number of light sources