Determining reflectance properties of an object using range andbrightness images

The authors discuss a method of recovering reflectance properties of a surface from a range image given by a range finder and a brightness image given by a standard TV camera. The Torrance-Sparrow model is used for the reflectance model. The model consists of the Lambertian and specular components: its reflectance properties consist of the relative strength between the Lambertian and specular components and specular sharpness as well as light source direction. An iterative least square fitting method is used to obtain these parameters based on the range and brightness images. An input image is segmented into four different parts using the parameters: Lambertian reflection, specular reflection, interreflection, and shadow part. The authors also reconstruct ideal images that consist of only Lambertian or specular reflection     

Terrain analysis using radar shape-from-shading

This paper develops a maximum a posteriori (MAP) probability estimation framework for shape-from-shading (SFS) from synthetic aperture radar (SAR) images. The aim is to use this method to reconstruct surface topography from a single radar image of relatively complex terrain. Our MAP framework makes explicit how the recovery of local surface orientation depends on the whereabouts of terrain edge features and the available radar reflectance information. To apply the resulting process to real world radar data, we require probabilistic models for the appearance of terrain features and the relationship between the orientation of surface normals and the radar reflectance. We show that the SAR data can be modeled using a Rayleigh-Bessel distribution and use this distribution to develop a maximum likelihood algorithm for detecting and labeling terrain edge features. Moreover, we show how robust statistics can be used to estimate the characteristic parameters of this distribution. We also develop an empirical model for the SAR reflectance function. Using the reflectance model, we perform Lambertian correction so that a conventional SFS algorithm can be applied to the radar data. The initial surface normal direction is constrained to point in the direction of the nearest ridge or ravine feature. Each surface normal must fall within a conical envelope whose axis is in the direction of the radar illuminant. The extent of the envelope depends on the corrected radar reflectance and the variance of the radar signal statistics. We explore various ways of smoothing the field of surface normals using robust statistics. Finally, we show how to reconstruct the terrain surface from the smoothed field of surface normal vectors. The proposed algorithm is applied to various SAR data sets containing relatively complex terrain structure.     

Light source position and reflectance estimation from a single view without the distant illumination assumption

Several techniques have been developed for recovering reflectance properties of real surfaces under unknown illumination. However, in most cases, those techniques assume that the light sources are located at infinity, which cannot be applied safely to, for example, reflectance modeling of indoor environments. In this paper, we propose two types of methods to estimate the surface reflectance property of an object, as well as the position of a light source from a single view without the distant illumination assumption, thus relaxing the conditions in the previous methods. Given a real image and a 3D geometric model of an object with specular reflection as inputs, the first method estimates the light source position by fitting to the Lambertian diffuse component, while separating the specular and diffuse components by using an iterative relaxation scheme. Our second method extends that first method by using as input a specular component image, which is acquired by analyzing multiple polarization images taken from a single view, thus removing its constraints on the diffuse reflectance property. This method simultaneously recovers the reflectance properties and the light source positions by optimizing the linearity of a log-transformed Torrance-Sparrow model. By estimating the object's reflectance property and the light source position, we can freely generate synthetic images of the target object under arbitrary lighting conditions with not only source direction modification but also source-surface distance modification. Experimental results show the accuracy of our estimation framework.     

Accurately estimating reflectance parameters for color and gloss reproduction

A new method is described to estimate diffuse and specular reflectance parameters using spectral images, which overcomes the dynamic range limitation of imaging devices. After eliminating the influences of illumination and camera on spectral images, reflection values are initially assumed as diffuse-only reflection components, and subjected to the least squares method to estimate diffuse reflectance parameters at each wavelength on each single surface particle. Based on the dichromatic reflection model, specular reflection components are obtained, and then subjected to the least squares method to estimate specular reflectance parameters for gloss intensity and surface roughness. Experiments were carried out using both simulation data and measured spectral images. Our results demonstrate that this method is capable of estimating diffuse and specular reflectance parameters precisely for color and gloss reproduction, without requiring preprocesses such as image segmentation and synthesis of high dynamic range images.     

Estimating the surface radiance function from single images

This paper describes a simple method for estimating the surface radiance function from single images of smooth surfaces made of materials whose reflectance function is isotropic and monotonic. The method makes implicit use of the Gauss map between the surface and a unit sphere. We assume that the material brightness is monotonic with respect to the angle between the illuminant direction and the surface normal. Under conditions in which the light source and the viewer directions are identical, we show how a tabular representation of the surface radiance function can be estimated using the cumulative distribution of image gradients. Using this tabular representation of the radiance function, surfaces may be rendered under varying light source direction by rotating the corresponding reflectance map on the Gauss sphere about the specular spike direction. We present a sensitivity study on synthetic and real-world imagery. We also present two applications which make use of the estimated radiance function. The first of these illustrates how the radiance function estimates can be used to render objects when the light and viewer directions are no longer coincident. The second application involves applying corrected Lambertian radiance to rough and shiny surfaces.     

Photometric stereo with an arbitrary number of illuminants

We present an optimal generalisation of the 4-light photometric stereo technique for an arbitrary number of Q illuminants. We assume that the surface reflectance can be approximated by the Lambertian model plus a specular reflection. The algorithm works in a recursive manner eliminating the pixel intensities affected by shadows or highlights, based on a least squares error technique, retaining only the information coming from illumination directions that can be used for photometric stereo reconstruction of the normal of the corresponding surface patch. We report results for both simulated and real surfaces and compare them with the results of other state of the art photometric stereo algorithms.     

The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows

We present an algorithm for separating the local gradient information and Lambertian color by using 4-source color photometric stereo in the presence of highlights and shadows. We assume that the surface reflectance can be approximated by the sum of a Lambertian and a specular component. The conventional photometric method is generalized for color images. Shadows and highlights in the input images are detected using either spectral or directional cues and excluded from the recovery process, thus giving more reliable estimates of local surface parameters.     

A Framework for the Construction of Reflectance Maps for Machine Vision

A reflectance map is the transfer function from surface orientation and illumination geometry to the surface normal, and in machine vision it plays a fundamental role in the reconstruction of surface by shape-from-shading and photometric stereo algorithms. While reflectance maps for Lambertain and specular surfaces are well understood, maps for real-world diffusely reflecting surfaces are scant. In this paper, the fundamental mechanisms of reflection from such surfaces are reviewed. Based on this, it is proposed that for point light source illumination, the diffuse component of the reflectance map has three terms: a forescatter term, a normal term, and a backscatter term. The physical origin of the three terms is discussed in detail and useful mathematical expressions are obtained for them. The range of applicability of the proposed reflectance maps is established, and an example of their use in photometric stereo is provided. The mathematical form of the reflectance map obtained from physical theories is amenable to generalization and such a generalization is called the m-lobed reflectance map is proposed.     

Separation of Reflection Components Using Color and Polarization

Specular reflections and interreflections produce strong highlights in brightness images. These highlights can cause vision algorithms for segmentation, shape from shading, binocular stereo, and motion estimation to produce erroneous results. A technique is developed for separating the specular and diffuse components of reflection from images. The approach is to use color and polarization information, simultaneously, to obtain constraints on the reflection components at each image point. Polarization yields local and independent estimates of the color of specular reflection. The result is a linear subspace in color space in which the local diffuse component must lie. This subspace constraint is applied to neighboring image points to determine the diffuse component. In contrast to previous separation algorithms, the proposed method can handle highlights on surfaces with substantial texture, smoothly varying diffuse reflectance, and varying material properties. The separation algorithm is applied to several complex scenes with textured objects and strong interreflections. The separation results are then used to solve three problems pertinent to visual perception; determining illumination color, estimating illumination direction, and shape recovery.     

Mixture of spherical distributions for single-view relighting

We present a method for simultaneously estimating the illumination of a scene and the reflectance property of an object from single view images - a single image or a small number of images taken from the same viewpoint. We assume that the illumination consists of multiple point light sources and the shape of the object is known. First, we represent the illumination on the surface of a unit sphere as a finite mixture of von Mises-Fisher distributions based on a novel spherical specular reflection model that well approximates the Torrance-Sparrow reflection model. Next, we estimate the parameters of this mixture model including the number of its component distributions and the standard deviation of them, which correspond to the number of light sources and the surface roughness, respectively. Finally, using these results as the initial estimates, we iteratively refine the estimates based on the original Torrance-Sparrow reflection model. The final estimates can be used to relight single-view images such as altering the intensities and directions of the individual light sources. The proposed method provides a unified framework based on directional statistics for simultaneously estimating the intensities and directions of an unknown number of light sources as well as the specular reflection parameter of the object in the scene.     

存在镜面反射时的立体匹配研究

传统的立体匹配方法建立在Lambertian的漫反射模型之上,漫反射模型的立体匹配在一个图像中大部分是有效的,但是在处理图像中包含镜面反射部分时结果会产生严重的匹配错误.为了解决个问题,根据二色反射模型引入一种漫反射和镜面反射的分离方法,匹配图像中存在镜面反射部分时先滤除掉镜面反射再进行匹配,在镜面反射部分也能匹配得到正确的视差.实验结果证明该方法很有效.     

Neural computation approach for developing a 3D shapereconstruction model

The shape from shading problem refers to the well-known fact that most real images usually contain specular components and are affected by unknown reflectivity. In this paper, these limitations are addressed and a new neural-based 3D shape reconstruction model is proposed. The idea behind this approach is to optimize a proper reflectance model by learning the parameters of the proposed neural reflectance model. In order to do this, new neural-based reflectance models are presented. The feedforward neural network (FNN) model is able to generalize the diffuse term, while the RBF model is able to generalize the specular term. A hybrid structure of FNN-based and RBF-based models is also presented because most real surfaces are usually neither Lambertian models nor ideally specular models. Experimental results, including synthetic and real images, are presented to demonstrate the performance of our approach given different specular effects, unknown illuminate conditions, and different noise environments.     

A study on local photometric models and their application to robust tracking

Since modeling reflections in image processing is a difficult task, most computer vision algorithms assume that objects are Lambertian and that no lighting change occurs. Some photometric models can partly answer this issue by assuming that the lighting changes are the same at each point of a small window of interest. Through a study based on specular reflection models, we explicit the assumptions on which these models are implicitly based and the situations in which they could fail.This paper proposes two photometric models, which compensate for specular highlights and lighting variations. They assume that photometric changes vary smoothly on the window of interest. Contrary to classical models, the characteristics of the object surface and the lighting changes can vary in the area being observed. First, we study the validity of these models with respect to the acquisition setup: relative locations between the light source, the sensor and the object as well as the roughness of the surface. Then, these models are used to improve feature points tracking by simultaneously estimating the photometric and geometric changes. The proposed methods are compared to well-known tracking methods robust to affine photometric changes. Experimental results on specular objects demonstrate the robustness of our approaches to specular highlights and lighting changes.     

基于混合反射模型的SFS有限元方法的研究

提出了基于混合反射模型的由明暗恢复物体三维形状的有限元算法。用正方形面元逼近光滑曲面,把曲面表示为所有节点基函数的线性组合;基于既含有漫反射成分又有镜面反射成分的混合模型,结合节点基函数,将反射图线性化。考虑数字图像的特点,直接使用离散形式的SFS问题的亮度约束形式,用最小化方法得到高度满足的线性方程;使用Kaczmarz算法计算出表面三维形状。使用合成图像和实际图像验证该文算法的有效性,探讨了该算法的性能。     

Improved Diffuse Reflection Models for Computer Vision

There are many computational vision techniques that fundamentally rely upon assumptions about the nature of diffuse reflection from object surfaces consisting of commonly occurring nonmetallic materials. Probably the most prevalent assumption made about diffuse reflection by computer vision researchers is that its reflected radiance distribution is described by the Lambertian model, whether the surface is rough or smooth. While computationally and mathematically a relatively simple model, in physical reality the Lambertian model is deficient in accurately describing the reflected radiance distribution for both rough and smooth nonmetallic surfaces. Recently, in computer vision diffuse reflectance models have been proposed separately for rough, and, smooth nonconducting dielectric surfaces each of these models accurately predicting salient non-Lambertian phenomena that have important bearing on computer vision methods relying upon assumptions about diffuse reflection. Together these reflectance models are complementary in their respective applicability to rough and smooth surfaces. A unified treatment is presented here detailing important deviations from Lambertian behavior for both rough and smooth surfaces. Some speculation is given as to how these separate diffuse reflectance models may be combined.     

Rendering Caustics on Non-Lambertian Surfaces

This paper presents a new technique for rendering caustics on non-Lambertian surfaces. The method is based on an extension of the photon map which removes previous restrictions limiting the usage to Lambertian surfaces. We add information about the incoming direction to the photons and this allows us to combine the photon map with arbitrary reflectance functions. By using a cone-filter we improve the quality of the radiance estimate in particular at discontinuities. Furthermore we introduce balancing of the photon map which not only reduces the memory requirements but also significantly reduces the rendering time. We have used the method to render caustics on surfaces with reflectance functions varying from Lambertian to glossy specular.     

Vector transport for shape-from-shading

In this paper we describe a new shape-from-shading method. We show how the parallel transport of surface normals can be used to impose curvature consistency and also to iteratively update surface normal directions so as to improve the brightness error. We commence by showing how to make local estimates of the Hessian matrix from surface normal information. With the local Hessian matrix to hand, we develop an “EM-like” algorithm for updating the surface normal directions. At each image location, parallel transport is applied to the neighbouring surface normals to generate a sample of local surface orientation predictions. From this sample, a local weighted estimate of the image brightness is made. The transported surface normal which gives the brightness prediction which is closest to this value is selected as the revised estimate of surface orientation. The revised surface normals obtained in this way may in turn be used to re-estimate the Hessian matrix, and the process iterated until stability is reached. We experiment with the method on a variety of real world and synthetic data. Here we explore the properties of the fields of surface normals and the height data delivered by the method.     

Shape and motion from simultaneous equations with closed-loop solution

This paper proposes a method for simultaneously estimating 2D image motion and 3D object shape and motion from only two frames. The problem is formulated in a system of equations, including the differential epipolar constraint, a newly derived optical flow equation and surface normal constraint, under the assumption of perspective projection, rigid motion, Lambertian reflectance and distant lighting. A closed-loop solver is constructed based on the simultaneous equations to export accurate estimate for optical flow as well as dense shape and motion. Experimental results are also provided.     

Enhanced 3D shape recovery using the neural-based hybrid reflectance model.

It is known that most real surfaces usually are neither perfectly Lambertian model nor ideally specular model; rather, they are formed by the hybrid structure of these two models. This hybrid reflectance model still suffers from the noise, strong specular, and unknown reflectivity conditions. In this article, these limitations are addressed, and a new neural-based hybrid reflectance model is proposed. The goal of this method is to optimize a proper reflectance model by learning the weight and parameters of the hybrid structure of feedforward neural networks and radial basis function networks and to recover the 3D object shape by the shape from shading technique with this resulting model. Experimental results, including synthetic and real images, were performed to demonstrate the performance of the proposed reflectance model in the case of different specular effects and noise environments.     

双边滤波的改进高光去除

目的 由于非均匀光照条件下,物体表面通常出现块状的强反射区域,传统的去高光方法在还原图像时容易造成颜色失真或者边缘的丢失。针对这些缺点,提出一种改进的基于双边滤波的去高光方法。方法 首先通过双色反射模型变换得到镜面反射分量与最大漫反射色度之间的转换关系,然后利用阈值将图像的像素点分为两类,将仅含漫反射分量的像素点与含有镜面反射分量的像素点分离开来,对两类像素点的最大漫反射色度分别做估计,接着以估计的最大漫反射色度的相似度作为双边滤波器的值域,同时以图像的最大色度图作为双边滤波的引导图保边去噪,进而达到去除镜面反射分量的目的。结果 以经典的高光图像作为处理对象,对含有镜面反射和仅含漫反射的像素点分别做最大漫反射色度估计,再以该估计图作为双边滤波的引导图,不仅能去除镜面反射分量还能有效的保留图像的边缘信息,最大程度的还原图像细节颜色,并且解决了原始算法处理结果中R、G、B三通道相似的像素点所出现的颜色退化问题。用改进的双边滤波去高光算法对50幅含高光的图像做处理,并将该算法与Yang方法和Shen方法分别作对比,结果图的峰值信噪比（PSNR）也分别平均提高4.17%和8.40%,所提算法的处理效果更符合人眼视觉,图像质量更好。结论 实验结果表明针对含镜面反射的图像,本文方法能够更有效去除图像的多区域局部高光,完成对图像的复原,可为室内外光照不匀情况下所采集图像的复原提供有效理论基础。