Interactive Rendering of Translucent Objects†

This paper presents a rendering method for translucent objects, in which viewpoint and illumination can be modified at interactive rates. In a preprocessing step, the impulse response to incoming light impinging at each surface point is computed and stored in two different ways: The local effect on close‐by surface points is modeled as a per‐texel filter kernel that is applied to a texture map representing the incident illumination. The global response (i.e. light shining through the object) is stored as vertex‐to‐vertex throughput factors for the triangle mesh of the object. During rendering, the illumination map for the object is computed according to the current lighting situation and then filtered by the precomputed kernels. The illumination map is also used to derive the incident illumination on the vertices which is distributed via the vertex‐to‐vertex throughput factors to the other vertices. The final image is obtained by combining the local and global response. We demonstrate the performance of our method for several models. ACM CSS: I.3.7 Computer Graphics—Three‐Dimensional Graphics and Realism Color Radiosity     

Gamut Constrained Illuminant Estimation

This paper presents a novel solution to the illuminant estimation problem: the problem of how, given an image of a scene taken under an unknown illuminant, we can recover an estimate of that light. The work is founded on previous gamut mapping solutions to the problem which solve for a scene illuminant by determining the set of diagonal mappings which take image data captured under an unknown light to a gamut of reference colours taken under a known light. Unfortunately, a diagonal model is not always a valid model of illumination change and so previous approaches sometimes return a null solution. In addition, previous methods are difficult to implement. We address these problems by recasting the problem as one of illuminant classification: we define a priori a set of plausible lights thus ensuring that a scene illuminant estimate will always be found. A plausible light is represented by the gamut of colours observable under it and the illuminant in an image is classified by determining the plausible light whose gamut is most consistent with the image data. We show that this step (the main computational burden of the algorithm) can be performed simply and efficiently by means of a non-negative least-squares optimisation. We report results on a large set of real images which show that it provides excellent illuminant estimation, outperforming previous algorithms. First online version published in February, 2006     

PIAFusion: A progressive infrared and visible image fusion network based on illumination aware

Infrared and visible image fusion aims to synthesize a single fused image containing salient targets and abundant texture details even under extreme illumination conditions. However, existing image fusion algorithms fail to take the illumination factor into account in the modeling process. In this paper, we propose a progressive image fusion network based on illumination-aware, termed as PIAFusion, which adaptively maintains the intensity distribution of salient targets and preserves texture information in the background. Specifically, we design an illumination-aware sub-network to estimate the illumination distribution and calculate the illumination probability. Moreover, we utilize the illumination probability to construct an illumination-aware loss to guide the training of the fusion network. The cross-modality differential aware fusion module and halfway fusion strategy completely integrate common and complementary information under the constraint of illumination-aware loss. In addition, a new benchmark dataset for infrared and visible image fusion, i.e., Multi-Spectral Road Scenarios (available at https://github.com/Linfeng-Tang/MSRS), is released to support network training and comprehensive evaluation. Extensive experiments demonstrate the superiority of our method over state-of-the-art alternatives in terms of target maintenance and texture preservation. Particularly, our progressive fusion framework could round-the-clock integrate meaningful information from source images according to illumination conditions. Furthermore, the application to semantic segmentation demonstrates the potential of our PIAFusion for high-level vision tasks. Our codes will be available at https://github.com/Linfeng-Tang/PIAFusion.     

Adaptive Records for Irradiance Caching

Irradiance Caching is one of the most widely used algorithms to speed up global illumination. In this paper, we propose an algorithm based on the Irradiance Caching scheme that allows us (1) to adjust the density of cached records according to illumination changes and (2) to efficiently render the high‐frequency illumination changes. To achieve this, a new record footprint is presented. Although the original method uses records having circular footprints depending only on geometrical features, our record footprints have a more complex shape which accounts for both geometry and irradiance variations. Irradiance values are computed using a classical Monte Carlo ray tracing method that simplifies the determination of nearby objects and the pre‐computation of the shape of the influence zone of the current record. By gathering irradiance due to all the incident rays, illumination changes are evaluated to adjust the footprint’s records. As a consequence, the record footprints are smaller where illumination gradients are high. With this technique, the record density depends on the irradiance variations. Strong variations of irradiance (due to direct contributions for example) can be handled and evaluated accurately. Caching direct illumination is of high importance, especially in the case of scenes having many light sources with complex geometry as well as surfaces exposed to daylight. Recomputing direct illumination for the whole image can be very time‐consuming, especially for walkthrough animation rendering or for high‐resolution pictures. Storing such contributions in the irradiance cache seems to be an appropriate solution to accelerate the final rendering pass.     

Specular Highlight Removal for Real‐world Images

Removing specular highlight in an image is a fundamental research problem in computer vision and computer graphics. While various methods have been proposed, they typically do not work well for real‐world images due to the presence of rich textures, complex materials, hard shadows, occlusions and color illumination, etc. In this paper, we present a novel specular highlight removal method for real‐world images. Our approach is based on two observations of the real‐world images: (i) the specular highlight is often small in size and sparse in distribution; (ii) the remaining diffuse image can be represented by linear combination of a small number of basis colors with the sparse encoding coefficients. Based on the two observations, we design an optimization framework for simultaneously estimating the diffuse and specular highlight images from a single image. Specifically, we recover the diffuse components of those regions with specular highlight by encouraging the encoding coefficients sparseness using L₀ norm. Moreover, the encoding coefficients and specular highlight are also subject to the non‐negativity according to the additive color mixing theory and the illumination definition, respectively. Extensive experiments have been performed on a variety of images to validate the effectiveness of the proposed method and its superiority over the previous methods.     

Adapting softcopy color reproduction to ambient illumination

Abstract— The perceived colors of an image seen on a self‐luminous display are affected by ambient illumination. The ambient light reflected from the display faceplate is mixed with the image‐forming light emitted by the display. In addition to this direct physical effect of viewing flare, ambient illumination causes perceptual changes by affecting the adaptation state of the viewer's visual system. This paper first discusses these effects and how they can be compensated, outlining a display system able to adjust its output based on prevailing lighting conditions. The emphasis is on compensating for the perceptual effects of viewing conditions by means of color‐appearance modeling. The effects of varying the degree of chromatic adaptation parameter D and the surround compensation parameters c and N_c of the CIECAM97s color‐appearance model were studied in psychophysical experiments. In these memory‐based paired comparison experiments, the observers judged the appearance of images shown on an LCD under three different ambient‐illumination conditions. The dependence of the optimal parameter values on the level of ambient illumination was evident. The results of the final experiment, using a category scaling technique, showed the benefit of using the color‐appearance model with the optimized parameters in compensating for the perceptual changes caused by varying ambient illumination.     

基于图像的光照模型研究综述

从传统图形学的绘制技术与基于图像的绘制技术相结合的角度出发,以全光函数这个基于图像的绘制技术的理论基础为核心,概括性地提出基于图像的光照研究的基本任务实际上是对全光函数的采样、重建、合成和重采样的过程,并进一步地指出,基于图像的光照研究的重要意义在于扩展了原有基于图像的绘制技术中只能改变视点位置和视线方向的限制,使之可以通过改变场景本身的组成成分产生出更加丰富的光照效果。同时,该文综述性地分析了近期内有关基于图像的光照问题的部分研究工作,并从如何改变场景光照条件的角度出发,按照所使用的光照模型的不同,将这些方法分成三大类,即利用传统光照模型的方法、利用基于图像的光照模型的方法以及无需光照模型的方法。并从这个分类框架出发,进一步分析指出,利用基于图像的光照模型的方法将是未来研究的重点,并沿着这一方向尝试性地提出了一种新的模型。     

Relighting with the Reflected Irradiance Field: Representation,Sampling and Reconstruction

Image-based relighting (IBL) is a technique to change the illumination of an image-based object/scene. In this paper, we define a representation called the reflected irradiance field which records the light reflected from a scene as viewed at a fixed viewpoint as a result of moving a point light source on a plane. It synthesizes a novel image under a different illumination by interpolating and superimposing appropriate recorded samples. Furthermore, we study the minimum sampling problem of the reflected irradiance field, i.e., how many light source positions are needed. We find that there exists a geometry-independent bound for the sampling interval whenever the second-order derivatives of the surface BRDF and the minimum depth of the scene are bounded. This bound ensures that when the novel light source is on the plane, the error in the reconstructed image is controlled by a given tolerance, regardless of the geometry. We also analyze the bound of depth error so that the extra reconstruction error can also be governed when the novel light source is off-plane. Experiments on both synthetic and real surfaces are conducted to verify our analysis.     

The same origin ray set query for realistic illumination: Algorithm and analysis

The same origin ray set (SORS) is a computational primitive which can be used by ray tracing, radiosity and multiple pass illumination simulation algorithms for realistic image synthesis. A SORS consists of a set of rays emanating from the same point in space. The SORS query computes the first object intersected by each ray and the intersection point. In this paper we present an efficient projection algorithm for computing a SORS query for polygonal scenes. The algorithm achieves its efficiency by separating ray-polygon intersection detection from the computation of the intersection point between the ray and the polygon's plane. The algorithm can be integrated with all current illumination acceleration schemes. We analyse the projection algorithm and compare it to the alternative of computing the SORS query one ray at a time. The analysis' results are expressed in terms of a few intuitive parameters, measuring the success of the acceleration scheme in culling irrelevant objects and the concentration of the ray set. The projection algorithm can be up to five times more efficient, depending on these parameters and the quality of the image. The relative advantage of the projection increases with image quality.     

Flood fill mean shift: A robust segmentation algorithm

In this paper, the flood fill mean shift (FFMS) is introduced. This algorithm is developed for robust segmentation by improving the mean shift (MS) through the flood fill (FF) technique, instead of relying on spatial bandwidth. Due to this exchange, the FFMS involves only one parameter, the range bandwidth, which is not sensitive and is able to acquire global characteristics. If the image parts affected by the illumination changes are sufficiently small and their boundaries are not clear, the illumination effects do not influence the mode seeking procedure of the proposed FFMS. To prove the usefulness and the validity of our algorithm, we present several experiments and analysis of the results.     

Illumination of image-based objects

A new data representation of image-based objects is presented. With this representation, the user can change the illumination as well as the viewpoint of an image-based scene. Physically correct imagery can be generated without knowing any geometrical information (e.g. depth or surface normal) of the scene. By treating each pixel on the image plane as a surface element, we can measure its apparent BRDF (bidirectional reflectance distribution function) by collecting information in the sampled images. These BRDFs allow us to calculate the correct pixel colour under a new illumination set-up by fitting the intensity, direction and number of the light sources. We demonstrate that the proposed representation allows re-rendering of the scene illuminated by different types of light sources. Moreover, two compression schemes, spherical harmonics and discrete cosine transform, are proposed to compress the huge amount of tabular BRDF data. © 1998 John Wiley & Sons, Ltd.     

What Is the Set of Images of an Object Under All Possible Illumination Conditions?

The appearance of an object depends on both the viewpoint from which it is observed and the light sources by which it is illuminated. If the appearance of two objects is never identical for any pose or lighting conditions, then–in theory–the objects can always be distinguished or recognized. The question arises: What is the set of images of an object under all lighting conditions and pose? In this paper, we consider only the set of images of an object under variable illumination, including multiple, extended light sources and shadows. We prove that the set of n-pixel images of a convex object with a Lambertian reflectance function, illuminated by an arbitrary number of point light sources at infinity, forms a convex polyhedral cone in IRⁿ and that the dimension of this illumination cone equals the number of distinct surface normals. Furthermore, the illumination cone can be constructed from as few as three images. In addition, the set of n-pixel images of an object of any shape and with a more general reflectance function, seen under all possible illumination conditions, still forms a convex cone in IRⁿ. Extensions of these results to color images are presented. These results immediately suggest certain approaches to object recognition. Throughout, we present results demonstrating the illumination cone representation.

     

Diffusion Based Photon Mapping

Density estimation employed in multi‐pass global illumination algorithms give cause to a trade‐off problem between bias and noise. The problem is seen most evident as blurring of strong illumination features. In particular, this blurring erodes fine structures and sharp lines prominent in caustics. To address this problem, we introduce a photon mapping algorithm based on nonlinear anisotropic diffusion. Our algorithm adapts according to the structure of the photon map such that smoothing occurs along edges and structures and not across. In this way, we preserve important illumination features, while eliminating noise. We demonstrate the applicability of our algorithm through a series of tests. In the tests, we evaluate the visual and computational performance of our algorithm comparing it to existing popular algorithms.     

显著性检测指导的高光区域修复

目的为解决传统的基于光照模型的高光修复算法无法很好地对高光区域存在饱和现象的单幅图像进行处理这一问题,提出一种显著性检测指导的高光区域修复算法。方法首先在亮度空间应用显著性模型,实现高光区域的自动检测和标记,之后运用改进的Exemplar-Based算法,综合利用图像的邻域和边缘信息,对标记的高光区域进行自适应修复,去除图像中的高光。结果分别对仿真及自然场景下的高光图像进行测试,实验结果表明,与原修复算法和传统高光去除算法相比,所提算法的修复效果更符合人眼视觉、修复后的图像质量更好。结论本文算法与Exemplar-Based算法及Tan方法相比,对高光区域存在饱和现象的单幅图像有较好的修复效果,并且有效地克服了传统高光去除算法受光照模型限制的缺点。