The Shadow Meets the Mask: Pyramid-Based Shadow Removal

In this paper we propose a novel method for detecting and removing shadows from a single image thereby obtaining a high‐quality shadow‐free image. With minimal user assistance, we first identify shadowed and lit areas on the same surface in the scene using an illumination‐invariant distance measure. These areas are used to estimate the parameters of an affine shadow formation model. A novel pyramid‐based restoration process is then applied to produce a shadow‐free image, while avoiding loss of texture contrast and introduction of noise. Unlike previous approaches, we account for varying shadow intensity inside the shadowed region by processing it from the interior towards the boundaries. Finally, to ensure a seamless transition between the original and the recovered regions we apply image inpainting along a thin border. We demonstrate that our approach produces results that are in most cases superior in quality to those of previous shadow removal methods. We also show that it is possible to easily composite the extracted shadow onto a new background or modify its size and direction in the original image.     

Efficient Shadow Removal Using Subregion Matching Illumination Transfer

This paper proposes a new shadow removal approach for input single natural image by using subregion matching illumination transfer We first propose an effective and automatic shadow detection algorithm incorporating global successive thresholding scheme and local boundary refinement. Then we present a novel shadow removal algorithm by performing illumination transfer on the matched subregion pairs between the shadow regions and non‐shadow regions, and this method can process complex images with different kinds of shadowed texture regions and illumination conditions. In addition, we develop an efficient shadow boundary processing method by using alpha matte interpolation, which produces seamless transition between the shadow and non‐shadow regions. Experimental results demonstrate the capabilities of our algorithm in both the shadow removal quality and performance.     

Editing Soft Shadows in a Digital Photograph

In this article, we develop tools for shadow modification in images where a shadowed region is characterized by soft boundaries with varying sharpness along the shadow edges. Modeling shadow edges presents an interesting challenge because they can vary from infinitely sharp edges for shadows produced by a point light source to extremely soft edges for shadows produced by large area light sources. We propose an entirely image-based shadow editing tool for a single-input image. This technique for modeling, editing, and rendering shadow edges in a photograph or a synthetic image lets users separate the shadow from the rest of the image and make arbitrary adjustments to its position, sharpness, and intensity. These machine-adjustable photographs can offer interactivity that might improve images' expressiveness and help us investigate the influence of boundary sharpness on the perception of object-to-object contact, as well as understand how humans assess shadows to estimate object height above a ground plane     

利用光源投影坐标系处理SFS中的自阴影问题

对于由明暗恢复形状(SFS)问题来说，自阴影的存在是影响其重构效果的因素之一，应该加以特殊的处理。为了使自阴影的处理不延伸破坏其他区域，并能保护整体的重构形状，提出了一套从阴影分割、坐标系变换到数据拟合的自阴影处理方法。该方法使用了根据图像平面以及光源方向定义的光源投影坐标系，以降低自阴影处理问题的复杂性，从而使得可以通过逐行恢复图像内跨各自阴影数据区段高度变化量的方式来实现其基本处理要求，并可采用抛物线拟合的方法来完成重构的阴影区形状与其他部分的光滑连接。实验表明，该方法能有效地改善SFS的重构形状。     

Detecting and removing specularities in facial images

Specularities often confound algorithms designed to solve computer vision tasks such as image segmentation, object detection, and tracking. These tasks usually require color image segmentation to partition an image into regions, where each region corresponds to a particular material. Due to discontinuities resulting from shadows and specularities, a single material is often segmented into several sub-regions. In this paper, a specularity detection and removal technique is proposed that requires no camera calibration or other a priori information regarding the scene. The approach specifically addresses detecting and removing specularities in facial images. The image is first processed by the Luminance Multi-Scale Retinex [B.V. Funt, K. Barnard, M. Brockington, V. Cardei, Luminance-Based Multi-Scale Retinex, AIC’97, Kyoto, Japan, May 1997]. Second, potential specularities are detected and a wavefront is generated outwards from the peak of the specularity to its boundary or until a material boundary has been reached. Upon attaining the specularity boundary, the wavefront contracts inwards while coloring in the specularity until the latter no longer exists. The third step is discussed in a companion paper [M.D. Levine, J. Bhattacharyya, Removing shadows, Pattern Recognition Letters, 26 (2005) 251–265] where a method for detecting and removing shadows has also been introduced. The approach involves training Support Vector Machines to identify shadow boundaries based on their boundary properties. The latter are used to identify shadowed regions in the image and then assign to them the color of non-shadow neighbors of the same material as the shadow. Based on these three steps, we show that more meaningful color image segmentations can be achieved by compensating for illumination using the Illumination Compensation Method proposed in this paper. It is also demonstrated that the accuracy of facial skin detection improves significantly when this illumination compensation approach is used. Finally, we show how illumination compensation can increase the accuracy of face recognition.     

Estimating ground fractional vegetation cover using the double-exposure method

Fractional vegetation cover (FVC) is a key parameter in ecological models. It is important to determine the ground FVC quickly and accurately in studies of soil erosion, surface energy balance, and carbon cycling. As one of the FVC ground measurement methods, the photographic method is easy to operate with relatively high precision. However, its classification result showed poor accuracy when an image of a high-contrast scene contained a shadow region where a low signal-to-noise ratio (SNR) existed, because the single-exposure image in the photographic method did not contain sufficient surface information about both the illuminated and shadowed parts. This article presents application of a double-exposure photographic method to determine vegetation cover in the shadow region of an image. It consists of two measurements used in acquiring images (normal and over-exposure) and one image-processing part to handle the obtained images. Illuminated vegetation and soil, as well as the shadow region, was classified with the normally exposed image in the intensity, hue, and saturation (IHS) colour space, and the shadow region was further classified as shadowed vegetation and shadowed soil using the over-exposed image. The results indicate that the over-exposed image reduced the average bias of the FVC in the shadow region from 15.40% to ?4.14% and the root mean square error (RMSE) from 0.174 to 0.066. The RMSE of the entire scene was 0.055 in the over-exposed image and 0.092 in the single-exposed image. The double-exposure method also showed a better classification result than the high dynamic range method in deep shadow regions. This study shows that this method is capable of distinguishing vegetation and soil in the shadow region and thus it is an effective and accurate method for ground FVC measurement.     

基于图割的单幅图像影子检测

为了准确检测单幅图像中的影子, 提出一种基于图割的影子检测方法. 首先,使用均值漂移将原始图像分割为若干区域并记录区域之间的边界. 其次,利用支持向量机分类器分别获得分割图像中的候选影子边界和候选影子非影子区域对. 然后,利用候选影子边界两侧的区域信息及候选影子非影子区域对信息构造一个能量函数, 该能量函数反映了将图像中一部分区域划分为影子区域而另一部分区域划分为非影子区域时所需的代价. 再次,结合该能量函数构造出无向图,并证明所构造的无向图的最小割对应能量函数的最小值. 最后,通过图割算法求解该能量函数得到最终的影子检测结果. 实验结果表明,与现有代表最新进展的单幅图像影子检测方法相比,所提方法提高了影子检测结果的准确性和连续性.     

Restoration of shadows in multispectral imagery using surface reflectance relationships with nearby similar areas

The presence of shadows in optical satellite images limits the application of remote-sensing technology. It is important to restore shadow radiance information for improving information extraction from remote-sensing images. Several shadow-restoration methods have been developed using complex statistical relationships between shadowed areas and their nearby sunlit areas. In this study, a simple shadow-restoration approach was proposed based on the surface reflectance equality relationship (RER) under the assumption that the surface reflectance of a feature in the shadowed area is equal to that of the same feature in the nearby sunlit area. This approach reduces the number of parameters, thus reducing the error propagated by the uncertainties of extra parameters. The new RER method was tested with three multispectral images with different shadow features. By comparing RER with the widely used mean and variance transformation, the RER was shown to be capable of restoring the image colours, texture, tone, and brightness of the shadowed areas to a visually satisfactory level. Quantitative analysis suggests that RER can help to restore the reflectance of shadow features accurately and has robust performance for a variety of land-surface types. Moreover, RER can be effectively used to restore the spectral shape information of shadow features, which is particularly important when applying RER to the restoration of multispectral imagery for the purpose of image classification.     

Video Shadow Removal Using Spatio‐temporal Illumination Transfer

Shadow removal for videos is an important and challenging vision task. In this paper, we present a novel shadow removal approach for videos captured by free moving cameras using illumination transfer optimization. We first detect the shadows of the input video using interactive fast video matting. Then, based on the shadow detection results, we decompose the input video into overlapped 2D patches, and find the coherent correspondences between the shadow and non‐shadow patches via discrete optimization technique built on the patch similarity metric. We finally remove the shadows of the input video sequences using an optimized illumination transfer method, which reasonably recovers the illumination information of the shadow regions and produces spatio‐temporal shadow‐free videos. We also process the shadow boundaries to make the transition between shadow and non‐shadow regions smooth. Compared with previous works, our method can handle videos captured by free moving cameras and achieve better shadow removal results. We validate the effectiveness of the proposed algorithm via a variety of experiments.     

A Randomized Approach for Patch-based Texture Synthesis using Wavelets

We present a wavelet‐based approach for selecting patches in patch‐based texture synthesis. We randomly select the first block that satisfies a minimum error criterion, computed from the wavelet coefficients (using 1D or 2D wavelets) for the overlapping region. We show that our wavelet‐based approach improves texture synthesis for samples where previous work fails, mainly textures with prominent aligned features. Also, it generates similar quality textures when compared against texture synthesis using feature maps with the advantage that our proposed method uses implicit edge information (since it is embedded in the wavelet coefficients) whereas feature maps rely explicitly on edge features. In previous work, the best patches are selected among all possible using a L2 norm on the RGB or grayscale pixel values of boundary zones. The L2 metric provides the raw pixel‐to‐pixel difference, disregarding relevant image structures — such as edges — that are relevant in the human visual system and therefore on synthesis of new textures.     

Shadowed sets of type‐II: Representing and computing shadowiness in shadowed sets

Shadowed sets were developed to interpret and determine the required pair of thresholds, which resolves the issue with Zadeh's proposal about a three‐way approximation of fuzzy sets. However, a serious shortcoming of shadowed sets is that they are not capable of coming to grips with the degree of shadowiness (or doubt) associated with the elements of a shadowed set, especially the elements in the doubtful zone (i.e., shadow region) and as a result, all the elements in the shadow region are shown no commitment during the process of decision making. This paper proposes an alternative construct that keeps record of the degree of doubt associated with the elements in a shadowed set. It aims at introducing, discussing the concepts of shadowed sets of type‐II, and studying their related operations. Shadowed sets of type‐II provide a capable framework that despite localizing the uncertainty associated with fuzzy sets; adheres to the gradual transition of membership grades of the elements in the doubtful zone.     

Fast Shadow Removal Using Adaptive Multi‐Scale Illumination Transfer

In this paper, we present a new method for removing shadows from images. First, shadows are detected by interactive brushing assisted with a Gaussian Mixture Model. Secondly, the detected shadows are removed using an adaptive illumination transfer approach that accounts for the reflectance variation of the image texture. The contrast and noise levels of the result are then improved with a multi‐scale illumination transfer technique. Finally, any visible shadow boundaries in the image can be eliminated based on our Bayesian framework. We also extend our method to video data and achieve temporally consistent shadow‐free results.     

Corresponding regions for shadow restoration in satellite high-resolution images

ABSTRACT

High spatial resolution images available by satellites such as Ikonos, Quickbird, and WorldView-2 provide more information for remote sensing applications, such as object detection, classification, change detection, and object mapping. The presence of shadow reduces the amount of information that can be extracted and consequently makes these applications more difficult or even impossible. In this article, a shadow restoration approach for high-resolution satellite images is proposed. The approach detects the shadow area and segments the image into regions according to the land surface type. Then, shadow restoration is carried out for each region based on the degree of correspondence between shadow and neighbouring non-shadow regions. The proposed approach is applied to study areas from Ikonos and WorldView-2 satellite images. A comparison to the standard approaches for shadow restoration is performed, and an accuracy assessment is carried out by visual inspection and land-cover classification. The results show that the enhanced shadow regions using the proposed approach have better appearances and are highly compatible with their surrounding non-shadow regions. In addition, the overall accuracy is higher than those of the standard approaches.     

Soft Shadow Maps: Efficient Sampling of Light Source Visibility

Shadows, particularly soft shadows, play an important role in the visual perception of a scene by providing visual cues about the shape and position of objects. Several recent algorithms produce soft shadows at interactive rates, but they do not scale well with the number of polygons in the scene or only compute the outer penumbra. In this paper, we present a new algorithm for computing interactive soft shadows on the GPU. Our new approach provides both inner‐ and outer‐penumbra for a modest computational cost, providing interactive frame‐rates for models with hundreds of thousands of polygons. Our technique is based on a sampled image of the occluders, as in shadow map techniques. These shadow samples are used in a novel manner, computing their effect on a second projective shadow texture using fragment programs. In essence, the fraction of the light source area hidden by each sample is accumulated at each texel position of this Soft Shadow Map. We include an extensive study of the approximations caused by our algorithm, as well as its computational costs.     

Three-Class Markovian Segmentation of High-Resolution Sonar Images

This paper presents an original method for analyzing, in an unsupervised way, images supplied by high resolution sonar. We aim at segmenting the sonar image into three kinds of regions: echo areas (due to the reflection of the acoustic wave on the object), shadow areas (corresponding to a lack of acoustic reverberation behind an object lying on the sea-bed), and sea-bottom reverberation areas. This unsupervised method estimates the parameters of noise distributions, modeled by a Weibull probability density function (PDF), and the label field parameters, modeled by a Markov random field (MRF). For the estimation step, we adopt a maximum likelihood technique for the noise model parameters and a least-squares method to estimate the MRF prior model. Then, in order to obtain an accurate segmentation map, we have designed a two-step process that finds the shadow and the echo regions separately, using the previously estimated parameters. First, we introduce a scale-causal and spatial model called SCM (scale causal multigrid), based on a multigrid energy minimization strategy, to find the shadow class. Second, we propose a MRF monoscale model using a priori information (at different level of knowledge) based on physical properties of each region, which allows us to distinguish echo areas from sea-bottom reverberation. This technique has been successfully applied to real sonar images and is compatible with automatic processing of massive amounts of data.     

Partitioned Shadow Volumes

Real‐time shadows remain a challenging problem in computer graphics. In this context, shadow algorithms generally rely either on shadow mapping or shadow volumes. This paper rediscovers an old class of algorithms that build a binary space partition over the shadow volumes. For almost 20 years, such methods have received little attention as they have been considered lacking of both robustness and efficiency. We show that these issues can be overcome, leading to a simple and robust shadow algorithm. Hence we demonstrate that this kind of approach can reach a high level of performance. Our algorithm uses a new partitioning strategy which avoids any polygon clipping. It relies on a Ternary Object Partitioning tree, a new data structure used to find if an image point is shadowed. Our method works on a triangle soup and its memory footprint is fixed. Our experiments show that it is efficient and robust, including for finely tessellated models.     

一种基于无抽样小波变换的云阴影去除算法

提出了一种基于无抽样小波变换的人机交互半自动云阴影去除算法。首先通过无抽样小波变换将图像分解成高频部分和低频部分,通过半自动人机交互方式在低频图像中将图像分割为云区域、阴影区域、清晰地物区域3个部分,然后对各个独立云阴影区域分别进行低频信息补偿和高频信息补偿,最后通过无抽样小波反变换得到一幅过渡平滑,并对图像细节保存完好地去除了云阴影的图像。实验结果表明,该方法能够取得比传统的直方图匹配更好的效果。     

Point-Based Rendering of Non-Manifold Surfaces

We are concerned with producing high‐quality images of parametric and implicit surfaces, in particular those with non‐manifold features. We present a point‐based technique for rendering implicit surfaces that uses octree spatial subdivision with a natural interval exclusion test that guarantees that no parts of the surface are missed. This allows us to render non‐manifold implicit surfaces at speeds comparable to parametric surfaces. We also derive criteria that guarantee complete pixel coverage of the surface. The point‐based method allows for hidden surface elimination using a z‐buffer, and shadow casting using a shadow buffer. We illustrate the technique with a number of surfaces, and discuss its advantages and disadvantages.