Image-based face illumination transferring using logarithmic total variation models

In this paper, we present a novel image-based technique that transfers illumination from a source face image to a target face image based on the Logarithmic Total Variation (LTV) model. Our method does not require any prior information regarding the lighting conditions or the 3D geometries of the underlying faces. We first use a Radial Basis Functions (RBFs)-based deformation technique to align key facial features of the reference 2D face with those of the target face. Then, we employ the LTV model to factorize each of the two aligned face images to an illumination-dependent component and an illumination-invariant component. Finally, illumination transferring is achieved by replacing the illumination-dependent component of the target face by that of the reference face. We tested our technique on numerous grayscale and color face images from various face datasets including the Yale face Database, as well as the application of illumination-preserved face coloring.     

Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain.

This paper presents a novel illumination normalization approach for face recognition under varying lighting conditions. In the proposed approach, a discrete cosine transform (DCT) is employed to compensate for illumination variations in the logarithm domain. Since illumination variations mainly lie in the low-frequency band, an appropriate number of DCT coefficients are truncated to minimize variations under different lighting conditions. Experimental results on the Yale B database and CMU PIE database show that the proposed approach improves the performance significantly for the face images with large illumination variations. Moreover, the advantage of our approach is that it does not require any modeling steps and can be easily implemented in a real-time face recognition system.     

Illumination invariant face recognition

The appearance of a face will vary drastically when the illumination changes. Variations in lighting conditions make face recognition an even more challenging and difficult task. In this paper, we propose a novel approach to handle the illumination problem. Our method can restore a face image captured under arbitrary lighting conditions to one with frontal illumination by using a ratio-image between the face image and a reference face image, both of which are blurred by a Gaussian filter. An iterative algorithm is then used to update the reference image, which is reconstructed from the restored image by means of principal component analysis (PCA), in order to obtain a visually better restored image. Image processing techniques are also used to improve the quality of the restored image. To evaluate the performance of our algorithm, restored images with frontal illumination are used for face recognition by means of PCA. Experimental results demonstrate that face recognition using our method can achieve a higher recognition rate based on the Yale B database and the Yale database. Our algorithm has several advantages over other previous algorithms: (1) it does not need to estimate the face surface normals and the light source directions, (2) it does not need many images captured under different lighting conditions for each person, nor a set of bootstrap images that includes many images with different illuminations, and (3) it does not need to detect accurate positions of some facial feature points or to warp the image for alignment, etc.     

Homomorphic filtering based illumination normalization method for face recognition

The appearance of a face image is severely affected by illumination conditions that will hinder the automatic face recognition process. To recognize faces under varying lighting conditions, a homomorphic filtering-based illumination normalization method is proposed in this paper. In this work, the effect of illumination is effectively reduced by a modified implementation of homomorphic filtering whose key component is a Difference of Gaussian (DoG) filter, and the contrast is enhanced by histogram equalization. The resulted face image is not only reduced illumination effect but also preserved edges and details that will facilitate the further face recognition task. Among others, our method has the following advantages: (1) neither does it need any prior information of 3D shape or light sources, nor many training samples thus can be directly applied to single training image per person condition; and (2) it is simple and computationally fast because there are mature and fast algorithms for the Fourier transform used in homomorphic filter. The Eigenfaces method is chosen to recognize the normalized face images. Experimental results on the Yale face database B and the CMU PIE face database demonstrate the significant performance improvement of the proposed method in the face recognition system for the face images with large illumination variations.     

Multiscale facial structure representation for face recognition under varying illumination

Facial structure of face image under lighting lies in multiscale space. In order to detect and eliminate illumination effect, a wavelet-based face recognition method is proposed in this paper. In this work, the effect of illuminations is effectively reduced by wavelet-based denoising techniques, and meanwhile the multiscale facial structure is generated. Among others, the proposed method has the following advantages: (1) it can be directly applied to single face image, without any prior information of 3D shape or light sources, nor many training samples; (2) due to the multiscale nature of wavelet transform, it has better edge-preserving ability in low frequency illumination fields; and (3) the parameter selection process is computationally feasible and fast. Experiments are carried out upon the Yale B and CMU PIE face databases, and the results demonstrate that the proposed method achieves satisfactory recognition rates under varying illumination conditions.     

Face recognition under arbitrary illumination using illuminated exemplars

Recently, the importance of face recognition has been increasingly emphasized since popular CCD cameras are distributed to various applications. However, facial images are dramatically changed by lighting variations, so that facial appearance changes caused serious performance degradation in face recognition. Many researchers have tried to overcome these illumination problems using diverse approaches, which have required a multiple registered images per person or the prior knowledge of lighting conditions. In this paper, we propose a new method for face recognition under arbitrary lighting conditions, given only a single registered image and training data under unknown illuminations. Our proposed method is based on the illuminated exemplars which are synthesized from photometric stereo images of training data. The linear combination of illuminated exemplars can represent the new face and the weighted coefficients of those illuminated exemplars are used as identity signature. We make experiments for verifying our approach and compare it with two traditional approaches. As a result, higher recognition rates are reported in these experiments using the illumination subset of Max-Planck Institute face database and Korean face database.     

基于球面谐波基图像的任意光照下的人脸识别

提出了一种基于球面谐波基图像的光照补偿算法,用以在任意光照条件下进行人脸识别.算法分两步进行：光照估计和光照补偿.基于人脸形状大致相同和每个人脸的反射率基本相等的假设,首先估计了输入人脸图像光照的9个低频谐波系数.根据光照估计的结果,提出了两种光照补偿方法：纹理图像和差图像.纹理图像为输入图像与其光照辐照图之商,与输入图像的光照条件无关.差图像为输入图像与平均人脸在相同光照下的图像之差,通过减去平均人脸在相同光照下的图像,减弱了光照的影响.在CMU-PIE人脸库和Yale B人脸库上的实验表明,通过光照补偿,不同光照下人脸图像识别率有了很大提高.     

OptiFuzz: a robust illumination invariant face recognition system and its implementation

Vision-based human face detection and recognition are widely used and have been shown to be effective in normal illumination conditions. Under severe illumination conditions, however, it is very challenging. In this paper, we address the effect of illumination on the face detection and the face recognition problem by introducing a novel illumination invariant method, called OptiFuzz. It is an optimized fuzzy-based illumination invariant method to solve the effect of illumination for photometric-based human face recognition. The rule of the Fuzzy Inference System is optimized by using a genetic algorithm. The Fuzzy’s output controls an illumination invariant model that is extended from Land’s reflectance model. We test our method by using Yale B Extended and CAS-PEAL face databases to represent the offline experiments, and several videos are recorded at our campus to represent the online indoor and outdoor experiments. Viola–Jones face detector and mutual subspace method are employed to handle the online face detection and face recognition experiments. Based on the experimental results, we can show that our algorithm outperforms the existing and the state-of-the-art methods in recognizing a specific person under variable lighting conditions with a significantly improved computation time. Other than that, using illumination invariant images is also effective in improving the face detection performance.     

Local contrast enhancement and adaptive feature extraction for illumination-invariant face recognition

Recognizing human faces in various lighting conditions is quite a difficult problem. The problem becomes more difficult when face images are taken in extremely high dynamic range scenes. Most of the automatic face recognition systems assume that images are taken under well-controlled illumination. The face segmentation as well as recognition becomes much simpler under such a constrained condition. However, illumination control is not feasible when a surveillance system is installed in any location at will. Without compensating for uneven illumination, it is impossible to get a satisfactory recognition rate. In this paper, we propose an integrated system that first compensates uneven illumination through local contrast enhancement. Then the enhanced images are fed into a robust face recognition system which adaptively selects the most important features among all candidate features and performs classification by support vector machines (SVMs). The dimension of feature space as well as the selected types of features is customized for each hyperplane. Three face image databases, namely Yale, Yale Group B, and Extended Yale Group B, are used to evaluate performance. The experimental result shows that the proposed recognition system give superior results compared to recently published literatures.     

Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions

In this paper, we present a new method to modify the appearance of a face image by manipulating the illumination condition, when the face geometry and albedo information is unknown. This problem is particularly difficult when there is only a single image of the subject available. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using a spherical harmonic representation. Moreover, morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework by proposing a 3D spherical harmonic basis morphable model (SHBMM). The proposed method can represent a face under arbitrary unknown lighting and pose simply by three low-dimensional vectors, i.e., shape parameters, spherical harmonic basis parameters, and illumination coefficients, which are called the SHBMM parameters. However, when the image was taken under an extreme lighting condition, the approximation error can be large, thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion-based framework that uses a Markov random field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to extreme lighting conditions, but also insensitive to partial occlusions. The performance of our framework is demonstrated through various experimental results, including the improved rates for face recognition under extreme lighting conditions.     

基于形态学商图像的光照归一化算法

复杂光照条件下的人脸识别是一个困难但需迫切解决的问题,为此提出了一种有效的光照归一化算法.该方法根据面部光照特点,基于数学形态学和商图像技术对各种光照条件下的人脸图像进行归一化处理,并且将它发展到动态地估计光照强度,进一步增强消除光照和保留特征的效果.与传统的技术相比,该方法无须训练数据集以及假定光源位置,并且每人只需一幅注册图像.在耶鲁人脸图像库B上的测试表明,该算法以较小的计算代价取得了优良的识别性能.     

可变光照条件下的人脸图像识别

对于人脸图像识别中光照变化的影响,传统的解决方法是对待识别图像进行光照补偿,先使它成为标准光照条件下的图像,然后和模板图像匹配来进行识别。为了提高在光照条件大范围变化时,人脸图像的识别率,提出了一种新的可变光照条件下的人脸图像识别方法。该方法首先利用在9个基本光照方向下分别获得的9幅图像来构成人脸光照特征空间,再通过这个光照特征空间,将图像库中的人脸图像变换成与待识别图像具有相同光照条件的图像,并将其作为模板图像;然后利用特征脸方法进行识别。实验结果表明,这种方法不仅能够有效地解决人脸识别中由于光照变化影响所造成的识别率下降的问题,而且对于光照条件大范围变化的情况,也可以得到比较高的正确识别率。     

From few to many: illumination cone models for face recognitionunder variable lighting and pose

We present a generative appearance-based method for recognizing human faces under variation in lighting and viewpoint. Our method exploits the fact that the set of images of an object in fixed pose, but under all possible illumination conditions, is a convex cone in the space of images. Using a small number of training images of each face taken with different lighting directions, the shape and albedo of the face can be reconstructed. In turn, this reconstruction serves as a generative model that can be used to render (or synthesize) images of the face under novel poses and illumination conditions. The pose space is then sampled and, for each pose, the corresponding illumination cone is approximated by a low-dimensional linear subspace whose basis vectors are estimated using the generative model. Our recognition algorithm assigns to a test image the identity of the closest approximated illumination cone. Test results show that the method performs almost without error, except on the most extreme lighting directions     

基于总变分模型的光照不变人脸识别算法

提出了一种基于L1总变分模型的对数商图像光照不变人脸识别算法。用L1总变分模型作为低通滤波算子对图像平滑滤波,得到图像光照分量的估计,然后在对数域中定义原图像与其光照分量的商为光照归一化图像,并用该图像作为光照不变量进行人脸识别。基于L1总变分模型的平滑滤波具有较好的边缘保持作用,能有效地消除光晕现象,并且参数设置简单。在YaleB和CMU PIE 人脸图像库上的试验结果表明,该算法能有效地提高人脸识别系统在不同光照条件下的识别率。     

Hybrid image matching combining Hausdorff distance with normalized gradient matching

Image matching has been a central problem in computer vision and image processing for decades. Most of the previous approaches to image matching can be categorized into the intensity-based and edge-based comparison. Hausdorff distance has been widely used for comparing point sets or edge maps since it does not require point correspondences. In this paper, we propose a new image similarity measure combining the Hausdorff distance with a normalized gradient consistency score for image matching. The normalized gradient consistency score is designed to compare the normalized image gradient fields between two images to alleviate the illumination variation problem in image matching. By combining the edge-based and intensity-based information for image matching, we are able to achieve robust image matching under different lighting conditions. We show the superior robustness property of the proposed image matching technique through experiments on face recognition under different lighting conditions.     

Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics

In this paper, we propose two novel methods for face recognition under arbitrary unknown lighting by using spherical harmonics illumination representation, which require only one training image per subject and no 3D shape information. Our methods are based on the result which demonstrated that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace. We provide two methods to estimate the spherical harmonic basis images spanning this space from just one image. Our first method builds the statistical model based on a collection of 2D basis images. We demonstrate that, by using the learned statistics, we can estimate the spherical harmonic basis images from just one image taken under arbitrary illumination conditions if there is no pose variation. Compared to the first method, the second method builds the statistical models directly in 3D spaces by combining the spherical harmonic illumination representation and a 3D morphable model of human faces to recover basis images from images across both poses and illuminations. After estimating the basis images, we use the same recognition scheme for both methods: we recognize the face for which there exists a weighted combination of basis images that is the closest to the test face image. We provide a series of experiments that achieve high recognition rates, under a wide range of illumination conditions, including multiple sources of illumination. Our methods achieve comparable levels of accuracy with methods that have much more onerous training data requirements. Comparison of the two methods is also provided.     

Face recognition under varying illumination based on a 2D face shape model

This paper proposes a novel illumination compensation algorithm, which can compensate for the uneven illuminations on human faces and reconstruct face images in normal lighting conditions. A simple yet effective local contrast enhancement method, namely block-based histogram equalization (BHE), is first proposed. The resulting image processed using BHE is then compared with the original face image processed using histogram equalization (HE) to estimate the category of its light source. In our scheme, we divide the light source for a human face into 65 categories. Based on the category identified, a corresponding lighting compensation model is used to reconstruct an image that will visually be under normal illumination. In order to eliminate the influence of uneven illumination while retaining the shape information about a human face, a 2D face shape model is used. Experimental results show that, with the use of principal component analysis for face recognition, the recognition rate can be improved by 53.3% to 62.6% when our proposed algorithm for lighting compensation is used.     

Extraction of illumination invariant facial features from a single image using nonsubsampled contourlet transform

Face recognition under varying lighting conditions is challenging, especially for single image based recognition system. Exacting illumination invariant features is an effective approach to solve this problem. However, existing methods are hard to extract both multi-scale and multi-directivity geometrical structures at the same time, which is important for capturing the intrinsic features of a face image. In this paper, we propose to utilize the logarithmic nonsubsampled contourlet transform (LNSCT) to estimate the reflectance component from a single face image and refer it as the illumination invariant feature for face recognition, where NSCT is a fully shift-invariant, multi-scale, and multi-direction transform. LNSCT can extract strong edges, weak edges, and noise from a face image using NSCT in the logarithm domain. We analyze that in the logarithm domain the low-pass subband of a face image and the low frequency part of strong edges can be regarded as the illumination effects, while the weak edges and the high frequency part of strong edges can be considered as the reflectance component. Moreover, even though a face image is polluted by noise (in particular the multiplicative noise), the reflectance component can still be well estimated and meanwhile the noise is removed. The LNSCT can be applied flexibly as neither assumption on lighting condition nor information about 3D shape is required. Experimental results show the promising performance of LNSCT for face recognition on Extended Yale B and CMU-PIE databases.     

Robust face recognition using generalized neural reflectance model

A generalized neural reflectance (GNR) model for enhancing face recognition under variations in illumination and posture is presented in this paper. Our work is based on training a number of synthesis images of each face taken at single lighting direction with frontal/posture view. This way of synthesizing images can be used to build training cases for each face under different known illumination conditions from which face recognition can be significantly improved. However, reconstructing face shape may not easily be achieved and the human face images usually form by highly complex structure which suffers from strong specular and unknown reflective conditions. In this paper, these limitations are addressed by Cho and Chow (IEEE Trans Neural Netw 12(5):1204–1214, 2002). Face surfaces are recovered by this GNR model and face images in different poses are synthesized to create a database for training. Our training algorithm assigns to recognize the face identity by similarity measure on face features extracting first by the principle component analysis (PCA) method and then further processing by the Fisher’s discrimination analysis (FDA) to acquire lower dimensional patterns. Experimental results conducted on the Yale Face Database B show that lower error rates of classification and recognition are achieved under different variations in lighting and pose and the performance significantly outperforms the recognition without using the proposed GNR model.