Fully automatic face recognition framework based on local and global features

Face recognition algorithms can be divided into two categories: holistic and local feature-based approaches. Holistic methods are very popular in recent years due to their good performance and high efficiency. However, they depend on careful positioning of the face images into the same canonical pose, which is not an easy task. On the contrary, some local feature-based approaches can achieve good recognition performances without additional alignment. But their computational burden is much heavier than holistic approaches. To solve these problems in holistic and local feature-based approaches, we propose a fully automatic face recognition framework based on both the local and global features. In this work, we propose to align the input face images using multi-scale local features for the holistic approach, which serves as a filter to narrow down the database for further fine matching. The computationally heavy local feature-based approach is then applied on the narrowed database. This fully automatic framework not only speeds up the local feature-based approach, but also improves the recognition accuracy comparing with the holistic and local approaches as shown in the experiments.     

Hybrid image matching combining Hausdorff distance with normalized gradient matching

Image matching has been a central problem in computer vision and image processing for decades. Most of the previous approaches to image matching can be categorized into the intensity-based and edge-based comparison. Hausdorff distance has been widely used for comparing point sets or edge maps since it does not require point correspondences. In this paper, we propose a new image similarity measure combining the Hausdorff distance with a normalized gradient consistency score for image matching. The normalized gradient consistency score is designed to compare the normalized image gradient fields between two images to alleviate the illumination variation problem in image matching. By combining the edge-based and intensity-based information for image matching, we are able to achieve robust image matching under different lighting conditions. We show the superior robustness property of the proposed image matching technique through experiments on face recognition under different lighting conditions.     

Face recognition using Intrinsicfaces

In this paper, we propose a novel face model, called intrinsic face model. Under this model, each face image is divided into three components, i.e., facial commonness difference, individuality difference and intrapersonal difference, to characterize some certain differences conveyed by this image. Then, a new supervised dimensionality reduction technique coined Intrinsic Discriminant Analysis (IDA) is developed. Intrinsic Discriminant Analysis tries to best classify different face images by maximizing the individuality difference, while minimizing the intrapersonal difference. By using perturbation technique to tackle the singularity problem of IDA which occurs frequently in face recognition, we obtain a new appearance-based face recognition method called Intrinsicfaces. A series of experiments to compare our proposed approach with other dimensionality reduction methods are tested on three well-known face databases. Experimental results demonstrate the efficacy of the proposed Intrinsicfaces approach in face recognition.     

Face recognition based on 3D ridge images obtained from range data

In this paper, we present an approach for 3D face recognition from frontal range data based on the ridge lines on the surface of the face. We use the principal curvature, k_max, to represent the face image as a 3D binary image called ridge image. The ridge image shows the locations of the ridge points around the important facial regions on the face (i.e., the eyes, the nose, and the mouth). We utilized the robust Hausdorff distance and the iterative closest points (ICP) for matching the ridge image of a given probe image to the ridge images of the facial images in the gallery. To evaluate the performance of our approach for 3D face recognition, we performed experiments on GavabDB face database (a small size database) and Face Recognition Grand Challenge V2.0 (a large size database). The results of the experiments show that the ridge lines have great capability for 3D face recognition. In addition, we found that as long as the size of the database is small, the performance of the ICP-based matching and the robust Hausdorff matching are comparable. But, when the size of the database increases, ICP-based matching outperforms the robust Hausdorff matching technique.     

基于分块相位一致性的人脸识别算法

针对基于可见光的人脸图像的识别容易受光照和表情变化的影响,人脸的表情变化仅限于局部等问题,以及图像的相位一致性特征不受图像的亮度或对比度影响的特点,提出了一种基于分块相位一致性的人脸识别算法。该算法用log-gabor滤波器对图像进行滤波,利用相位一致性模型提取相位一致性特征图像;对每幅特征图像进行分块主元分析(PCA)处理;融合所有子图像的距离信息,采用最近邻分类器进行分类识别。实验证明该方法具有更好的识别性能。     

2D face recognition based on supervised subspace learning from 3D models

One of the main challenges in face recognition is represented by pose and illumination variations that drastically affect the recognition performance, as confirmed by the results of recent face recognition large-scale evaluations. This paper presents a new technique for face recognition, based on the joint use of 3D models and 2D images, specifically conceived to be robust with respect to pose and illumination changes. A 3D model of each user is exploited in the training stage (i.e. enrollment) to generate a large number of 2D images representing virtual views of the face with varying pose and illumination. Such images are then used to learn in a supervised manner a set of subspaces constituting the user's template. Recognition occurs by matching 2D images with the templates and no 3D information (neither images nor face models) is required. The experiments carried out confirm the efficacy of the proposed technique.     

Face anti-spoofing with generated near-infrared images

Reflection differences between live faces and spoof faces under near-infrared spectrum make near-infrared image based methods obtain superior performance for face anti-spoofing. However, for conventional face recognition systems, near-infrared image based methods need additional near-infrared equipment to capture the input near-infrared images. In this paper, we propose a novel face anti-spoofing method which exploits the clues in both visible light (VIS) images and near-infrared (NIR) images without utilizing any near-infrared equipment during testing. Specifically, we first propose a novel multiple categories image translation generative adversarial network (MCT-GAN) which generates corresponding NIR images for VIS live and spoof face images. Then we utilize convolution neural network to learn fusing features from both VIS images and corresponding generated NIR images for the goal of live and spoof face classification. Qualitative and quantitative experiments demonstrate that our method obtains excellent results compared to the state-of-the-art methods.

     

Effects of classifier structures and training regimes on integrated segmentation and recognition of handwritten numeral strings

In integrated segmentation and recognition of character strings, the underlying classifier is trained to be resistant to noncharacters. We evaluate the performance of state-of-the-art pattern classifiers of this kind. First, we build a baseline numeral string recognition system with simple but effective presegmentation. The classification scores of the candidate patterns generated by presegmentation are combined to evaluate the segmentation paths and the optimal path is found using the beam search strategy. Three neural classifiers, two discriminative density models, and two support vector classifiers are evaluated. Each classifier has some variations depending on the training strategy: maximum likelihood, discriminative learning both with and without noncharacter samples. The string recognition performances are evaluated on the numeral string images of the NIST special database 19 and the zipcode images of the CEDAR CDROM-1. The results show that noncharacter training is crucial for neural classifiers and support vector classifiers, whereas, for the discriminative density models, the regularization of parameters is important. The string recognition results compare favorably to the best ones reported in the literature though we totally ignored the geometric context. The best results were obtained using a support vector classifier, but the neural classifiers and discriminative density models show better trade-off between accuracy and computational overhead.     

Face relighting using discriminative 2D spherical spaces for face recognition

As part of the face recognition task in a robust security system, we propose a novel approach for the illumination recovery of faces with cast shadows and specularities. Given a single 2D face image, we relight the face object by extracting the nine spherical harmonic bases and the face spherical illumination coefficients by using the face spherical spaces properties. First, an illumination training database is generated by computing the properties of the spherical spaces out of face albedo and normal values estimated from 2D training images. The training database is then discriminately divided into two directions in terms of the illumination quality and light direction of each image. Based on the generated multi-level illumination discriminative training space, we analyze the target face pixels and compare them with the appropriate training subspace using pre-generated tiles. When designing the framework, practical real-time processing speed and small image size were considered. In contrast to other approaches, our technique requires neither 3D face models nor restricted illumination conditions for the training process. Furthermore, the proposed approach uses one single face image to estimate the face albedo and face spherical spaces. In this work, we also provide the results of a series of experiments performed on publicly available databases to show the significant improvements in the face recognition rates.     

Face recognition using decimated redundant discrete wavelet transforms

As discrete wavelet transform (DWT) is sensitive to the translation/shift of input signals, its effectiveness could be lessened for face recognition, particularly when the face images are translated. To alleviate drawbacks resulted from this translation effect, we propose a decimated redundant DWT (DRDWT)-based face recognition method, where the decimation-based DWTs are performed on the original signal and its 1-stepshift, respectively. Even though the DRDWT realizes the decimation, it enables us to explore the translation invariant DWT representation for the periodic shifts of the probe image that is the most similar to the gallery images. Therefore, it can solve the problem of translation sensitivity of the original DWT and address the translation effect occurring between the probe image and the gallery image. To further improve the recognition performance, we combine the global wavelet features obtained from the entire face and the local wavelet features obtained from face patches to represent both holistic and detail facial features, apply separate classifiers to global and local features and combine the resulted global and local classifiers to form an ensemble classifier. Experimental results reported for the FERET and FRGCv2.0 databases show the effectiveness of the DRDWT method and quantify its performance.     

N-division output coding method applied to face recognition

Most research on face recognition has focused on representation of face appearances rather than the classifiers. For robust classification performance, we need to adopt elaborate classifiers. Output coding is suitable for this purpose because it can allow online learning. In this paper, we propose an N-division output coding method. In the experiments we demonstrate such properties as problem complexity, margin of separation, machine relevance and the recognition performance among different output coding methods.     

Illumination invariant feature extraction and mutual-information-based local matching for face recognition under illumination variation and occlusion

An efficient method for face recognition which is robust under illumination variations is proposed. The proposed method achieves the illumination invariants based on the illumination-reflection model employing local matching for best classification. Different filters have been tested to achieve the reflectance part of the image, which is illumination invariant, and maximum filter is suggested as the best method for this purpose. A set of adaptively weighted classifiers vote on different sub-images of each input image and a decision is made based on their votes. Image entropy and mutual information are used as weight factors. The proposed method does not need any prior information about the face shape or illumination and can be applied on each image separately. Unlike most available methods, our method does not need multiple images in training stage to get the illumination invariants. Support vector machines and k-nearest neighbors methods are used as classifier. Several experiments are performed on Yale B, Extended Yale B and CMU-PIE databases. Recognition results show that the proposed method is suitable for efficient face recognition under illumination variations.     

Conformal geometry and its applications on 3D shape matching, recognition, and stitching

Three-dimensional shape matching is a fundamental issue in computer vision with many applications such as shape registration, 3D object recognition, and classification. However, shape matching with noise, occlusion, and clutter is a challenging problem. In this paper, we analyze a family of quasi-conformal maps including harmonic maps, conformal maps, and least-squares conformal maps with regards to 3D shape matching. As a result, we propose a novel and computationally efficient shape matching framework by using least-squares conformal maps. According to conformal geometry theory, each 3D surface with disk topology can be mapped to a 2D domain through a global optimization and the resulting map is a diffeomorphism, i.e., one-to-one and onto. This allows us to simplify the 3D shape-matching problem to a 2D image-matching problem, by comparing the resulting 2D parametric maps, which are stable, insensitive to resolution changes and robust to occlusion, and noise. Therefore, highly accurate and efficient 3D shape matching algorithms can be achieved by using the above three parametric maps. Finally, the robustness of least-squares conformal maps is evaluated and analyzed comprehensively in 3D shape matching with occlusion, noise, and resolution variation. In order to further demonstrate the performance of our proposed method, we also conduct a series of experiments on two computer vision applications, i.e., 3D face recognition and 3D nonrigid surface alignment and stitching.     

Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition

We present a fully automated algorithm for facial feature extraction and 3D face modeling from a pair of orthogonal frontal and profile view images of a person's face taken by calibrated cameras. The algorithm starts by automatically extracting corresponding 2D landmark facial features from both view images, then compute their 3D coordinates. Further, we estimate the coordinates of the features that are hidden in the profile view based on the visible features extracted in the two orthogonal face images. The 3D coordinates of the selected feature points obtained from the images are used first to align, then to locally deform the corresponding facial vertices of the generic 3D model. Preliminary experiments to assess the applicability of the resulted models for face recognition show encouraging results.     

Class-specific multiple classifiers scheme to recognize emotions from speech signals

Automatic emotion recognition from speech signals is one of the important research areas, which adds value to machine intelligence. Pitch, duration, energy and Mel-frequency cepstral coefficients (MFCC) are the widely used features in the field of speech emotion recognition. A single classifier or a combination of classifiers is used to recognize emotions from the input features. The present work investigates the performance of the features of Autoregressive (AR) parameters, which include gain and reflection coefficients, in addition to the traditional linear prediction coefficients (LPC), to recognize emotions from speech signals. The classification performance of the features of AR parameters is studied using discriminant, k-nearest neighbor (KNN), Gaussian mixture model (GMM), back propagation artificial neural network (ANN) and support vector machine (SVM) classifiers and we find that the features of reflection coefficients recognize emotions better than the LPC. To improve the emotion recognition accuracy, we propose a class-specific multiple classifiers scheme, which is designed by multiple parallel classifiers, each of which is optimized to a class. Each classifier for an emotional class is built by a feature identified from a pool of features and a classifier identified from a pool of classifiers that optimize the recognition of the particular emotion. The outputs of the classifiers are combined by a decision level fusion technique. The experimental results show that the proposed scheme improves the emotion recognition accuracy. Further improvement in recognition accuracy is obtained when the scheme is built by including MFCC features in the pool of features.     

Structural diversity for decision tree ensemble learning

Decision trees are a kind of off-the-shelf predictive models, and they have been successfully used as the base learners in ensemble learning. To construct a strong classifier ensemble, the individual classifiers should be accurate and diverse. However, diversity measure remains a mystery although there were many attempts. We conjecture that a deficiency of previous diversity measures lies in the fact that they consider only behavioral diversity, i.e., how the classifiers behave when making predictions, neglecting the fact that classifiers may be potentially different even when they make the same predictions. Based on this recognition, in this paper, we advocate to consider structural diversity in addition to behavioral diversity, and propose the TMD (tree matching diversity) measure for decision trees. To investigate the usefulness of TMD, we empirically evaluate performances of selective ensemble approaches with decision forests by incorporating different diversity measures. Our results validate that by considering structural and behavioral diversities together, stronger ensembles can be constructed. This may raise a new direction to design better diversity measures and ensemble methods.     

COMPARING FACE IMAGES USING THE MODIFIED HAUSDORFF DISTANCE

We introduce a novel methodology applicable to face matching and fast screening of large facial databases. The proposed shape comparison method operates on edge maps and derives holistic similarity measures, yet, it does not require solving the point correspondence problem. While the use of edge images is important to introduce robustness to changes in illumination, the lack of point-to-point matching delivers speed and tolerance to local non-rigid distortions. In particular, we propose a face similarity measure derived as a variant of the Hausdorff distance by introducing the notion of a neighborhood function (N) and associated penalties (P). Experimental results on a large set of face images demonstrate that our approach produces excellent recognition results even when less than 3% of the original grey-scale face image information is stored in the face database (gallery). These results implicate that the process of face recognition may start at a much earlier stage of visual processing than it was earlier suggested. We argue, that edge-like retinal images of faces are initially screened “at a glance” without the involvement of high-level cognitive functions thus delivering high speed and reducing computational complexity.     

Information theoretic combination of pattern classifiers

Combining several classifiers has proved to be an effective machine learning technique. Two concepts clearly influence the performances of an ensemble of classifiers: the diversity between classifiers and the individual accuracies of the classifiers. In this paper we propose an information theoretic framework to establish a link between these quantities. As they appear to be contradictory, we propose an information theoretic score (ITS) that expresses a trade-off between individual accuracy and diversity. This technique can be directly used, for example, for selecting an optimal ensemble in a pool of classifiers. We perform experiments in the context of overproduction and selection of classifiers, showing that the selection based on the ITS outperforms state-of-the-art diversity-based selection techniques.     

Face verification via error correcting output codes

We propose a novel approach to face verification based on the Error Correcting Output Coding (ECOC) classifier design concept. In the training phase, the client set is repeatedly divided into two ECOC specified sub-sets (super-classes) to train a set of binary classifiers. The output of the classifiers defines the ECOC feature space, in which it is easier to separate transformed patterns representing clients and impostors. As a matching score in this space, we propose the average first order Minkowski distance between the probe and gallery images. The proposed method exhibits superior verification performance on the well known XM2VTS data set as compared with previously reported results.     

Human Face Recognition and the Face Image Set′s Topology

If we consider an n × n image as an n²-dimensional vector, then images of faces can be considered as points in this n²-dimensional image space. Our previous studies of physical transformations of the face, including translation, small rotations, and illumination changes, showed that the set of face images consists of relatively simple connected subregions in image space. Consequently linear matching techniques can be used to obtain reliable face recognition. However, for more general transformations, such as large rotations or scale changes, the face subregions become highly non-convex. We have therefore developed a scale-space matching technique that allows us to take advantage of knowledge about important geometrical transformations and about the topology of the face subregion in image space. While recognition of faces is the focus of this paper, the algorithm is sufficiently general to be applicable to a large variety of object recognition tasks