Benchmarking lightweight face architectures on specific face recognition scenarios

Artificial Intelligence Review - This paper studies the impact of lightweight face models on real applications. Lightweight architectures proposed for face recognition are analyzed and evaluated on...     

Bidirectional representation for face recognition across pose

Conventional representation methods try to express the test sample as a weighting sum of training samples and exploit the deviation between the test sample and the weighting sum of the training samples from each class (also referred to as deviation between the test sample and each class) to classify the test sample. In particular, the methods assign the test sample to the class that has the smallest deviation among all the classes. This paper analyzes the relationship between face images under different poses and, for the first time, devises a bidirectional representation method-based pattern classification (BRBPC) method for face recognition across pose. BRBPC includes the following three steps: the first step uses the procedure of conventional representation methods to express the test sample and calculates the deviation between the test sample and each class. The second step first expresses the training sample of a class as a weighting sum of the test sample and the training samples from all the other classes and then obtains the corresponding deviation (referred to as complementary deviation). The third step uses the score-level fusion to integrate the scores, that is, deviations generated from the first and second steps for final classification. The experimental results show that BRBPC classifies more accurately than conventional representation methods.     

Orthogonal discriminant vector for face recognition across pose

Recognizing face images across pose is one of the challenging tasks for reliable face recognition. This paper presents a new method to tackle this challenge based on orthogonal discriminant vector (ODV). The result of our theoretical analysis shows that an individual’s probe image captured with a new pose can be represented by a linear combination of his/her gallery images. Based on this observation, in contrast to the conventional methods which model face images of different individuals on a single manifold, we propose to model face images of different individuals on different linear manifolds. The contribution of our approach includes: (1) to prove that the orthogonality to ODVs is a pose-invariant feature.; (2) to categorize each person with a set of ODVs, where his/her face images posses zero projections while other persons’ images are characterized by maximum projections; (3) to define a metric to measure the distance between a face image and an ODV, and classify the face images based on this metric. Our experimental results validate the feasibility of modeling the face images of different individuals on different linear manifolds. The proposed method achieves higher accuracy on face recognition and verification than the existing techniques.     

Illumination robust single sample face recognition using multi-directional orthogonal gradient phase faces

Due to the limitation of the storage space in the real-world face recognition application systems, only one sample image per person is often stored in the system, which is the so-called single sample problem. Moreover, real-world illumination has impact on recognition performance. This paper presents an illumination robust single sample face recognition approach, which utilizes multi-directional orthogonal gradient phase faces to solve the above limitations. In the proposed approach, an illumination insensitive orthogonal gradient phase face is obtained by using two vertical directional gradient values of the original image. Multi-directional orthogonal gradient phase faces can be used to extend samples for single sample face recognition. Simulated experiments and comparisons on a subset of Yale B database, Yale database, a subset of PIE database and VALID face database show that the proposed approach is not only an outstanding method for single sample face recognition under illumination but also more effective when addressing illumination, expression, decoration, etc.     

Subject-specific and pose-oriented facial features for face recognition across poses

Most face recognition scenarios assume that frontal faces or mug shots are available for enrollment to the database, faces of other poses are collected in the probe set. Given a face from the probe set, one needs to determine whether a match in the database exists. This is under the assumption that in forensic applications, most suspects have their mug shots available in the database, and face recognition aims at recognizing the suspects when their faces of various poses are captured by a surveillance camera. This paper considers a different scenario: given a face with multiple poses available, which may or may not include a mug shot, develop a method to recognize the face with poses different from those captured. That is, given two disjoint sets of poses of a face, one for enrollment and the other for recognition, this paper reports a method best for handling such cases. The proposed method includes feature extraction and classification. For feature extraction, we first cluster the poses of each subject's face in the enrollment set into a few pose classes and then decompose the appearance of the face in each pose class using Embedded Hidden Markov Model, which allows us to define a set of subject-specific and pose-priented (SSPO) facial components for each subject. For classification, an Adaboost weighting scheme is used to fuse the component classifiers with SSPO component features. The proposed method is proven to outperform other approaches, including a component-based classifier with local facial features cropped manually, in an extensive performance evaluation study.     

Comparison of human and computer performance across face recognition experiments

Since 2005, human and computer performance has been systematically compared as part of face recognition competitions, with results being reported for both still and video imagery. The key results from these competitions are reviewed. To analyze performance across studies, the cross-modal performance analysis (CMPA) framework is introduced. The CMPA framework is applied to experiments that were part of face a recognition competition. The analysis shows that for matching frontal faces in still images, algorithms are consistently superior to humans. For video and difficult still face pairs, humans are superior. Finally, based on the CMPA framework and a face performance index, we outline a challenge problem for developing algorithms that are superior to humans for the general face recognition problem.     

Tied factor analysis for face recognition across large pose differences

Face recognition algorithms perform very unreliably when the pose of the probe face is different from the gallery face: typical feature vectors vary more with pose than with identity. We propose a generative model that creates a one-to-many mapping from an idealized "identity" space to the observed data space. In identity space, the representation for each individual does not vary with pose. We model the measured feature vector as being generated by a pose-contingent linear transformation of the identity variable in the presence of Gaussian noise. We term this model "tied" factor analysis. The choice of linear transformation (factors) depends on the pose, but the loadings are constant (tied) for a given individual. We use the EM algorithm to estimate the linear transformations and the noise parameters from training data. We propose a probabilistic distance metric which allows a full posterior over possible matches to be established. We introduce a novel feature extraction process and investigate recognition performance using the FERET, XM2VTS and PIE databases. Recognition performance compares favourably to contemporary approaches.     

Degradation adaptive texture classification for real-world application scenarios

Images captured under non-laboratory conditions potentially suffer from various degradations. Especially noise, blur and scale-variations are often prevalent in real world images and are known to potentially affect the classification process of textured images. We show that these degradations not necessarily strongly affect the discriminative powers of computer based classifiers in a scenario with similar degradations in training and evaluation set. We propose a degradation-adaptive classification approach, which exploits this knowledge by dividing one large data set into several smaller ones, each containing images with some kind of degradation-similarity. In a large experimental study, it can be shown that our method continuously enhances the classification accuracies in case of simulated as well as real world image degradations. Surprisingly, by means of a pre-classification, the framework turns out to be beneficial even in case of idealistic images which are free from strong degradations.     

View-based recognition of real-world textures

A new method for recognizing 3D textured surfaces is proposed. Textures are modeled with multiple histograms of micro-textons, instead of more macroscopic textons used in earlier studies. The micro-textons are extracted with the recently proposed multiresolution local binary pattern operator. Our approach has many advantages compared to the earlier approaches and provides the leading performance in the classification of Columbia-Utrecht database textures imaged under different viewpoints and illumination directions. It also provides very promising results in the classification of outdoor scene images. An approach for learning appearance models for view-based texture recognition using self-organization of feature distributions is also proposed. The method performs well in experiments. It can be used for quickly selecting model histograms and rejecting outliers, thus providing an efficient tool for vision system training even when the feature data has a large variability.     

Animating expressive faces across languages

This paper describes a morphing-based audio driven facial animation system. Based on an incoming audio stream, a face image is animated with full lip synchronization and synthesized expressions. A novel scheme to implement a language independent system for audio-driven facial animation given a speech recognition system for just one language, in our case, English, is presented. The method presented here can also be used for text to audio-visual speech synthesis. Visemes in new expressions are synthesized to be able to generate animations with different facial expressions. An animation sequence using optical flow between visemes is constructed, given an incoming audio stream and still pictures of a face representing different visemes. The presented techniques give improved lip synchronization and naturalness to the animated video.     

The analysis and recognition of real-world textures in threedimensions

The observed image texture for a rough surface has a complex dependence on the illumination and viewing angles due to effects such as foreshortening, local shading, interreflections, and the shadowing and occlusion of surface elements. We introduce the dimensionality surface as a representation for the visual complexity of a material sample. The dimensionality surface defines the number of basis textures that are required to represent the observed textures for a sample as a function of ranges of illumination and viewing angles. Basis textures are represented using multiband correlation functions that consider both within and between color band correlations. We examine properties of the dimensionality surface for real materials using the Columbia Utrecht Reflectance and Texture (CUReT) database. The analysis shows that the dependence of the dimensionality surface on ranges of illumination and viewing angles is approximately linear with a slope that depends on the complexity of the sample. We extend the analysis to consider the problem of recognizing rough surfaces in color images obtained under unknown illumination and viewing geometry. We show, using a set of 12,505 images from 61 material samples, that the information captured by the multiband correlation model allows surfaces to be recognized over a wide range of conditions. We also show that the use of color information provides significant advantages for three-dimensional texture recognition     

动态人脸识别系统的构建

提出了一种实际应用环境下的动态人脸识别系统。首先讨论了动态人脸识别系统硬件环境的搭建,然后详细介绍了动态人脸识别系统的软件流程、主要的功能模块、人脸库的构建及管理等,最后对系统进行了测试。测试结果表明,所设计的软硬件系统能够满足实际应用需求。     

Face recognition using common faces method

In this paper, we propose a face recognition method called the commonface by using the common vector approach. A face image is regarded as a summation of a common vector which represents the invariant properties of the corresponding face class, and a difference vector which presents the specific properties of the corresponding face image such as face appearance, pose and expression. Thus, by deriving the common vector of each face class, the common feature of each person is obtained which removes the differences of face images belonging to the same person. For test face image, the remaining vector with each face class is derived with the similar procedure to the common vector, which is then compared with the common vector of each face class to predict the class label of query face by finding the minimum distance between the remaining vector and the common vector. Furthermore, we extend the common vector approach (CVP) to kernel CVP to improve the performance of CVP. The experimental results suggest that the proposed commonface approach provides a better representation of individual common feature and achieves lower error rates in face recognition.     

Multibiometrics for face recognition

Fusion is a popular practice to combine multiple sources of biometric information to achieve systems with greater performance and flexibility. In this paper various approaches to fusion within a multibiometrics context are considered and an application to the fusion of 2D and 3D face information is discussed. An optimal method for fusing the accept/reject decisions of individual biometric sources by means of simple logical rules is presented. Experimental results on the FRGC 2D and 3D face data show that the proposed technique performs effectively without the need for score normalization.     

Cancelable fusion-based face recognition

Multimedia Tools and Applications - Biometric recognition refers to the automated process of recognizing individuals using their biometric patterns. Recent advancements in deep learning and...     

Linguistics and face recognition

We describe in this paper a novel biometric methodology for face recognition suitable to address pose, illumination, and expression (PIE) image variability, temporal change, flexible matching, and last but not least occlusion and disguise that are usually referred to as denial and deception. The adverse conditions listed above affect the scope and performance of biometric analysis vis-à-vis both training and testing. The conceptual framework proposed here draws support from discriminative methods using likelihood ratios. At the conceptual level it links forensics and biometrics, while at the implementation level it links the Bayesian framework and statistical learning theory. As many of the concerns listed usually affect only parts of the face, a non-parametric recognition-by-part approach is advanced here for the purpose of reliable face recognition. Recognition-by-parts facilitates authentication because it does not seek for explicit invariance. Instead, it handles variability using component-based configurations that are flexible enough to compensate among others for limited pose changes, if any, and limited occlusion and disguise. The recognition-by-parts approach proposed here supports incremental and progressive processing. It is similar in nature to modern linguistics and practical intelligence with the emphasis on semantics and pragmatics. Layered categorization starts with face detection using implicit rather than explicit segmentation. It proceeds with face authentication that involves feature selection of local patch instances including dimensionality reduction, exemplar-based clustering of patches into parts, and data fusion for matching using boosting driven by parts that play the role of weak learners. The implementation, driven by transduction, employs proximity and typicality (ranking) realized using strangeness and random deficiency p-values, respectively. The feasibility and reliability of the proposed architecture has been validated using FERET and FRGC data. The paper concludes with suggestions for augmenting and enhancing the scope and utility of the recognition-by-parts architecture.     

Robust semi-automatic head pose labeling for real-world face video sequences

Automatic head pose estimation from real-world video sequences is of great interest to the computer vision community since pose provides prior knowledge for tasks, such as face detection and classification. However, developing pose estimation algorithms requires large, labeled real-world video databases on which computer vision systems can be trained and tested. Manual labeling of each frame is tedious, time consuming, and often difficult due to the high uncertainty in head pose angle estimate, particularly in unconstrained environments that include arbitrary facial expression, occlusion, illumination etc. To overcome these difficulties, a semi-automatic framework is proposed for labeling temporal head pose in real-world video sequences. The proposed multi-stage labeling framework first detects a subset of frames with distinct head poses over a video sequence, which is then manually labeled by the expert to obtain the ground truth for those frames. The proposed framework provides a continuous head pose label and corresponding confidence value over the pose angles. Next, the interpolation scheme over a video sequence estimates i) labels for the frames without manual labels and ii) corresponding confidence values for interpolated labels. This confidence value permits an automatic head pose estimation framework to determine the subset of frames to be used for further processing, depending on the labeling accuracy required. The experiments performed on an in-house, labeled, large, real-world face video database (which will be made publicly available) show that the proposed framework achieves 96.98 % labeling accuracy when manual labeling is only performed on 30 % of the video frames.     

Bimode model for face recognition and face representation

Tensorface based approaches decompose an image into its constituent factors (i.e., person, lighting, viewpoint, etc.), and then utilize these factor spaces for recognition. However, tensorface is not a preferable choice, because of the complexity of its multimode. In addition, a single mode space, except the person-space, could not be used for recognition directly. From the viewpoint of practical application, we propose a bimode model for face recognition and face representation. This new model can be treated as a simplified model representation of tensorface. However, their respective algorithms for training are completely different, due to their different definitions of subspaces. Thanks to its simpler model form, the proposed model requires less iteration times in the process of training and testing. Moreover bimode model can be further applied to an image reconstruction and image synthesis via an example image. Comprehensive experiments on three face image databases (PEAL, YaleB frontal and Weizmann) validate the effectiveness of the proposed new model.