Profile-based 3D-aided face recognition

This paper presents a framework for automatic face recognition based on a silhouetted face profile (URxD-PV). Previous research has demonstrated the high discriminative potential of this biometric. Compared to traditional approaches in profile-based recognition, our approach is not limited to only standard side-view faces. We propose to explore the feature space of profiles under various rotations with the aid of a 3D face model. In the enrollment mode, 3D data of subjects are acquired and used to create profiles under different rotations. The features extracted from these profiles are used to train a classifier. In the identification mode, the profile is extracted from the side-view image and its metadata is matched with the gallery metadata. We validate the accuracy of URxD-PV using data from publicly available databases.     

From scores to face templates: a model-based approach

Regeneration of templates from match scores has security and privacy implications related to any biometric authentication system. We propose a novel paradigm to reconstruct face templates from match scores using a linear approach. It proceeds by first modeling the behavior of the given face recognition algorithm by an affine transformation. The goal of the modeling is to approximate the distances computed by a face recognition algorithm between two faces by distances between points, representing these faces, in an affine space. Given this space, templates from an independent image set (break-in) are matched only once with the enrolled template of the targeted subject and match scores are recorded. These scores are then used to embed the targeted subject in the approximating affine (non-orthogonal) space. Given the coordinates of the targeted subject in the affine space, the original template of the targeted subject is reconstructed using the inverse of the affine transformation. We demonstrate our ideas using three, fundamentally different, face recognition algorithms: Principal Component Analysis (PCA) with Mahalanobis cosine distance measure, Bayesian intra-extrapersonal classifier (BIC), and a feature-based commercial algorithm. To demonstrate the independence of the break-in set with the gallery set, we select face templates from two different databases: Face Recognition Grand Challenge (FRGC) and Facial Recognition Technology (FERET) Database (FERET). With an operational point set at 1 percent False Acceptance Rate (FAR) and 99 percent True Acceptance Rate (TAR) for 1,196 enrollments (FERET gallery), we show that at most 600 attempts (score computations) are required to achieve a 73 percent chance of breaking in as a randomly chosen target subject for the commercial face recognition system. With similar operational set up, we achieve a 72 percent and 100 percent chance of breaking in for the Bayesian and PCA based face recognition systems, respectively. With three different levels of score quantization, we achieve 69 percent, 68 percent and 49 percent probability of break-in, indicating the robustness of our proposed scheme to score quantization. We also show that the proposed reconstruction scheme has 47 percent more probability of breaking in as a randomly chosen target subject for the commercial system as compared to a hill climbing approach with the same number of attempts. Given that the proposed template reconstruction method uses distinct face templates to reconstruct faces, this work exposes a more severe form of vulnerability than a hill climbing kind of attack where incrementally different versions of the same face are used. Also, the ability of the proposed approach to reconstruct actual face templates of the users increases privacy concerns in biometric systems.     

Learning warps based similarity for pose-unconstrained face recognition

Face recognition techniques are widely used in many applications, such as automatic detection of crime scenes from surveillance cameras for public safety. In these real cases, the pose and illumination variances between two matching faces have a big influence on the identification performance. Handling pose changes is an especially challenging task. In this paper, we propose the learning warps based similarity method to deal with face recognition across the pose problem. Warps are learned between two patches from probe faces and gallery faces using the Lucas-Kanade algorithm. Based on these warps, a frontal face registered in the gallery is transformed into a series of non-frontal viewpoints, which enables non-frontal probe face matching with the frontal gallery face. Scale-invariant feature transform (SIFT) keypoints (interest points) are detected from the generated viewpoints and matched with the probe faces. Moreover, based on the learned warps, the probability likelihood is used to calculate the probability of two faces being the same subject. Finally, a hybrid similarity combining the number of matching keypoints and the probability likelihood is proposed to describe the similarity between a gallery face and a probe face. Experimental results show that our proposed method achieves better recognition accuracy than other algorithms it was compared to, especially when the pose difference is within 40 degrees.

     

Person authentication using brainwaves (EEG) and maximum a posteriori model adaptation

In this paper, we investigate the use of brain activity for person authentication. It has been shown in previous studies that the brainwave pattern of every individual is unique and that the electroencephalogram (EEG) can be used for biometric identification. EEG-based biometry is an emerging research topic and we believe that it may open new research directions and applications in the future. However, very little work has been done in this area and was focusing mainly on person identification but not on person authentication. Person authentication aims to accept or to reject a person claiming an identity, i.e., comparing a biometric data to one template, while the goal of person identification is to match the biometric data against all the records in a database. We propose the use of a statistical framework based on Gaussian mixture models and maximum a posteriori model adaptation, successfully applied to speaker and face authentication, which can deal with only one training session. We perform intensive experimental simulations using several strict train/test protocols to show the potential of our method. We also show that there are some mental tasks that are more appropriate for person authentication than others     

Part-based spatio-temporal model for multi-person re-identification

In this paper we propose an adaptive part-based spatio-temporal model that characterizes person’s appearance using color and facial features. Face image selection based on low level cues is used to select usable face images to build a face model. Color features that capture the distribution of colors as well as the representative colors are used to build the color model. The model is built over a sequence of frames of an individual and hence captures the characteristic appearance as well as its variations over time. We also address the problem of multiple person re-identification in the absence of calibration data or prior knowledge about the camera layout. Multiple person re-identification is a open set matching problem with a dynamically evolving and open gallery set and an open probe set. Re-identification is posed as a rectangular assignment problem and is solved to find a bijection that minimizes the overall assignment cost. Open and closed set re-identification is tested on 30 videos collected with nine non-overlapping cameras spanning outdoor and indoor areas, with 40 subjects under observation. A false acceptance reduction scheme based on the developed model is also proposed.     

Audio-visual human recognition using semi-supervised spectral learning and hidden Markov models

This paper presents a multimodal system for reliable human identity recognition under variant conditions. Our system fuses the recognition of face and speech with a general probabilistic framework. For face recognition, we propose a new spectral learning algorithm, which considers not only the discriminative relations among the training data but also the generative models for each class. Due to the tedious cost of face labeling in practice, our spectral face learning utilizes a semi-supervised strategy. That is, only a small number of labeled faces are used in our training step, and the labels are optimally propagated to other unlabeled training faces. Besides requiring much less labeled data, our algorithm also enables a natural way to explicitly train an outlier model that approximately represents unauthorized faces. To boost the robustness of our system for human recognition under various environments, our face recognition is further complemented by a speaker identification agent. Specifically, this agent models the statistical variations of fixed-phrase speech using speaker-dependent word hidden Markov models. Experiments on benchmark databases validate the effectiveness of our face recognition and speaker identification agents, and demonstrate that the recognition accuracy can be apparently improved by integrating these two independent biometric sources together.     

Automatic interpretation and coding of face images using flexiblemodels

Face images are difficult to interpret because they are highly variable. Sources of variability include individual appearance, 3D pose, facial expression, and lighting. We describe a compact parametrized model of facial appearance which takes into account all these sources of variability. The model represents both shape and gray-level appearance, and is created by performing a statistical analysis over a training set of face images. A robust multiresolution search algorithm is used to fit the model to faces in new images. This allows the main facial features to be located, and a set of shape, and gray-level appearance parameters to be recovered. A good approximation to a given face can be reconstructed using less than 100 of these parameters. This representation can be used for tasks such as image coding, person identification, 3D pose recovery, gender recognition, and expression recognition. Experimental results are presented for a database of 690 face images obtained under widely varying conditions of 3D pose, lighting, and facial expression. The system performs well on all the tasks listed above     

Linguistics and face recognition

We describe in this paper a novel biometric methodology for face recognition suitable to address pose, illumination, and expression (PIE) image variability, temporal change, flexible matching, and last but not least occlusion and disguise that are usually referred to as denial and deception. The adverse conditions listed above affect the scope and performance of biometric analysis vis-à-vis both training and testing. The conceptual framework proposed here draws support from discriminative methods using likelihood ratios. At the conceptual level it links forensics and biometrics, while at the implementation level it links the Bayesian framework and statistical learning theory. As many of the concerns listed usually affect only parts of the face, a non-parametric recognition-by-part approach is advanced here for the purpose of reliable face recognition. Recognition-by-parts facilitates authentication because it does not seek for explicit invariance. Instead, it handles variability using component-based configurations that are flexible enough to compensate among others for limited pose changes, if any, and limited occlusion and disguise. The recognition-by-parts approach proposed here supports incremental and progressive processing. It is similar in nature to modern linguistics and practical intelligence with the emphasis on semantics and pragmatics. Layered categorization starts with face detection using implicit rather than explicit segmentation. It proceeds with face authentication that involves feature selection of local patch instances including dimensionality reduction, exemplar-based clustering of patches into parts, and data fusion for matching using boosting driven by parts that play the role of weak learners. The implementation, driven by transduction, employs proximity and typicality (ranking) realized using strangeness and random deficiency p-values, respectively. The feasibility and reliability of the proposed architecture has been validated using FERET and FRGC data. The paper concludes with suggestions for augmenting and enhancing the scope and utility of the recognition-by-parts architecture.     

Open-set face recognition across look-alike faces in real-world scenarios

The open-set problem is among the problems that have significantly changed the performance of face recognition algorithms in real-world scenarios. Open-set operates under the supposition that not all the probes have a pair in the gallery. Most face recognition systems in real-world scenarios focus on handling pose, expression and illumination problems on face recognition. In addition to these challenges, when the number of subjects is increased for face recognition, these problems are intensified by look-alike faces for which there are two subjects with lower intra-class variations. In such challenges, the inter-class similarity is higher than the intra-class variation for these two subjects. In fact, these look-alike faces can be created as intrinsic, situation-based and also by facial plastic surgery. This work introduces three real-world open-set face recognition methods across facial plastic surgery changes and a look-alike face by 3D face reconstruction and sparse representation. Since some real-world databases for face recognition do not have multiple images per person in the gallery, with just one image per subject in the gallery, this paper proposes a novel idea to overcome this challenge by 3D modeling from gallery images and synthesizing them for generating several images. Accordingly, a 3D model is initially reconstructed from frontal face images in a real-world gallery. Then, each 3D reconstructed face in the gallery is synthesized to several possible views and a sparse dictionary is generated based on the synthesized face image for each person. Also, a likeness dictionary is defined and its optimization problem is solved by the proposed method. Finally, the face recognition is performed for open-set face recognition using three proposed representation classifications. Promising results are achieved for face recognition across plastic surgery and look-alike faces on three databases including the plastic surgery face, look-alike face and LFW databases compared to several state-of-the-art methods. Also, several real-world and open-set scenarios are performed to evaluate the proposed method on these databases in real-world scenarios.     

Finger surface as a biometric identifier

We present a novel approach for personal identification and identity verification which utilizes 3D finger surface features as a biometric identifier. Using 3D range images of the hand, a surface representation for the index, middle, and ring finger is calculated and used for comparison to determine subject similarity. We use the curvature based shape index to represent the fingers’ surface. Gallery and probe shape index signatures are compared using the normalized correlation coefficient to compute a match score. A large unique database of hand images supports the research. We use data sets obtained over time to examine the performance of each individual finger surface as a biometric identifier as well as the performance obtained when combining them. Both identification and verification experiments are conducted. In addition, probe and gallery sets sizes are increased to further improve recognition performance in our experiments. Our approach yields good results for a first-of-its-kind biometric technique, indicating that this approach warrants further research.     

Energy normalization for pose-invariant face recognition based on MRF model image matching

A pose-invariant face recognition system based on an image matching method formulated on MRFs is presented. The method uses the energy of the established match between a pair of images as a measure of goodness-of-match. The method can tolerate moderate global spatial transformations between the gallery and the test images and alleviate the need for geometric preprocessing of facial images by encapsulating a registration step as part of the system. It requires no training on non-frontal face images. A number of innovations, such as a dynamic block size and block shape adaptation, as well as label pruning and error pre-whitening measures have been introduced to increase the effectiveness of the approach. The experimental evaluation of the method is performed on two publicly available databases. First, the method is tested on the rotation shots of the XM2VTS data set in a verification scenario. Next, the evaluation is conducted in an identification scenario on the CMU-PIE database. The method compares favorably with the existing 2D or 3D generative model-based methods on both databases in both identification and verification scenarios.     

Facial expression analysis and expression-invariant face recognition by manifold-based synthesis

Over the last decades, expression classification and face recognition have received substantial attention in computer vision and pattern recognition with more recent efforts focusing on understanding and modelling expression variations. In this paper, we present an expression classification and expression-invariant face recognition method by synthesising photorealistic expression manifolds to expand the gallery set. By means of synthesising expression images from neutral faces, more within-subject variability can be obtained. Eigentransformation is utilised to generate both shape and expression details for novel subjects. Expression classification and face recognition are then performed on the extended training set with synthesised expressions. Experimental results on various datasets show that the proposed method is robust for recognising various expressions and faces with varying degrees of expression. Comprehensive experiments conducted and comparisons with the existing methods are reported. Cross-database synthesis and effect of landmark quality are also studied.     

Face Recognition from Caption-Based Supervision

In this paper, we present methods for face recognition using a collection of images with captions. We consider two tasks: retrieving all faces of a particular person in a data set, and establishing the correct association between the names in the captions and the faces in the images. This is challenging because of the very large appearance variation in the images, as well as the potential mismatch between images and their captions.     

基于因子分析与稀疏表示的多姿态人脸识别

在非可控环境下,人脸识别面临的最大难题之一是姿态变化与遮挡问题。基于稀疏表示的人脸识别方法将测试人脸表示成训练人脸的稀疏线性组合,根据其组合系数的稀疏性进行人脸识别。该方法对人脸的噪声和遮挡变化具有很好的鲁棒性,但对人脸的姿态变化表现力极差,这是因为当人脸具有姿态变化时,同一个人不同姿态情况下很难对应起来,这违背线性组合的前提条件。为了克服稀疏表示方法对人脸姿态变化表现力极差问题,对人脸进行因子分析,分离出人脸姿态因子,得到合成的正面人脸;利用稀疏表示进行人脸分类识别。实验结果表明,该方法对人脸的遮挡和姿态变化具有很好的鲁棒性。     

An algorithm for the detection of faces on the basis of Gabor features and information maximization

We propose an algorithm for the detection of facial regions within input images. The characteristics of this algorithm are (1) a vast number of Gabor-type features (196,800) in various orientations, and with various frequencies and central positions, which are used as feature candidates in representing the patterns of an image, and (2) an information maximization principle, which is used to select several hundred features that are suitable for the detection of faces from among these candidates. Using only the selected features in face detection leads to reduced computational cost and is also expected to reduce generalization error. We applied the system, after training, to 42 input images with complex backgrounds (Test Set A from the Carnegie Mellon University face data set). The result was a high detection rate of 87.0%, with only six false detections. We compared the result with other published face detection algorithms.     

Evolutionary search for faces from line drawings

Single 2D line drawing is a straightforward method to illustrate 3D objects. The faces of an object depicted by a line drawing give very useful information for the reconstruction of its 3D geometry. Two recently proposed methods for face identification from line drawings are based on two steps: finding a set of circuits that may be faces and searching for real faces from the set according to some criteria. The two steps, however, involve two combinatorial problems. The number of the circuits generated in the first step grows exponentially with the number of edges of a line drawing. These circuits are then used as the input to the second combinatorial search step. When dealing with objects having more faces, the combinatorial explosion prevents these methods from finding solutions within feasible time. This paper proposes a new method to tackle the face identification problem by a variable-length genetic algorithm with novel heuristic and geometric constraints incorporated for local search. The hybrid GA solves the two combinatorial problems simultaneously. Experimental results show that our algorithm can find the faces of a line drawing having more than 30 faces much more efficiently. In addition, simulated annealing for solving the face identification problem is also implemented for comparison.     

Dynamic tongueprint: A novel biometric identifier

Biometrics, which use human physiological or behavioral features for personal identification, currently face the challenge of designing a secure biometric system that will accept only the legitimate presentation of the biometric identifiers without being fooled by the doctored or spoofed measurements that are input into the system. More biometric traits are required for improving the performance of authentication systems. In this paper, we present a new number for the biometrics family, i.e. tongueprint, which uses particularly interesting properties of the human tongue to base a technology for noninvasive biometric assessment. The tongue is a unique organ which can be stuck out of the mouth for inspection, whose appearance is amenable to examination with the aid of a machine vision system. Yet it is otherwise well protected in the mouth and difficult to be forged. Furthermore, the involuntary squirm of the tongue is not only a convincing proof that the subject is alive, but also a feature for recognition. That is to say, the tongue can present both static features and dynamic features for authentication. However, little work has hitherto been done on the tongue as a biometric identifier. In this work, we make use of a database of tongue images obtained over a long period to examine the performance of the tongueprint as a biometric identifier. Our research shows that tongueprint is a promising candidate for biometric identification and worthy of further research.     

A novel framework for automatic 3D face recognition using quality assessment

The quality of biometric samples plays an important role in biometric authentication systems because it has a direct impact on verification or identification performance. In this paper, we present a novel 3D face recognition system which performs quality assessment on input images prior to recognition. More specifically, a reject option is provided to allow the system operator to eliminate the incoming images of poor quality, e.g. failure acquisition of 3D image, exaggerated facial expressions, etc.. Furthermore, an automated approach for preprocessing is presented to reduce the number of failure cases in that stage. The experimental results show that the 3D face recognition performance is significantly improved by taking the quality of 3D facial images into account. The proposed system achieves the verification rate of 97.09% at the False Acceptance Rate (FAR) of 0.1% on the FRGC v2.0 data set.     

复杂景物中人面的定位

人面是常见的复杂模式.在复杂景物图片中自动找出人面是一个困难但有重要意义的课 题.这是自动人面识别的第一个重要步骤.该文提出了一个在复杂背景中找出不同尺寸人面 的方法,能在较大的尺寸范围内在复杂背景的黑白图片中找到人面.