A 3D face matching framework for facial curves

Among the many 3D face matching techniques that have been developed, are variants of 3D facial curve matching, which reduce the amount of face data to one or a few 3D curves. The face’s central profile, for instance, proved to work well. However, the selection of the optimal set of 3D curves and the best way to match them has not been researched systematically. We propose a 3D face matching framework that allows profile and contour based face matching. Using this framework we evaluate profile and contour types including those described in the literature, and select subsets of facial curves for effective and efficient face matching. With a set of eight geodesic contours we achieve a mean average precision (MAP) of 0.70 and 92.5% recognition rate (RR) on the 3D face retrieval track of the Shape Retrieval Contest (SHREC’08), and a MAP of 0.96 and 97.6% RR on the University of Notre Dame (UND) test set. Face matching with these curves is time-efficient and performs better than other sets of facial curves and depth map comparison.     

An efficient 3D face recognition approach using local geometrical signatures

This paper presents a computationally efficient 3D face recognition system based on a novel facial signature called Angular Radial Signature (ARS) which is extracted from the semi-rigid region of the face. Kernel Principal Component Analysis (KPCA) is then used to extract the mid-level features from the extracted ARSs to improve the discriminative power. The mid-level features are then concatenated into a single feature vector and fed into a Support Vector Machine (SVM) to perform face recognition. The proposed approach addresses the expression variation problem by using facial scans with various expressions of different individuals for training. We conducted a number of experiments on the Face Recognition Grand Challenge (FRGC v2.0) and the 3D track of Shape Retrieval Contest (SHREC 2008) datasets, and a superior recognition performance has been achieved. Our experimental results show that the proposed system achieves very high Verification Rates (VRs) of 97.8% and 88.5% at a 0.1% False Acceptance Rate (FAR) for the “neutral vs. nonneutral” experiments on the FRGC v2.0 and the SHREC 2008 datasets respectively, and 96.7% for the ROC III experiment of the FRGC v2.0 dataset. Our experiments also demonstrate the computational efficiency of the proposed approach.     

An Expression Deformation Approach to Non-rigid 3D Face Recognition

The accuracy of non-rigid 3D face recognition approaches is highly influenced by their capacity to differentiate between the deformations caused by facial expressions from the distinctive geometric attributes that uniquely characterize a 3D face, interpersonal disparities. We present an automatic 3D face recognition approach which can accurately differentiate between expression deformations and interpersonal disparities and hence recognize faces under any facial expression. The patterns of expression deformations are first learnt from training data in PCA eigenvectors. These patterns are then used to morph out the expression deformations. Similarity measures are extracted by matching the morphed 3D faces. PCA is performed in such a way it models only the facial expressions leaving out the interpersonal disparities. The approach was applied on the FRGC v2.0 dataset and superior recognition performance was achieved. The verification rates at 0.001 FAR were 98.35% and 97.73% for scans under neutral and non-neutral expressions, respectively.     

2.5D Elastic graph matching

In this paper, we propose novel elastic graph matching (EGM) algorithms for face recognition assisted by the availability of 3D facial geometry. More specifically, we conceptually extend the EGM algorithm in order to exploit the 3D nature of human facial geometry for face recognition/verification. In order to achieve that, first we extend the matching module of the EGM algorithm in order to capitalize on the 2.5D facial data. Furthermore, we incorporate the 3D geometry into the multiscale analysis used and build a novel geodesic multiscale morphological pyramid of dilations/erosions in order to fill the graph jets. We show that the proposed advances significantly enhance the performance of EGM algorithms. We demonstrate the efficiency of the proposed advances in the face recognition/verification problem using photometric stereo.     

Illuminant-Dependence of Von Kries Type Quotients

An expression-invariant 3D face recognition approach is presented. Our basic assumption is that facial expressions can be modelled as isometries of the facial surface. This allows to construct expression-invariant representations of faces using the bending-invariant canonical forms approach. The result is an efficient and accurate face recognition algorithm, robust to facial expressions, that can distinguish between identical twins (the first two authors). We demonstrate a prototype system based on the proposed algorithm and compare its performance to classical face recognition methods.The numerical methods employed by our approach do not require the facial surface explicitly. The surface gradients field, or the surface metric, are sufficient for constructing the expression-invariant representation of any given face. It allows us to perform the 3D face recognition task while avoiding the surface reconstruction stage.     

Three-dimensional facial surface modeling applied to recognition

Applications related to game technology, law-enforcement, security, medicine or biometrics are becoming increasingly important, which, combined with the proliferation of three-dimensional (3D) scanning hardware, have made that 3D face recognition is now becoming a promising and feasible alternative to two-dimensional (2D) face methods. The main advantage of 3D data, when compared with traditional 2D approaches, is that it provides information that is invariant to rigid geometric transformations and to pose and illumination conditions. One key element for any 3D face recognition system is the modeling of the available scanned data. This paper presents new 3D models for facial surface representation and evaluates them using two matching approaches: one based on support vector machines and another one on principal component analysis (with a Euclidean classifier). Also, two types of environments were tested in order to check the robustness of the proposed models: a controlled environment with respect to facial conditions (i.e. expressions, face rotations, etc.) and a non-controlled one (presenting face rotations and pronounced facial expressions). The recognition rates obtained using reduced spatial resolution representations (a 77.86% for non-controlled environments and a 90.16% for controlled environments, respectively) show that the proposed models can be effectively used for practical face recognition applications.     

Constructing 3D facial hierarchical structure based on surface measurements

In this paper, we propose a novel framework for 3D facial similarity measures and facial data organization. The 3D facial similarity measures of our method are based on iso-geodesic stripes and conformal parameterization. Using the conformal parameterization, the 3D facial surface can be mapped into a 2D domain and the iso-geodesic stripes of the face can be measured. The measure results can be regarded as the similarity of faces, which is robust to head poses and facial expressions. Based on the measure result, a hierarchical structure of faces can be constructed, which is used to organize different faces. The structure can be utilized to accelerate the face searching speed in a large database. In experiment, we construct the hierarchical structures from two public facial databases: Gavab and Texas3D. The searching speed based on the structure can be increased by 4-6 times without accuracy loss of recognition.

     

Deformation modeling for robust 3D face matching

Face recognition based on 3D surface matching is promising for overcoming some of the limitations of current 2D image-based face recognition systems. The 3D shape is generally invariant to the pose and lighting changes, but not invariant to the non-rigid facial movement, such as expressions. Collecting and storing multiple templates to account for various expressions for each subject in a large database is not practical. We propose a facial surface modeling and matching scheme to match 2.5D facial scans in the presence of both non-rigid deformations and pose changes (multiview) to a 3D face template. A hierarchical geodesic-based resampling approach is applied to extract landmarks for modeling facial surface deformations. We are able to synthesize the deformation learned from a small group of subjects (control group) onto a 3D neutral model (not in the control group), resulting in a deformed template. A user-specific (3D) deformable model is built by combining the templates with synthesized deformations. The matching distance is computed by fitting this generative deformable model to a test scan. A fully automatic and prototypic 3D face matching system has been developed. Experimental results demonstrate that the proposed deformation modeling scheme increases the 3D face matching accuracy.     

Generating discriminating cartoon faces using interacting snakes

As a computational bridge between the high-level a priori knowledge of object shape and the low-level image data, active contours (or snakes) are useful models for the extraction of deformable objects. We propose an approach for manipulating multiple snakes iteratively, called interacting snakes, that minimizes the attraction energy functionals on both contours and enclosed regions of individual snakes and the repulsion energy functionals among multiple snakes that interact with each other. We implement the interacting snakes through explicit curve (parametric active contours) representation in the domain of face recognition. We represent human faces semantically via facial components such as eyes, mouth, face outline, and the hair outline. Each facial component is encoded by a closed (or open) snake that is drawn from a 3D generic face model. A collection of semantic facial components form a hypergraph, called semantic face graph, which employs interacting snakes to align the general facial topology onto the sensed face images. Experimental results show that a successful interaction among multiple snakes associated with facial components makes the semantic face graph a useful model for face representation, including cartoon faces and caricatures, and recognition.     

结合加权局部旋度模式的3维人脸表情识别

目的 3维人脸的表情信息不均匀地分布在五官及脸颊附近,对表情进行充分的描述和合理的权重分配是提升识别效果的重要途径。为提高3维人脸表情识别的准确率,提出了一种基于带权重局部旋度模式的3维人脸表情识别算法。方法 首先,为了提取具有较强表情分辨能力的特征,提出对3维人脸的旋度向量进行编码,获取局部旋度模式作为表情特征;然后,提出将ICNP（interactive closest normal points）算法与最小投影偏差算法结合,前者实现3维人脸子区域的不规则划分,划分得到的11个子区域保留了表情变化下面部五官和肌肉的完整性,后者根据各区域对表情识别的贡献大小为各区域的局部旋度模式特征分配权重;最后,带有权重的局部旋度模式特征被输入到分类器中实现表情识别。结果 基于BU-3DFE 3维人脸表情库对本文提出的局部旋度模式特征进行评估,结果表明其分辨能力较其他表情特征更强;基于BU-3DFE库进行表情识别实验,与其他3维人脸表情识别算法相比,本文算法取得了最高的平均识别率,达到89.67%,同时对易混淆的“悲伤”、“愤怒”和“厌恶”等表情的误判率也较低。结论 局部旋度模式特征对3维人脸的表情有较强的表征能力; ICNP算法与最小投影偏差算法的结合,能够实现区域的有效划分和权重的准确计算,有效提高特征对表情的识别能力。试验结果表明本文算法对3维人脸表情具有较高的识别率,并对易混淆的相似表情仍具有较好的识别效果。     

对表情鲁棒的面部轮廓线3维人脸识别

目的表情变化是3维人脸识别面临的主要问题。为克服表情影响,提出了一种基于面部轮廓线对表情鲁棒的3维人脸识别方法。方法首先,对人脸进行预处理,包括人脸区域切割、平滑处理和姿态归一化,将所有的人脸置于姿态坐标系下;然后,从3维人脸模型的半刚性区域提取人脸多条垂直方向的轮廓线来表征人脸面部曲面;最后,利用弹性曲线匹配算法计算不同3维人脸模型间对应的轮廓线在预形状空间(preshape space)中的测地距离,将其作为相似性度量,并且对所有轮廓线的相似度向量加权融合,得到总相似度用于分类。结果在FRGC v2.0数据库上进行识别实验,获得97.1%的Rank-1识别率。结论基于面部轮廓线的3维人脸识别方法,通过从人脸的半刚性区域提取多条面部轮廓线来表征人脸,在一定程度上削弱了表情的影响,同时还提高了人脸匹配速度。实验结果表明,该方法具有较强的识别性能,并且对表情变化具有较好的鲁棒性。     

Multiple nose region matching for 3D face recognition under varying facial expression

An algorithm is proposed for 3D face recognition in the presence of varied facial expressions. It is based on combining the match scores from matching multiple overlapping regions around the nose. Experimental results are presented using the largest database employed to date in 3D face recognition studies, over 4,000 scans of 449 subjects. Results show substantial improvement over matching the shape of a single larger frontal face region. This is the first approach to use multiple overlapping regions around the nose to handle the problem of expression variation.     

Recognizing faces using Adaptively Weighted Sub-Gabor Array from a single sample image per enrolled subject

In this paper, we propose a new approach for face representation and recognition based on Adaptively Weighted Sub-Gabor Array (AWSGA) when only one sample image per enrolled subject is available. Instead of using holistic representation of face images which is not effective under different facial expressions and partial occlusions, the proposed algorithm utilizes a local Gabor array to represent faces partitioned into sub-patterns. Especially, in order to perform matching in the sense of the richness of identity information rather than the size of a local area and to handle the partial occlusion problem, the proposed method employs an adaptively weighting scheme to weight the Sub-Gabor features extracted from local areas based on the importance of the information they contain and their similarities to the corresponding local areas in the general face image. An extensive experimental investigation is conducted using AR and Yale face databases covering face recognition under controlled/ideal condition, different illumination condition, different facial expression and partial occlusion. The system performance is compared with the performance of four benchmark approaches. The promising experimental results indicate that the proposed method can greatly improve the recognition rates under different conditions.     

3D face recognition with sparse spherical representations

This paper addresses the problem of 3D face recognition using simultaneous sparse approximations on the sphere. The 3D face point clouds are first aligned with a fully automated registration process. They are then represented as signals on the 2-sphere in order to preserve depth and geometry information. Next, we implement a dimensionality reduction process with simultaneous sparse approximations and subspace projection. It permits to represent each 3D face by only a few spherical functions that are able to capture the salient facial characteristics, and hence to preserve the discriminant facial information. We eventually perform recognition by effective matching in the reduced space, where linear discriminant analysis can be further activated for improved recognition performance. The 3D face recognition algorithm is evaluated on the FRGC v.1.0 data set, where it is shown to outperform classical state-of-the-art solutions that work with depth images.     

Line-based face recognition under varying pose

Much research in human face recognition involves fronto-parallel face images, constrained rotations in and out of the plane, and operates under strict imaging conditions such as controlled illumination and limited facial expressions. Face recognition using multiple views in the viewing sphere is a more difficult task since face rotations out of the imaging plane can introduce occlusion of facial structures. In this paper, we propose a novel image-based face recognition algorithm that uses a set of random rectilinear line segments of 2D face image views as the underlying image representation, together with a nearest-neighbor classifier as the line matching scheme. The combination of 1D line segments exploits the inherent coherence in one or more 2D face image views in the viewing sphere. The algorithm achieves high generalization recognition rates for rotations both in and out of the plane, is robust to scaling, and is computationally efficient. Results show that the classification accuracy of the algorithm is superior compared with benchmark algorithms and is able to recognize test views in quasi-real-time     

PCA-based image recombination for multimodal 2D + 3D face recognition

Most of the existing approaches of multimodal 2D + 3D face recognition exploit the 2D and 3D information at the feature or score level. They do not fully benefit from the dependency between modalities. Exploiting this dependency at the early stage is more effective than the later stage. Early fusion data contains richer information about the input biometric than the compressed features or matching scores. We propose an image recombination for face recognition that explores the dependency between modalities at the image level. Facial cues from the 2D and 3D images are recombined into a more independent and discriminating data by finding transformation axes that account for the maximal amount of variances in the images. We also introduce a complete framework of multimodal 2D + 3D face recognition that utilizes the 2D and 3D facial information at the enrollment, image and score levels. Experimental results based on NTU-CSP and Bosphorus 3D face databases show that our face recognition system using image recombination outperforms other face recognition systems based on the pixel- or score-level fusion.     

A novel framework for automatic 3D face recognition using quality assessment

The quality of biometric samples plays an important role in biometric authentication systems because it has a direct impact on verification or identification performance. In this paper, we present a novel 3D face recognition system which performs quality assessment on input images prior to recognition. More specifically, a reject option is provided to allow the system operator to eliminate the incoming images of poor quality, e.g. failure acquisition of 3D image, exaggerated facial expressions, etc.. Furthermore, an automated approach for preprocessing is presented to reduce the number of failure cases in that stage. The experimental results show that the 3D face recognition performance is significantly improved by taking the quality of 3D facial images into account. The proposed system achieves the verification rate of 97.09% at the False Acceptance Rate (FAR) of 0.1% on the FRGC v2.0 data set.     

Face recognition via fast dense correspondence

Face recognition plays a significant role in computer vision. It is well know that facial images are complex stimuli signals that suffer from non-rigid deformations, including misalignment, orientation, pose changes, and variations of facial expression, etc. In order to address these variations, this paper introduces an improved sparse-representation based face recognition method, which constructs dense pixel correspondences between training and testing facial samples. Specifically, we first construct a deformable spatial pyramid graph model that simultaneously regularizes matching consistency at multiple spatial extents - ranging from an entire image, though coarse grid cells, to every single pixel. Secondly, a matching energy function is designed to perform face alignment based on dense pixel correspondence, which is very effective to address the issue of non-rigid deformations. Finally, a novel coarse-to-fine matching scheme is designed so that we are able to speed up the optimization of the matching energy function. After the training samples are aligned with respect to testing samples, an improved sparse representation model is employed to perform face recognition. The experimental results demonstrate the superiority of the proposed method over other methods on ORL, AR, and LFWCrop datasets. Especially, the proposed approach improves nearly 4.4 % in terms of recognition accuracy and runs nearly 10 times faster than previous sparse approximation methods.