Multiple nose region matching for 3D face recognition under varying facial expression

An algorithm is proposed for 3D face recognition in the presence of varied facial expressions. It is based on combining the match scores from matching multiple overlapping regions around the nose. Experimental results are presented using the largest database employed to date in 3D face recognition studies, over 4,000 scans of 449 subjects. Results show substantial improvement over matching the shape of a single larger frontal face region. This is the first approach to use multiple overlapping regions around the nose to handle the problem of expression variation.     

Deformation modeling for robust 3D face matching

Face recognition based on 3D surface matching is promising for overcoming some of the limitations of current 2D image-based face recognition systems. The 3D shape is generally invariant to the pose and lighting changes, but not invariant to the non-rigid facial movement, such as expressions. Collecting and storing multiple templates to account for various expressions for each subject in a large database is not practical. We propose a facial surface modeling and matching scheme to match 2.5D facial scans in the presence of both non-rigid deformations and pose changes (multiview) to a 3D face template. A hierarchical geodesic-based resampling approach is applied to extract landmarks for modeling facial surface deformations. We are able to synthesize the deformation learned from a small group of subjects (control group) onto a 3D neutral model (not in the control group), resulting in a deformed template. A user-specific (3D) deformable model is built by combining the templates with synthesized deformations. The matching distance is computed by fitting this generative deformable model to a test scan. A fully automatic and prototypic 3D face matching system has been developed. Experimental results demonstrate that the proposed deformation modeling scheme increases the 3D face matching accuracy.     

The correspondence framework for 3D surface matching algorithms

Beyond the inherent technical challenges, current research into the three dimensional surface correspondence problem is hampered by a lack of uniform terminology, an abundance of application specific algorithms, and the absence of a consistent model for comparing existing approaches and developing new ones. This paper addresses these challenges by presenting a framework for analysing, comparing, developing, and implementing surface correspondence algorithms. The framework uses five distinct stages to establish correspondence between surfaces. It is general, encompassing a wide variety of existing techniques, and flexible, facilitating the synthesis of new correspondence algorithms. This paper presents a review of existing surface correspondence algorithms, and shows how they fit into the correspondence framework. It also shows how the framework can be used to analyse and compare existing algorithms and develop new algorithms using the framework’s modular structure. Six algorithms, four existing and two new, are implemented using the framework. Each implemented algorithm is used to match a number of surface pairs. Results demonstrate that the correspondence framework implementations are faithful implementations of existing algorithms, and that powerful new surface correspondence algorithms can be created.     

A multimodal approach for 3D face modeling and recognition using 3D deformable facial mask

We present a multimodal approach for face modeling and recognition. The algorithm uses three cameras to capture stereo images, two frontal and one profile, of the face. 2D facial features are extracted from one of the frontal images and a dense disparity map is computed from the two frontal images. Using the extracted 2D features and their corresponding disparities, we compute their 3D coordinates. We next align a low resolution 3D mesh model to the 3D features, re-project its vertices onto the frontal 2D image and adjust its profile silhouette vertices using the profile view image. We increase the resolution of the resulting 2D model at its center region to obtain a facial mask model covering distinctive features of the face. The 2D coordinates of the vertices, along with their disparities, result in a deformed 3D mask model specific to a given subject’s face. Our method integrates information from the extracted facial features from the 2D image modality with information from the 3D modality obtained from the stereo images. Application of the models in 3D face recognition, for 112 subjects, validates the algorithm with a 95% identification rate and 92% verification rate at 0.1% false acceptance rate.

三维人脸建模及在跨姿态人脸匹配中的有效性验证

针对现有三维人脸采集技术对采集场景存在诸多限制,提出了自由场景下基于多张图像的三维人脸建模技术,并对其进行了有效性验证。首先,提出一个姿态及深度值迭代计算模型,实现了特征点深度值的准确估计;然后,进行了基于多张图像的深度值融合及整体形状建模;最后,将深度迭代优化算法（IPDO）与目前最优的非线性最小二乘法（NLS1_SR）在Bosphorus Database数据集上进行了对比,建模精度提高了9%,所重建的三维人脸模型投影图像与二维图像具有较高的相似度。实验结果表明,在大姿态变化条件下,该识别算法借助三维信息相较于未借助的情况下,其识别率可以提高50%以上。     

A novel framework for automatic 3D face recognition using quality assessment

The quality of biometric samples plays an important role in biometric authentication systems because it has a direct impact on verification or identification performance. In this paper, we present a novel 3D face recognition system which performs quality assessment on input images prior to recognition. More specifically, a reject option is provided to allow the system operator to eliminate the incoming images of poor quality, e.g. failure acquisition of 3D image, exaggerated facial expressions, etc.. Furthermore, an automated approach for preprocessing is presented to reduce the number of failure cases in that stage. The experimental results show that the 3D face recognition performance is significantly improved by taking the quality of 3D facial images into account. The proposed system achieves the verification rate of 97.09% at the False Acceptance Rate (FAR) of 0.1% on the FRGC v2.0 data set.     

三维人脸建模中关键点的自动定位

针对3D人脸建模形变方法中关键点的获取问题,提出一种关键点自动定位的方法。首先根据需求,确定关键点位置并进行分类;接着利用HSI空间S分量实现头部检测;再采用Harris角点检测技术,结合人脸结构特征和颜色信息,实现特征部位的提取;然后分别在正、侧面照片各特征部位,对不同类别的关键点采用不同的方法,提取相应的2D坐标;将两个角度上提取的2D坐标组合,最终得到关键点的3D坐标。实验结果表明,所提出的关键点定位方法在速度和准确性方面具有良好性能。     

2D representation of facial surfaces for multi-pose 3D face recognition

The increasing availability of 3D facial data offers the potential to overcome the intrinsic difficulties faced by conventional face recognition using 2D images. Instead of extending 2D recognition algorithms for 3D purpose, this letter proposes a novel strategy for 3D face recognition from the perspective of representing each 3D facial surface with a 2D attribute image and taking the advantage of the advances in 2D face recognition. In our approach, each 3D facial surface is mapped homeomorphically onto a 2D lattice, where the value at each site is an attribute that represents the local 3D geometrical or textural properties on the surface, therefore invariant to pose changes. This lattice is then interpolated to generate a 2D attribute image. 3D face recognition can be achieved by applying the traditional 2D face recognition techniques to obtained attribute images. In this study, we chose the pose invariant local mean curvature calculated at each vertex on the 3D facial surface to construct the 2D attribute image and adopted the eigenface algorithm for attribute image recognition. We compared our approach to state-of-the-art 3D face recognition algorithms in the FRGC (Version 2.0), GavabDB and NPU3D database. Our results show that the proposed approach has improved the robustness to head pose variation and can produce more accurate 3D multi-pose face recognition.     

Viewpoint independent matching of planar curves in 3D space

We present a new approach to match planar curves using the weak perspective projection model. This is based on a set of shape parameters that can be extracted from a closed or open contour, derived from the original image as a θ(s) boundary code. In order to reduce the complexity and increase the robustness of the matching process, the original parameters are reduced to a set of three intermediate variables, each of which can be calculated independently. These variables are contained within a system of linear equations which define the angles and the ratio of the heights of corresponding point pairs on the two contours with respect to a floating coordinate system. The shape matching process is scale and orientation independent, and the original parameters that describe the relative pose of the two contours in 3D space can be recovered subsequently. The approach can be applied to “featured” and “featureless” contours, to whole and partial contours, and is demonstrated on images of contours and mechanical parts and tools to recover identity and pose.     

对表情鲁棒的面部轮廓线3维人脸识别

目的表情变化是3维人脸识别面临的主要问题。为克服表情影响,提出了一种基于面部轮廓线对表情鲁棒的3维人脸识别方法。方法首先,对人脸进行预处理,包括人脸区域切割、平滑处理和姿态归一化,将所有的人脸置于姿态坐标系下;然后,从3维人脸模型的半刚性区域提取人脸多条垂直方向的轮廓线来表征人脸面部曲面;最后,利用弹性曲线匹配算法计算不同3维人脸模型间对应的轮廓线在预形状空间(preshape space)中的测地距离,将其作为相似性度量,并且对所有轮廓线的相似度向量加权融合,得到总相似度用于分类。结果在FRGC v2.0数据库上进行识别实验,获得97.1%的Rank-1识别率。结论基于面部轮廓线的3维人脸识别方法,通过从人脸的半刚性区域提取多条面部轮廓线来表征人脸,在一定程度上削弱了表情的影响,同时还提高了人脸匹配速度。实验结果表明,该方法具有较强的识别性能,并且对表情变化具有较好的鲁棒性。     

Real time 3D face and facial feature tracking

Detecting and tracking human faces in video sequences is useful in a number of applications such as gesture recognition and human-machine interaction. In this paper, we show that online appearance models (holistic approaches) can be used for simultaneously tracking the head, the lips, the eyebrows, and the eyelids in monocular video sequences. Unlike previous approaches to eyelid tracking, we show that the online appearance models can be used for this purpose. Neither color information nor intensity edges are used by our proposed approach. More precisely, we show how the classical appearance-based trackers can be upgraded in order to deal with fast eyelid movements. The proposed eyelid tracking is made robust by avoiding eye feature extraction. Experiments on real videos show the usefulness of the proposed tracking schemes as well as their enhancement to our previous approach.

Three-dimensional face recognition using shapes of facial curves

We study shapes of facial surfaces for the purpose of face recognition. The main idea is to 1) represent surfaces by unions of level curves, called facial curves, of the depth function and 2) compare shapes of surfaces implicitly using shapes of facial curves. The latter is performed using a differential geometric approach that computes geodesic lengths between closed curves on a shape manifold. These ideas are demonstrated using a nearest-neighbor classifier on two 3D face databases: Florida State University and Notre Dame, highlighting a good recognition performance     

关键点匹配三维人脸识别方法*

提出了一种新颖的三维人脸识别算法,其基本思路是,把代表人脸的三维点云沿X、Y或Z轴旋转,反复多次把3D人脸关键点投影到2.5D图像上,然后提取2.5D图像的关键点并进行标记,而用这些比原来小得多的关键点代替原来的面扫描。面对未知的待测人脸首先通过执行相同的多视角特征点提取技术提取关键点,然后应用一个新的加权特征点匹配算法进行识别。通过用GavabDB三维面部识别数据集进行试验评估,这个方法对中性表情人脸可获得高达94％的识别精度,对人脸表情辨识（如微笑）的准确率也超过了88％。实验结果表明,此方法在识别精     

基于点匹配的3D人脸网格自动转换方法

为了实现3D人脸网格的自动转换,把两个拓扑结构不同的3D人脸网格看作两个顶点坐标集合,提出了一种基于点匹配的人脸网格自动转换方法.利用迭代最近点(ICP)算法对两个点集进行对齐,结合薄板样条函数与确定性退火技术对两个点集进行匹配,实现人脸网格的自动转换.针对转换后网格在面部细节方面的不足,利用克里金插值,提高转换后网格与源网格的相似性.最后在VC++6.0与OpenGL环境下实现了该系统,对算法进行了仿真分析,结果表明了系统的可行性和算法的有效性.     

Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition

We present a fully automated algorithm for facial feature extraction and 3D face modeling from a pair of orthogonal frontal and profile view images of a person's face taken by calibrated cameras. The algorithm starts by automatically extracting corresponding 2D landmark facial features from both view images, then compute their 3D coordinates. Further, we estimate the coordinates of the features that are hidden in the profile view based on the visible features extracted in the two orthogonal face images. The 3D coordinates of the selected feature points obtained from the images are used first to align, then to locally deform the corresponding facial vertices of the generic 3D model. Preliminary experiments to assess the applicability of the resulted models for face recognition show encouraging results.     

Efficient 3D face recognition handling facial expression and hair occlusion

This paper presents an efficient 3D face recognition method to handle facial expression and hair occlusion. The proposed method uses facial curves to form a rejection classifier and produce a facial deformation mapping and then adaptively selects regions for matching. When a new 3D face with an arbitrary pose and expression is queried, the pose is normalized based on the automatically detected nose tip and the principal component analysis (PCA) follows. Then, the facial curve in the nose region is extracted and used to form the rejection classifier which quickly eliminates dissimilar faces in the gallery for efficient recognition. Next, six facial regions which cover the face are segmented and curves in these regions are used to map facial deformation. Regions used for matching are automatically selected based on the deformation mapping. In the end, results of all the matching engines are fused by weighted sum rule. The approach is applied on the FRGC v2.0 dataset and a verification rate of 96.0% for ROC III is achieved as a false acceptance rate (FAR) of 0.1%. In the identification scenario, a rank-one accuracy of 97.8% is achieved.     

基于特征点表情变化的3维人脸识别

目的 为克服表情变化对3维人脸识别的影响,提出一种基于特征点提取局部区域特征的3维人脸识别方法。方法 首先,在深度图上应用2维图像的ASM(active shape model)算法粗略定位出人脸特征点,再根据Shape index特征在人脸点云上精确定位出特征点。其次,提取以鼻中为中心的一系列等测地轮廓线来表征人脸形状;然后,提取具有姿态不变性的Procrustean向量特征(距离和角度)作为识别特征;最后,对各条等测地轮廓线特征的分类结果进行了比较,并对分类结果进行决策级融合。结果 在FRGC V2.0人脸数据库分别进行特征点定位实验和识别实验,平均定位误差小于2.36 mm,Rank-1识别率为98.35%。结论 基于特征点的3维人脸识别方法,通过特征点在人脸近似刚性区域提取特征,有效避免了受表情影响较大的嘴部区域。实验证明该方法具有较高的识别精度,同时对姿态、表情变化具有一定的鲁棒性。     

Reshaping 3D facial scans for facial appearance modeling and 3D facial expression analysis

3D face scans have been widely used for face modeling and analysis. Due to the fact that face scans provide variable point clouds across frames, they may not capture complete facial data or miss point-to-point correspondences across various facial scans, thus causing difficulties to use such data for analysis. This paper presents an efficient approach to representing facial shapes from face scans through the reconstruction of face models based on regional information and a generic model. A new approach for 3D feature detection and a hybrid approach using two vertex mapping algorithms, displacement mapping and point-to-surface mapping, and a regional blending algorithm are proposed to reconstruct the facial surface detail. The resulting models can represent individual facial shapes consistently and adaptively, establishing facial point correspondences across individual models. The accuracy of the generated models is evaluated quantitatively. The applicability of the models is validated through the application of 3D facial expression recognition using the static 3DFE and dynamic 4DFE databases. A comparison with the state of the art has also been reported.