对表情鲁棒的面部轮廓线3维人脸识别

目的表情变化是3维人脸识别面临的主要问题。为克服表情影响,提出了一种基于面部轮廓线对表情鲁棒的3维人脸识别方法。方法首先,对人脸进行预处理,包括人脸区域切割、平滑处理和姿态归一化,将所有的人脸置于姿态坐标系下;然后,从3维人脸模型的半刚性区域提取人脸多条垂直方向的轮廓线来表征人脸面部曲面;最后,利用弹性曲线匹配算法计算不同3维人脸模型间对应的轮廓线在预形状空间(preshape space)中的测地距离,将其作为相似性度量,并且对所有轮廓线的相似度向量加权融合,得到总相似度用于分类。结果在FRGC v2.0数据库上进行识别实验,获得97.1%的Rank-1识别率。结论基于面部轮廓线的3维人脸识别方法,通过从人脸的半刚性区域提取多条面部轮廓线来表征人脸,在一定程度上削弱了表情的影响,同时还提高了人脸匹配速度。实验结果表明,该方法具有较强的识别性能,并且对表情变化具有较好的鲁棒性。     

基于改进的轮廓线算法的三维人脸识别方法

针对二维人脸识别中受表情、姿态以及光照等因素而影响识别率的问题,在分析人脸生理结构的基础上,提出了一种基于改进的轮廓线的三维人脸识别方法,即先提取三维人脸特征点,然后提取人脸轮廓线,最后利用人脸轮廓线和特征点构成的特征模型进行三维人脸识别。试验结果证明该方法提高了人脸识别率,并具有强抗干扰能力。     

An Expression Deformation Approach to Non-rigid 3D Face Recognition

The accuracy of non-rigid 3D face recognition approaches is highly influenced by their capacity to differentiate between the deformations caused by facial expressions from the distinctive geometric attributes that uniquely characterize a 3D face, interpersonal disparities. We present an automatic 3D face recognition approach which can accurately differentiate between expression deformations and interpersonal disparities and hence recognize faces under any facial expression. The patterns of expression deformations are first learnt from training data in PCA eigenvectors. These patterns are then used to morph out the expression deformations. Similarity measures are extracted by matching the morphed 3D faces. PCA is performed in such a way it models only the facial expressions leaving out the interpersonal disparities. The approach was applied on the FRGC v2.0 dataset and superior recognition performance was achieved. The verification rates at 0.001 FAR were 98.35% and 97.73% for scans under neutral and non-neutral expressions, respectively.     

Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition

We present a fully automated algorithm for facial feature extraction and 3D face modeling from a pair of orthogonal frontal and profile view images of a person's face taken by calibrated cameras. The algorithm starts by automatically extracting corresponding 2D landmark facial features from both view images, then compute their 3D coordinates. Further, we estimate the coordinates of the features that are hidden in the profile view based on the visible features extracted in the two orthogonal face images. The 3D coordinates of the selected feature points obtained from the images are used first to align, then to locally deform the corresponding facial vertices of the generic 3D model. Preliminary experiments to assess the applicability of the resulted models for face recognition show encouraging results.     

基于轮廓线的三维人脸识别的改进算法

对已有的基于轮廓线的人脸识别方法进行了改进,在人脸的任意位置利用PCA自动确定人脸纵方向,采用网格配准方法提取对称面和对称轮廓线。通过计算对称轮廓线上的曲率,提取其他3条横向轮廓线。对提取的4条轮廓线进行重采样和归一化,截取轮廓线的有价值部分作为ICP算法的输入,进行人脸识别。试验证明,该算法将人脸识别率从原来的86.5%提高到了94%,降低了误识率。     

Three-dimensional facial surface modeling applied to recognition

Applications related to game technology, law-enforcement, security, medicine or biometrics are becoming increasingly important, which, combined with the proliferation of three-dimensional (3D) scanning hardware, have made that 3D face recognition is now becoming a promising and feasible alternative to two-dimensional (2D) face methods. The main advantage of 3D data, when compared with traditional 2D approaches, is that it provides information that is invariant to rigid geometric transformations and to pose and illumination conditions. One key element for any 3D face recognition system is the modeling of the available scanned data. This paper presents new 3D models for facial surface representation and evaluates them using two matching approaches: one based on support vector machines and another one on principal component analysis (with a Euclidean classifier). Also, two types of environments were tested in order to check the robustness of the proposed models: a controlled environment with respect to facial conditions (i.e. expressions, face rotations, etc.) and a non-controlled one (presenting face rotations and pronounced facial expressions). The recognition rates obtained using reduced spatial resolution representations (a 77.86% for non-controlled environments and a 90.16% for controlled environments, respectively) show that the proposed models can be effectively used for practical face recognition applications.     

3D face authentication and recognition based on bilateral symmetry analysis

We present a novel and computationally fast method for automatic human face authentication. Taking a 3D triangular facial mesh as input, the approach first automatically extracts the bilateral symmetry plane of the facial surface. The intersection between the symmetry plane and the facial surface, namely the symmetry profile, is then computed. Using both the mean curvature plot of the facial surface and the curvature plot of the symmetry profile curve, three essential points of the nose on the symmetry profile are automatically extracted. The three essential points uniquely determine a Face Intrinsic Coordinate System (FICS). Different faces are aligned based on the FICS. The symmetry profile, together with two transverse profiles, composes a compact representation, called the SFC representation, of a 3D face surface. The face authentication and recognition steps are finally performed by comparing the SFC representations of the faces. The proposed method was tested on 382 face surfaces, which come from 166 individuals and cover a wide ethnic and age variety. The equal error rate (EER) of face authentication on scans with variable facial expressions is 10.8%. For scans with normal expression, the ERR is 0.8%.     

Recognizing Rotated Faces From Frontal and Side Views: An Approach Toward Effective Use of Mugshot Databases

Mug shot photography has been used to identify criminals by the police for more than a century. However, the common scenario of face recognition using frontal and side-view mug shots as gallery remains largely uninvestigated in computerized face recognition across pose. This paper presents a novel appearance-based approach using frontal and sideface images to handle pose variations in face recognition, which has great potential in forensic and security applications involving police mugshot databases. Virtual views in different poses are generated in two steps: 1) shape modelling and 2) texture synthesis. In the shape modelling step, a multilevel variation minimization approach is applied to generate personalized 3-D face shapes. In the texture synthesis step, face surface properties are analyzed and virtual views in arbitrary viewing conditions are rendered, taking diffuse and specular reflections into account. Appearance-based face recognition is performed with the augmentation of synthesized virtual views covering possible viewing angles to recognize probe views in arbitrary conditions. The encouraging experimental results demonstrated that the proposed approach by using frontal and side-view images is a feasible and effective solution to recognizing rotated faces, which can lead to a better and practical use of existing forensic databases in computerized human face-recognition applications.      

三维人脸数据的获取及人脸特征自动定位

介绍了一种快速获取人脸三维面貌数据的结构光相移测量技术,并利用这种高速的相移技术在获取三维面貌数据的同时获得人脸纹理背景图像,结合二维人脸图像中的人脸特征识别手段,应用到三维人脸图像中,可以让计算机自动提取人脸图像的主要特征点.首先介绍了高速相移技术的基本原理,介绍了二维人脸图像中的积分投影方法来求取人脸轮廓粗略位置的方法,接着介绍了将二维图像做纹理映射到三维数据里面的方法,结合三维高度信息的曲线分析、曲率判断等,快速的提取出了人脸的三维特征.经实验验证,此方法对于三维人脸特征的自动定位有很高的准确性和通用性.     

A multimodal approach for 3D face modeling and recognition using 3D deformable facial mask

We present a multimodal approach for face modeling and recognition. The algorithm uses three cameras to capture stereo images, two frontal and one profile, of the face. 2D facial features are extracted from one of the frontal images and a dense disparity map is computed from the two frontal images. Using the extracted 2D features and their corresponding disparities, we compute their 3D coordinates. We next align a low resolution 3D mesh model to the 3D features, re-project its vertices onto the frontal 2D image and adjust its profile silhouette vertices using the profile view image. We increase the resolution of the resulting 2D model at its center region to obtain a facial mask model covering distinctive features of the face. The 2D coordinates of the vertices, along with their disparities, result in a deformed 3D mask model specific to a given subject’s face. Our method integrates information from the extracted facial features from the 2D image modality with information from the 3D modality obtained from the stereo images. Application of the models in 3D face recognition, for 112 subjects, validates the algorithm with a 95% identification rate and 92% verification rate at 0.1% false acceptance rate.

刚性区域特征点的3维人脸识别

目的 针对3维人脸识别中存在表情变化的问题,提出了一种基于刚性区域特征点的3维人脸识别方法。方法 该方法首先在人脸纹理图像上提取人脸图像的特征点,并删除非刚性区域内的特征点,然后根据采样点的序号,在人脸空间几何信息上得到人脸图像特征点的3维几何信息,并建立以特征点为中心的刚性区域内的子区域,最后以子区域为局部特征进行人脸识别测试,得到不同子区域对人脸识别的贡献,并以此作为依据对人脸识别的结果进行加权统计。结果 在FRGC v2.0的3维人脸数据库上进行实验测试,该方法的识别准确率为98.5%,当错误接受率（FAR）为0.001时的验证率为99.2%,结果表明,该方法对非中性表情下的3维人脸识别具有很好的准确性。结论 该方法可以有效克服表情变化对3维人脸识别的影响,同时对3维数据中存在的空洞和尖锐噪声等因素具有较好的鲁棒性,对提高3维人脸识别性能具有重要意义。     

Passive Three Dimensional Face Recognition Using Iso-Geodesic Contours and Procrustes Analysis

We introduce a new model for personal recognition based on the 3-D geometry of the face. The model is designed for application scenarios where the acquisition conditions constrain the facial position. The 3-D structure of a facial surface is compactly represented by sets of contours (facial contours) extracted around automatically pinpointed nose tip and inner eye corners. The metric used to decide whether a point on the face belongs to a facial contour is its geodesic distance from a given landmark. Iso-geodesic contours are inherently robust to head pose variations, including in-depth rotations of the face. Since these contours are extracted from rigid parts of the face, the resulting recognition algorithms are insensitive to changes in facial expressions. The facial contours are encoded using innovative pose invariant features, including Procrustean distances defined on pose-invariant curves. The extracted features are combined in a hierarchical manner to create three parallel face recognizers. Inspired by the effectiveness of region ensembles approaches, the three recognizers constructed around the nose tip and inner corners of the eyes are fused both at the feature-level and the match score-level to create a unified face recognition algorithm with boosted performance. The performances of the proposed algorithms are evaluated and compared with other algorithms from the literature on a large public database appropriate for the assumed constrained application scenario.     

Line-based face recognition under varying pose

Much research in human face recognition involves fronto-parallel face images, constrained rotations in and out of the plane, and operates under strict imaging conditions such as controlled illumination and limited facial expressions. Face recognition using multiple views in the viewing sphere is a more difficult task since face rotations out of the imaging plane can introduce occlusion of facial structures. In this paper, we propose a novel image-based face recognition algorithm that uses a set of random rectilinear line segments of 2D face image views as the underlying image representation, together with a nearest-neighbor classifier as the line matching scheme. The combination of 1D line segments exploits the inherent coherence in one or more 2D face image views in the viewing sphere. The algorithm achieves high generalization recognition rates for rotations both in and out of the plane, is robust to scaling, and is computationally efficient. Results show that the classification accuracy of the algorithm is superior compared with benchmark algorithms and is able to recognize test views in quasi-real-time     

Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach

In this paper, we present the computational tools and a hardware prototype for 3D face recognition. Full automation is provided through the use of advanced multistage alignment algorithms, resilience to facial expressions by employing a deformable model framework, and invariance to 3D capture devices through suitable preprocessing steps. In addition, scalability in both time and space is achieved by converting 3D facial scans into compact metadata. We present our results on the largest known, and now publicly available, face recognition grand challenge 3D facial database consisting of several thousand scans. To the best of our knowledge, this is the highest performance reported on the FRGC v2 database for the 3D modality     

二维Gabor特征与三维NP-3DHOG特征融合的人脸识别算法

基于二维图像的人脸识别算法提取人脸纹理特征进行识别,但是光照、表情、人脸姿态等会对其产生不利影响。三维人脸特征能更精确地描述人脸的几何结构,并且不易受化妆和光照的影响,但只采用三维人脸数据进行人脸识别又缺少人脸纹理信息,因此文中将二维人脸特征与三维人脸特征相融合进行人脸识别。采用基于Gabor变换的二维特征与基于新的分块策略的三维梯度直方图特征相融合的算法进行人脸识别。首先,提取二维人脸的Gabor特征;然后,提取三维人脸基于新的分块策略的三维梯度直方图特征,旨在提取人脸的可辨别性特征;接下来,对二维人脸特征与三维人脸特征分别使用线性判别分析子空间算法进行训练,并使用加法原则融合两种特征的相似度矩阵;最后,输出识别结果。     

Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation

This study presents a facial expression recognition system which separates the non-rigid facial expression from the rigid head rotation and estimates the 3D rigid head rotation angle in real time. The extracted trajectories of the feature points contain both rigid head motion components and non-rigid facial expression motion components. A 3D virtual face model is used to obtain accurate estimation of the head rotation angle such that the non-rigid motion components can be precisely separated to enhance the facial expression recognition performance. The separation performance of the proposed system is further improved through the use of a restoration mechanism designed to recover feature points lost during large pan rotations. Having separated the rigid and non-rigid motions, hidden Markov models (HMMs) are employed to recognize a prescribed set of facial expressions defined in terms of facial action coding system (FACS) action units (AUs).     

Automatic 3D face recognition from depth and intensity Gabor features

As is well known, traditional 2D face recognition based on optical (intensity or color) images faces many challenges, such as illumination, expression, and pose variation. In fact, the human face generates not only 2D texture information but also 3D shape information. In this paper, we investigate what contributions depth and intensity information makes to face recognition when expression and pose variations are taken into account, and we propose a novel system for combining depth and intensity information to improve face recognition systems. In our system, local features described by Gabor wavelets are extracted from depth and intensity images, which are obtained from 3D data after fine alignment. Then a novel hierarchical selecting scheme embedded in linear discriminant analysis (LDA) and AdaBoost learning is proposed to select the most effective and most robust features and to construct a strong classifier. Experiments are performed on the CASIA 3D face database and the FRGC V2.0 database, two data sets with complex variations, including expressions, poses and long time lapses between two scans. Experimental results demonstrate the promising performance of the proposed method. In our system, all processes are performed automatically, thus providing a prototype of automatic face recognition combining depth and intensity information.     

An interpolation algorithm fitted for dynamic 3D face reconstruction

In order to solve the problem of low recognition accuracy in later period which is caused by the too few extracted parameters in the 3D face recognition, and the incapable formation of completed point cloud structure. An automatic iterative interpolation algorithm is proposed. The new and more accurate 3D face data points are obtained by automatic iteration. This algorithm can be used to restore the data point cloud information of 3D facial feature in 2D images by means of facial three-legged structure formed by 3D face and automatic interpolation. Thus, it can realize to shape the 3D facial dynamic model which can be recognized and has high saturability. Experimental results show that the interpolation algorithm can achieve the complete the construction of facial feature based on the facial feature after 3D dynamic reconstruction, and the validity is higher.     

结合径向线的局部遮挡下的3维人脸识别

目的 3维人脸点云的局部遮挡是影响3维人脸识别精度的一个重要因素。为克服局部遮挡对3维人脸识别的影响,提出一种基于径向线和局部特征的3维人脸识别方法。方法 首先为了充分利用径向线的邻域信息,提出用一组局部特征来表示径向线;其次对于点云稀疏引起的采样点不均匀,提出将部分相邻局部区域合并以减小采样不均匀的影响;然后,利用径向线的邻域信息构造代价函数,进而构造相应径向线间的相似向量。最后,利用相似向量来进行径向线匹配,从而完成3维人脸识别。结果 在FRGC v2.0数据库上进行不同局部特征识别率的测试实验,选取的局部特征Rank-1识别率达到了95.2%,高于其他局部特征的识别率;在Bosphorus数据库上进行不同算法局部遮挡下的人脸识别实验,Rank-1识别率达到了最高的92.0%;进一步在Bosphorus数据库上进行不同算法的时间复杂度对比实验,耗费时间最短,为8.17 s。该算法在准确率和耗时方面均取得了最好的效果。结论 基于径向线和局部特征的3维人脸方法能有效提取径向线周围的局部信息;局部特征的代价函数生成的相似向量有效减小了局部遮挡带来的影响。实验结果表明本文算法具有较高的精度和较短的耗时,同时对人脸的局部遮挡具有一定的鲁棒性。该算法适用于局部遮挡下的3维人脸识别,但是对于鼻尖部分被遮挡的人脸,无法进行识别。     

基于特征点表情变化的3维人脸识别

目的 为克服表情变化对3维人脸识别的影响,提出一种基于特征点提取局部区域特征的3维人脸识别方法。方法 首先,在深度图上应用2维图像的ASM(active shape model)算法粗略定位出人脸特征点,再根据Shape index特征在人脸点云上精确定位出特征点。其次,提取以鼻中为中心的一系列等测地轮廓线来表征人脸形状;然后,提取具有姿态不变性的Procrustean向量特征(距离和角度)作为识别特征;最后,对各条等测地轮廓线特征的分类结果进行了比较,并对分类结果进行决策级融合。结果 在FRGC V2.0人脸数据库分别进行特征点定位实验和识别实验,平均定位误差小于2.36 mm,Rank-1识别率为98.35%。结论 基于特征点的3维人脸识别方法,通过特征点在人脸近似刚性区域提取特征,有效避免了受表情影响较大的嘴部区域。实验证明该方法具有较高的识别精度,同时对姿态、表情变化具有一定的鲁棒性。