A multimodal approach for 3D face modeling and recognition using 3D deformable facial mask

We present a multimodal approach for face modeling and recognition. The algorithm uses three cameras to capture stereo images, two frontal and one profile, of the face. 2D facial features are extracted from one of the frontal images and a dense disparity map is computed from the two frontal images. Using the extracted 2D features and their corresponding disparities, we compute their 3D coordinates. We next align a low resolution 3D mesh model to the 3D features, re-project its vertices onto the frontal 2D image and adjust its profile silhouette vertices using the profile view image. We increase the resolution of the resulting 2D model at its center region to obtain a facial mask model covering distinctive features of the face. The 2D coordinates of the vertices, along with their disparities, result in a deformed 3D mask model specific to a given subject’s face. Our method integrates information from the extracted facial features from the 2D image modality with information from the 3D modality obtained from the stereo images. Application of the models in 3D face recognition, for 112 subjects, validates the algorithm with a 95% identification rate and 92% verification rate at 0.1% false acceptance rate.

New algorithm for 3D facial model reconstruction and its application in virtual reality

3D human face model reconstruction is essential to the generation of facial animations that is widely used in the field of virtual reality (VR). The main issues of 3D facial model reconstruction based on images by vision technologies are in twofold: one is to select and match the corresponding features of face from two images with minimal interaction and the other is to generate the realistic-looking human face model. In this paper, a new algorithm for realistic-looking face reconstruction is presented based on stereo vision. Firstly, a pattern is printed and attached to a planar surface for camera calibration, and corners generation and corners matching between two images are performed by integrating modified image pyramid Lucas-Kanade (PLK) algorithm and local adjustment algorithm, and then 3D coordinates of corners are obtained by 3D reconstruction. Individual face model is generated by the deformation of general 3D model and interpolation of the features. Finally, realistic-looking human face model     

二维Gabor特征与三维NP-3DHOG特征融合的人脸识别算法

基于二维图像的人脸识别算法提取人脸纹理特征进行识别,但是光照、表情、人脸姿态等会对其产生不利影响。三维人脸特征能更精确地描述人脸的几何结构,并且不易受化妆和光照的影响,但只采用三维人脸数据进行人脸识别又缺少人脸纹理信息,因此文中将二维人脸特征与三维人脸特征相融合进行人脸识别。采用基于Gabor变换的二维特征与基于新的分块策略的三维梯度直方图特征相融合的算法进行人脸识别。首先,提取二维人脸的Gabor特征;然后,提取三维人脸基于新的分块策略的三维梯度直方图特征,旨在提取人脸的可辨别性特征;接下来,对二维人脸特征与三维人脸特征分别使用线性判别分析子空间算法进行训练,并使用加法原则融合两种特征的相似度矩阵;最后,输出识别结果。     

基于二维半雕刻系统的三维人脸重建

三维人脸的重建是二维半雕刻系统的核心内容,提出了一种从正、侧面图像上自动提取特征点来修正通用模型从而重建三维人脸的方法。首先使用特征点模板匹配的方法自动提取出特征点,然后以这些特征点为依据,通过整体变换和径向基插值来修正通用模型得到特定人脸。同时建立了一个人脸模型库来获得特征点模板,并且提出了一种分块存储的方法可以更精确地针对脸部不同区域进行重建。实验验证,这种方法效率高、交互操作少、取得了满意的重建效果。     

An interpolation algorithm fitted for dynamic 3D face reconstruction

In order to solve the problem of low recognition accuracy in later period which is caused by the too few extracted parameters in the 3D face recognition, and the incapable formation of completed point cloud structure. An automatic iterative interpolation algorithm is proposed. The new and more accurate 3D face data points are obtained by automatic iteration. This algorithm can be used to restore the data point cloud information of 3D facial feature in 2D images by means of facial three-legged structure formed by 3D face and automatic interpolation. Thus, it can realize to shape the 3D facial dynamic model which can be recognized and has high saturability. Experimental results show that the interpolation algorithm can achieve the complete the construction of facial feature based on the facial feature after 3D dynamic reconstruction, and the validity is higher.     

Constructing a 3D individualized head model from two orthogonal views

A new scheme for constructing a 3D individualized head model automatically from only a side view and the front view of the face is presented. The approach instantiates a generic 3D head model based on a set of the individual's facial features extracted by a local maximum-curvature tracking (LMCT) algorithm that we have developed. A distortion vector field that deforms the generic model to that of the individual is computed by correspondence matching and interpolation. The input of the two facial images are blended and texture-mapped onto the 3D head model. Arbitrary views of a person can be generated from two orthogonal images and can be implemented efficiently on a low-cost, PC-based platform.     

基于特征分块的三维人脸重建和识别

针对传统三维人脸重建算法效率低且难以满足实际应用的缺陷,提出一种基于特征分块的三维人脸重建算法,并将此算法应用到三维人脸识别中,实现了基于特征分块的加权三维人脸识别。首先,利用基于平面模板的非均匀重采样法对原始数据进行归一化;其次,采用主动形状模型(ASM)算法对三维人脸和二维人脸图像进行特征定位和特征分块;然后,利用基于分块主元分析(PCA)的稀疏形变模型算法实现每个人脸分块的三维重建;最后,实现了此算法在三维人脸识别中的应用。实验表明,此重建算法具有较高的精度和重建效率,还可以达到全局最优,并且可以提高三维人脸的识别率。     

双目立体视觉的三维人脸重建方法

创建逼真的三维人脸模型始终是一个极具挑战性的课题.随着三维人脸模型在虚拟现实、视频监控、三维动画、人脸识别等领域的广泛应用,三维人脸重建成为计算机图像学和计算机视觉领域的一个研究热点.针对这一问题,提出一种基于双目立体视觉的三维人脸重建方法,重建过程中无需三维激光扫描仪和通用人脸模型.首先利用标定的2台摄像机获取人脸正面图像对,通过图像校正使图像对的极线对齐并且补偿摄像机镜头的畸变;在立体匹配方面,选择具有准确可靠视差的人脸边缘特征点作为种子像素,以种子像素的视差作为区域生长的视差,在外极线约束、单调性约束以及对应匹配的边缘特征点的约束下,进行水平扫描线上的区域生长,从而得到整个人脸区域的视差图,提高了对应点匹配的速度和准确度;最后,根据摄像机标定结果和立体匹配生成的视差图计算人脸空间散乱点的三维坐标,对人脸的三维点云进行三角剖分、网格细分和光顺处理.实验结果表明,该方法能够生成光滑、逼真的三维人脸模型,证明了该算法的有效性.     

基于改进的轮廓线算法的三维人脸识别方法

针对二维人脸识别中受表情、姿态以及光照等因素而影响识别率的问题,在分析人脸生理结构的基础上,提出了一种基于改进的轮廓线的三维人脸识别方法,即先提取三维人脸特征点,然后提取人脸轮廓线,最后利用人脸轮廓线和特征点构成的特征模型进行三维人脸识别。试验结果证明该方法提高了人脸识别率,并具有强抗干扰能力。     

基于SVM和D-S理论的三维人脸识别

针对三维人脸识别算法中的高精度分类器设计问题,采用人脸全局特征和局部特征共四个相互独立的多特征信息分类后进行D-S数据融合技术来实现。通过SVM分类器对三维人脸图像中相互独立的全局特征（面廓）和局部特征（眼睛、鼻子和嘴）共四个特征进行一对一的单特征识别,并将其结果进行数据归一化处理后,作为D-S证据理论的BPA,按照D-S理论融合全局特征和局部特征数据,计算出更加准确的识别结果。经过融合数据结果分析,发现该算法可靠有效,大大提高了三维人脸的识别效率。     

基于特征点表情变化的3维人脸识别

目的 为克服表情变化对3维人脸识别的影响,提出一种基于特征点提取局部区域特征的3维人脸识别方法。方法 首先,在深度图上应用2维图像的ASM(active shape model)算法粗略定位出人脸特征点,再根据Shape index特征在人脸点云上精确定位出特征点。其次,提取以鼻中为中心的一系列等测地轮廓线来表征人脸形状;然后,提取具有姿态不变性的Procrustean向量特征(距离和角度)作为识别特征;最后,对各条等测地轮廓线特征的分类结果进行了比较,并对分类结果进行决策级融合。结果 在FRGC V2.0人脸数据库分别进行特征点定位实验和识别实验,平均定位误差小于2.36 mm,Rank-1识别率为98.35%。结论 基于特征点的3维人脸识别方法,通过特征点在人脸近似刚性区域提取特征,有效避免了受表情影响较大的嘴部区域。实验证明该方法具有较高的识别精度,同时对姿态、表情变化具有一定的鲁棒性。     

改进的统计模型三维人脸特征点标定算法框架

自动三维人脸特征点标定是计算机视觉领域的研究热点,其广泛应用于人脸识别,人脸模型配准,表情识别,脸部动画等领域。通过对三维人脸样本统计建模,采用遗传算法对待匹配模型的生成数目进行参数优化,利用模型相似性匹配方法及其映射关系对三维人脸特征点进行自动标定。首先,对三维人脸数据预处理,然后对其统计建模并通过模型形变得到有映射关系的基准模型和待匹配模型。利用遗传算法对待匹配模型中的待匹配模型生成数目参数进行优化,生成与之对应的待匹配模型数;接着计算待测模型与待匹配模型的相似度。最后,利用模型相似度和模型映射关系,间接得到待测模型的特征点。实验结果表明,提出的算法是可行的,能够在一定程度上提高原有算法的效率。该算法可以自动标定三维人脸模型的特征点,当距离阈值为10像素时,39个三维人脸特征点定位的准确率都可以达到100%,并有效解决了传统方法中三维人脸模型平滑区域特征点精度不高的问题。     

对表情鲁棒的面部轮廓线3维人脸识别

目的表情变化是3维人脸识别面临的主要问题。为克服表情影响,提出了一种基于面部轮廓线对表情鲁棒的3维人脸识别方法。方法首先,对人脸进行预处理,包括人脸区域切割、平滑处理和姿态归一化,将所有的人脸置于姿态坐标系下;然后,从3维人脸模型的半刚性区域提取人脸多条垂直方向的轮廓线来表征人脸面部曲面;最后,利用弹性曲线匹配算法计算不同3维人脸模型间对应的轮廓线在预形状空间(preshape space)中的测地距离,将其作为相似性度量,并且对所有轮廓线的相似度向量加权融合,得到总相似度用于分类。结果在FRGC v2.0数据库上进行识别实验,获得97.1%的Rank-1识别率。结论基于面部轮廓线的3维人脸识别方法,通过从人脸的半刚性区域提取多条面部轮廓线来表征人脸,在一定程度上削弱了表情的影响,同时还提高了人脸匹配速度。实验结果表明,该方法具有较强的识别性能,并且对表情变化具有较好的鲁棒性。     

2D representation of facial surfaces for multi-pose 3D face recognition

The increasing availability of 3D facial data offers the potential to overcome the intrinsic difficulties faced by conventional face recognition using 2D images. Instead of extending 2D recognition algorithms for 3D purpose, this letter proposes a novel strategy for 3D face recognition from the perspective of representing each 3D facial surface with a 2D attribute image and taking the advantage of the advances in 2D face recognition. In our approach, each 3D facial surface is mapped homeomorphically onto a 2D lattice, where the value at each site is an attribute that represents the local 3D geometrical or textural properties on the surface, therefore invariant to pose changes. This lattice is then interpolated to generate a 2D attribute image. 3D face recognition can be achieved by applying the traditional 2D face recognition techniques to obtained attribute images. In this study, we chose the pose invariant local mean curvature calculated at each vertex on the 3D facial surface to construct the 2D attribute image and adopted the eigenface algorithm for attribute image recognition. We compared our approach to state-of-the-art 3D face recognition algorithms in the FRGC (Version 2.0), GavabDB and NPU3D database. Our results show that the proposed approach has improved the robustness to head pose variation and can produce more accurate 3D multi-pose face recognition.     

3D face recognition with sparse spherical representations

This paper addresses the problem of 3D face recognition using simultaneous sparse approximations on the sphere. The 3D face point clouds are first aligned with a fully automated registration process. They are then represented as signals on the 2-sphere in order to preserve depth and geometry information. Next, we implement a dimensionality reduction process with simultaneous sparse approximations and subspace projection. It permits to represent each 3D face by only a few spherical functions that are able to capture the salient facial characteristics, and hence to preserve the discriminant facial information. We eventually perform recognition by effective matching in the reduced space, where linear discriminant analysis can be further activated for improved recognition performance. The 3D face recognition algorithm is evaluated on the FRGC v.1.0 data set, where it is shown to outperform classical state-of-the-art solutions that work with depth images.     

2维双树复小波不确定度加权融合的人脸识别

在人脸识别中,传统小波、Gabor小波不能很好地表征人脸特征。提出2维双树复小波多频带不确定度加权融合的人脸识别算法,使用了人脸2维双树复小波多频带特征,计算多频带不确定度及其权值并结合多频带特征进行加权融合,能很好得到人脸的特征。该加权融合算法首先计算人脸2维双树复小波多个频带特征图,然后计算多个频带滤波不确定度权值,最后进行加权融合。同时使用了2维主成分分析(2DPCA)方法对特征向量进行子空间投影,应用欧氏距离作为相似测度实现分类识别。使用英国剑桥Olivetti实验室(ORL)图像库进行了测试,实验结果表明,提出的方法相对于使用2DPCA、Wavelet和Gabor小波的特征提取方法,取得了更好的识别效果。     

基于深度数据的空间人脸旋转角度估计

提出一种基于三维人脸深度数据的人脸姿态计算方法。利用人脸的深度数 据以及与其一一对应的灰度图像,根据微分几何原理和相应的曲率算法与人脸数据中的灰度 特征对人脸面部关键特征点定位,进而计算出人脸姿态在三维空间中的3 个姿态角。实验证 明该方法能在姿态变化情况下实现对人脸旋转角的准确估计,为进一步的人脸识别和表情分 析提供基础。     

EM enhancement of 3D head pose estimated by point at infinity

Head pose estimation is a key task for visual surveillance, HCI and face recognition applications. In this paper, a new approach is proposed for estimating 3D head pose from a monocular image. The approach assumes the full perspective projection camera model. Our approach employs general prior knowledge of face structure and the corresponding geometrical constraints provided by the location of a certain vanishing point to determine the pose of human faces. To achieve this, eye-lines, formed from the far and near eye corners, and mouth-line of the mouth corners are assumed parallel in 3D space. Then the vanishing point of these parallel lines found by the intersection of the eye-line and mouth-line in the image can be used to infer the 3D orientation and location of the human face. In order to deal with the variance of the facial model parameters, e.g. ratio between the eye-line and the mouth-line, an EM framework is applied to update the parameters. We first compute the 3D pose using some initially learnt parameters (such as ratio and length) and then adapt the parameters statistically for individual persons and their facial expressions by minimizing the residual errors between the projection of the model features points and the actual features on the image. In doing so, we assume every facial feature point can be associated to each of features points in 3D model with some a posteriori probability. The expectation step of the EM algorithm provides an iterative framework for computing the a posterori probabilities using Gaussian mixtures defined over the parameters. The robustness analysis of the algorithm on synthetic data and some real images with known ground-truth are included.     

采用草绘轮廓的3维人脸建模方法

为方便用户进行3维人脸形状设计,提出一种基于手绘轮廓的3维人脸建模方法。该方法的主要特点在于,一方面,引用姿态估计技术对人脸草图进行解析,将用户绘制的侧视人脸草图转换成对应的正视人脸草图,可支持用户选择多个视角绘制人脸;另一方面,采用多层映射机制建立人脸草图特征点与3维人脸特征点之间的一一对应关系,由对应特征点之间的形变量来控制生成3维人脸,保证草图笔画的几何形状信息能有效映射到3维模型中。实验结果表明,文中方法能快速生成形状新颖的特定人脸,可有效支持用户进行3维人脸形状的手绘建模。