基于三维数据与MMSV特征的二维人脸识别*

针对二维人脸识别对姿态与光照变化较为敏感的问题,提出了一种基于三维数据与混合多尺度奇异值特征MMSV(mixture of multi-scale singular value,MMSV)的二维人脸识别方法。在训练阶段,利用三维人脸数据与光照模型获取大量具有不同姿态和光照条件的二维虚拟图像,为构造完备的特征模板奠定基础;同时,通过子集划分有效地缓解了人脸特征提取过程中的非线性问题;最后对人脸图像进行MMSV特征提取,从而对人脸的全局与局部特征进行融合。在识别阶段,通过计算MMSV特征子空间距离完成分类识别。实验证明,提取到的MMSV特征包含有更多的鉴别信息,对姿态和光照变化具有理想的鲁棒性。该方法在WHU-3D数据库上取得了约98.4%的识别率。     

Automatic 3D face recognition from depth and intensity Gabor features

As is well known, traditional 2D face recognition based on optical (intensity or color) images faces many challenges, such as illumination, expression, and pose variation. In fact, the human face generates not only 2D texture information but also 3D shape information. In this paper, we investigate what contributions depth and intensity information makes to face recognition when expression and pose variations are taken into account, and we propose a novel system for combining depth and intensity information to improve face recognition systems. In our system, local features described by Gabor wavelets are extracted from depth and intensity images, which are obtained from 3D data after fine alignment. Then a novel hierarchical selecting scheme embedded in linear discriminant analysis (LDA) and AdaBoost learning is proposed to select the most effective and most robust features and to construct a strong classifier. Experiments are performed on the CASIA 3D face database and the FRGC V2.0 database, two data sets with complex variations, including expressions, poses and long time lapses between two scans. Experimental results demonstrate the promising performance of the proposed method. In our system, all processes are performed automatically, thus providing a prototype of automatic face recognition combining depth and intensity information.     

Kinect 下深度信息获取技术及其在三维目标识别中的应用综述

微软公司 2010 年推出的 Kinect 深度传感器能够同步提供场景深度和彩色信息,其应用的一个关键领域就是目标 识别。传统的目标识别大多限制在特殊的情形,如：手势识别、人脸识别,而大规模的目标识别是近年来的研究趋势。 通过 Kinect 得到的 RGB-D 数据集多为室内和办公环境下获取的多场景、多视角、分目标类型的数据集,为大规模的目标 识别算法设计提供了学习基础。同时,Kinect 获取的深度信息为目标识别提供了强有力的线索,利用深度信息的识别方法 较以前的方法具有无法比拟的优势,大大地提高了识别的精度。文章首先对 Kinect 的深度获取技术做了详细介绍;其次 对现有的 3D 目标识别方法进行综述,接着对已有的 3D 测试数据集进行分析和比较;最后对文章进行小结以及对未来 3D 目标识别算法和 3D 测试数据集的发展趋势作了简单的阐述。     

基于少量特征点的多模态人脸识别

针对二维人脸识别对姿态和光照变化较为敏感的问题,提出了一种基于少量特征点的多模态人脸识别方法。在训练阶段,对三维人脸数据进行二次处理和数据挖掘,为建立完备的特征模板奠定了基础;提出了一种简洁高效的样本聚类方法,克服了特征提取过程中的非线性问题;通过局部特征分析(Local Feature Analysis,LFA)实现了特征点"局部"与"全局"信息的融合。实验证明该方法在具有较高执行效率的同时,对人脸图像的姿态和光照变化具有理想的鲁棒性,在WHU-3D小规模人脸数据库上取得了98.06%的识别率。     

Learning for classification of traffic-related object on RGB-D data

In order to detect and recognize the traffic-related object, a learning-based classification approach is proposed on RGB-D data. Since RGB-D data can provide the depth information and thus make it capable of tackling the baffling issues such as overlapping, clustered background, the depth data obtained by Microsoft Kinect sensor is introduced in the proposed method for efficiently detecting and extracting the objects in the traffic scene. Moreover, we construct a feature vector, which combine the histograms of oriented gradients, 2D features and 3D Spin Image features, to represent the traffic-related objects. The feature vector is used as the input of the random forest for training a classifier and classifying the traffic-related objects. In experiments, by conducting efficiency and accuracy tests on RGB-D data captured in different traffic scenarios, the proposed method performs better than the typical support vector machine method. The results show that traffic-related objects can be efficiently detected, and the accuracy of classification can achieve higher than 98 %.     

Robust face recognition using 2D and 3D data: Pose and illumination compensation

The paper addresses the problem of face recognition under varying pose and illumination. Robustness to appearance variations is achieved not only by using a combination of a 2D color and a 3D image of the face, but mainly by using face geometry information to cope with pose and illumination variations that inhibit the performance of 2D face recognition. A face normalization approach is proposed, which unlike state-of-the-art techniques is computationally efficient and does not require an extended training set. Experimental results on a large data set show that template-based face recognition performance is significantly benefited from the application of the proposed normalization algorithms prior to classification.     

三维人脸识别研究综述

近二十多年来,虽然基于图像的人脸识别已取得很大进展,并可在约束环境下获得很好的识别性能,但仍受光照、姿态、表情等变化的影响很大,其本质原因在于图像是三维物体在二维空间的简约投影.因此,利用脸部曲面的显式三维表达进行人脸识别正成为近几年学术界的研究热点.文中分析了三维人脸识别的产生动机、概念与基本过程;根据特征形式,将三维人脸识别算法分为基于空域直接匹配、基于局部特征匹配、基于整体特征匹配三大类进行综述;对二维和三维的双模态融合方法进行分类阐述;列出了部分代表性的三维人脸数据库;对部分方法进行实验比较,并分析了方法有效性的原因;总结了目前三维人脸识别技术的优势与困难,并探讨了未来的研究趋势.     

基于kinect的改进RGB-D视觉里程计

针对RGB-D视觉里程计中kinect相机所捕获的图像深度区域缺失的问题,提出了一种基于PnP(perspective-n-point)和ICP(iterative closest point)的融合优化算法。传统ICP算法迭代相机位姿时由于深度缺失,经常出现特征点丢失导致算法无法收敛或误差过大。本算法通过对特征点的深度值判定,建立BA优化模型,并利用g2o求解器进行特征点与相机位姿的优化。实验证明了该方法的有效性,提高了相机位姿估计的精度及算法的收敛成功率,从而提高了RGB-D视觉里程计的精确性和鲁棒性。     

基于Kinect的三维视觉里程计的设计

针对移动服务机器人在未知环境下三维路径估计的问题,设计了一种基于Kinect的实时估计机器人运动轨迹的方法。该方法采用Kinect获取机器人运动过程中连续帧的彩色和深度信息,首先,提取并匹配目标帧和参考帧的SURF的特征点;然后,结合深度信息利用经典P3P问题的方法及改进的随机采样一致性(RANSAC)算法计算机器人的初始6自由度(DOF)位姿;最后,通过非线性最小二乘算法最小化初始位姿内点的双向投影误差来提高位姿精度,进而得到机器人的运动轨迹。同时对比了不同特征点及描述符结合下的里程计精度。实验结果表明,所提方法能够将里程计误差降低到3.1%,且能够满足实时要求,可为机器人同时定位与地图创建提供重要的先验信息。     

基于协作表示残差融合的3维人脸识别

目的 针对2维人脸难以克服光照、表情、姿态等复杂问题,提出了一种基于协作表示残差融合的新算法.方法 协作表示分类算法是将所有类的训练图像一起协作构成字典,通过正则化最小二乘法代替1范数求解稀疏系数,减小了计算的复杂度,由此系数重构测试人脸,根据重构误差最小原则,对测试人脸正确分类.该方法首先在3维人脸深度图上提取Gabor特征和Geodesic特征,然后在协作表示算法的基础上融合两者的残差信息,作为最终差异性度量,最后根据融合残差最小原则,进行人脸识别.结果 在不同的训练样本、特征维数条件下,在CIS和Texas 2 个人脸数据库上,本文算法的识别率可分别达到94.545%和99.286%.与Gabor-CRC算法相比,本文算法的识别率平均高出了10%左右.结论 在实时成像系统采集的人脸库和Texas 3维人脸库上的实验结果表明,该方法对有无姿态、表情、遮挡等变化问题具有较好的鲁棒性和有效性.     

Matching 2.5D face scans to 3D models

The performance of face recognition systems that use two-dimensional images depends on factors such as lighting and subject's pose. We are developing a face recognition system that utilizes three-dimensional shape information to make the system more robust to arbitrary pose and lighting. For each subject, a 3D face model is constructed by integrating several 2.5D face scans which are captured from different views. 2.5D is a simplified 3D (x,y,z) surface representation that contains at most one depth value (z direction) for every point in the (x, y) plane. Two different modalities provided by the facial scan, namely, shape and texture, are utilized and integrated for face matching. The recognition engine consists of two components, surface matching and appearance-based matching. The surface matching component is based on a modified iterative closest point (ICP) algorithm. The candidate list from the gallery used for appearance matching is dynamically generated based on the output of the surface matching component, which reduces the complexity of the appearance-based matching stage. Three-dimensional models in the gallery are used to synthesize new appearance samples with pose and illumination variations and the synthesized face images are used in discriminant subspace analysis. The weighted sum rule is applied to combine the scores given by the two matching components. Experimental results are given for matching a database of 200 3D face models with 598 2.5D independent test scans acquired under different pose and some lighting and expression changes. These results show the feasibility of the proposed matching scheme.     

Real-Time Accurate 3D Head Tracking and Pose Estimation with Consumer RGB-D Cameras

We demonstrate how 3D head tracking and pose estimation can be effectively and efficiently achieved from noisy RGB-D sequences. Our proposal leverages on a random forest framework, designed to regress the 3D head pose at every frame in a temporal tracking manner. One peculiarity of the algorithm is that it exploits together (1) a generic training dataset of 3D head models, which is learned once offline; and, (2) an online refinement with subject-specific 3D data, which aims for the tracker to withstand slight facial deformations and to adapt its forest to the specific characteristics of an individual subject. The combination of these works allows our algorithm to be robust even under extreme poses, where the user’s face is no longer visible on the image. Finally, we also propose another solution that utilizes a multi-camera system such that the data simultaneously acquired from multiple RGB-D sensors helps the tracker to handle challenging conditions that affect a subset of the cameras. Notably, the proposed multi-camera frameworks yields a real-time performance of approximately 8 ms per frame given six cameras and one CPU core, and scales up linearly to 30 fps with 25 cameras.     

加权信息熵与增强局部二值模式结合的人脸识别

针对人脸识别因光照、姿态、表情、遮挡及噪声等多种因素的影响而导致的识别率不高的问题，提出一种加权信息熵（IEw）与自适应阈值环形局部二值模式（ATRLBP）算子相结合的人脸识别方法（IE （w） ATR-LBP）。首先，从原始人脸图像分块提取信息熵，得到每个子块的IEw；然后，利用ATRLBP算子分别对每个人脸子块提取特征从而得到概率直方图；最后，将各个块的IEw与概率直方图相乘，再串联成为原始人脸图像最后的特征直方图，并利用支持向量机（SVM）对人脸进行识别。在AR人脸库的表情、光照、遮挡A和遮挡B四个数据集上，IE （w） ATR-LBP方法分别取得了98.37%、94.17%、98.20%和99.34%的识别率。在ORL人脸库上，IE （w） ATR-LBP方法的最大识别率为99.85%；而且在ORL人脸库5次不同训练样本的实验中，与无噪声时相比，加入高斯和椒盐噪声后的平均识别率分别下降了14.04和2.95个百分点。实验结果表明，IE （w） ATR-LBP方法能够有效提高人脸在受光照、姿态、遮挡等影响时的识别率，尤其是存在表情变化及脉冲类噪声干扰时的识别率。     

Illumination invariant extraction for face recognition using neighboring wavelet coefficients

The features of a face can change drastically as the illumination changes. In contrast to pose position and expression, illumination changes present a much greater challenge to face recognition. In this paper, we propose a novel wavelet based approach that considers the correlation of neighboring wavelet coefficients to extract an illumination invariant. This invariant represents the key facial structure needed for face recognition. Our method has better edge preserving ability in low frequency illumination fields and better useful information saving ability in high frequency fields using wavelet based NeighShrink denoise techniques. This method proposes different process approaches for training images and testing images since these images always have different illuminations. More importantly, by having different processes, a simple processing algorithm with low time complexity can be applied to the testing image. This leads to an easy application to real face recognition systems. Experimental results on Yale face database B and CMU PIE Face Database show that excellent recognition rates can be achieved by the proposed method.     

Robust face recognition using generalized neural reflectance model

A generalized neural reflectance (GNR) model for enhancing face recognition under variations in illumination and posture is presented in this paper. Our work is based on training a number of synthesis images of each face taken at single lighting direction with frontal/posture view. This way of synthesizing images can be used to build training cases for each face under different known illumination conditions from which face recognition can be significantly improved. However, reconstructing face shape may not easily be achieved and the human face images usually form by highly complex structure which suffers from strong specular and unknown reflective conditions. In this paper, these limitations are addressed by Cho and Chow (IEEE Trans Neural Netw 12(5):1204–1214, 2002). Face surfaces are recovered by this GNR model and face images in different poses are synthesized to create a database for training. Our training algorithm assigns to recognize the face identity by similarity measure on face features extracting first by the principle component analysis (PCA) method and then further processing by the Fisher’s discrimination analysis (FDA) to acquire lower dimensional patterns. Experimental results conducted on the Yale Face Database B show that lower error rates of classification and recognition are achieved under different variations in lighting and pose and the performance significantly outperforms the recognition without using the proposed GNR model.     

Expressions Recognition of North-East Indian (NEI) Faces

Facial expression is one of the major distracting factors for face recognition performance. Pose and illumination variations on face images also influence the performance of face recognition systems. The combination of three variations (facial expression, pose and illumination) seriously degrades the recognition accuracy. In this paper, three experimental protocols are designed in such a way that the successive performance degradation due to the increasing variations (expressions, expressions with illumination effect and expressions with illumination and pose effect) on face images can be examined. The whole experiment is carried out using North-East Indian (NEI) face images with the help of four well-known classification algorithms namely Linear Discriminant Analysis (LDA), K-Nearest Neighbor algorithm (KNN), combination of Principal Component Analysis and Linear Discriminant Analysis (PCA + LDA), combination of Principal Component Analysis and K-Nearest Neighbor algorithm (PCA + KNN). The experimental observations are analyzed through confusion matrices and graphs. This paper also describes the creation of NEI facial expression database, which contains visual static face images of different ethnic groups of the North-East states. The database is useful for future researchers in the area of forensic science, medical applications, affective computing, intelligent environments, lie detection, psychiatry, anthropology, etc.     

结合像素模式和特征点模式的实时表情识别

目的 目前2D表情识别方法对于一些混淆性较高的表情识别率不高并且容易受到人脸姿态、光照变化的影响,利用RGBD摄像头Kinect获取人脸3D特征点数据,提出了一种结合像素2D特征和特征点3D特征的实时表情识别方法。方法 首先,利用3种经典的LBP（局部二值模式）、Gabor滤波器、HOG（方向梯度直方图）提取了人脸表情2D像素特征,由于2D像素特征对于人脸表情描述能力的局限性,进一步提取了人脸特征点之间的角度、距离、法向量3种3D表情特征,以对不同表情的变化情况进行更加细致地描述。为了提高算法对混淆性高的表情识别能力并增加鲁棒性,将2D像素特征和3D特征点特征分别训练了3组随机森林模型,通过对6组随机森林分类器的分类结果加权组合,得到最终的表情类别。结果 在3D表情数据集Face3D上验证算法对9种不同表情的识别效果,结果表明结合2D像素特征和3D特征点特征的方法有利于表情的识别,平均识别率达到了84.7%,高出近几年提出的最优方法4.5%,而且相比单独地2D、3D融合特征,平均识别率分别提高了3.0%和5.8%,同时对于混淆性较强的愤怒、悲伤、害怕等表情识别率均高于80%,实时性也达到了10~15帧/s。结论 该方法结合表情图像的2D像素特征和3D特征点特征,提高了算法对于人脸表情变化的描述能力,而且针对混淆性较强的表情分类,对多组随机森林分类器的分类结果加权平均,有效地降低了混淆性表情之间的干扰,提高了算法的鲁棒性。实验结果表明了该方法相比普通的2D特征、3D特征等对于表情的识别不仅具有一定的优越性,同时还能保证算法的实时性。     

三维人脸建模及在跨姿态人脸匹配中的有效性验证

针对现有三维人脸采集技术对采集场景存在诸多限制,提出了自由场景下基于多张图像的三维人脸建模技术,并对其进行了有效性验证。首先,提出一个姿态及深度值迭代计算模型,实现了特征点深度值的准确估计;然后,进行了基于多张图像的深度值融合及整体形状建模;最后,将深度迭代优化算法（IPDO）与目前最优的非线性最小二乘法（NLS1_SR）在Bosphorus Database数据集上进行了对比,建模精度提高了9%,所重建的三维人脸模型投影图像与二维图像具有较高的相似度。实验结果表明,在大姿态变化条件下,该识别算法借助三维信息相较于未借助的情况下,其识别率可以提高50%以上。     

结合径向线的局部遮挡下的3维人脸识别

目的 3维人脸点云的局部遮挡是影响3维人脸识别精度的一个重要因素。为克服局部遮挡对3维人脸识别的影响,提出一种基于径向线和局部特征的3维人脸识别方法。方法 首先为了充分利用径向线的邻域信息,提出用一组局部特征来表示径向线;其次对于点云稀疏引起的采样点不均匀,提出将部分相邻局部区域合并以减小采样不均匀的影响;然后,利用径向线的邻域信息构造代价函数,进而构造相应径向线间的相似向量。最后,利用相似向量来进行径向线匹配,从而完成3维人脸识别。结果 在FRGC v2.0数据库上进行不同局部特征识别率的测试实验,选取的局部特征Rank-1识别率达到了95.2%,高于其他局部特征的识别率;在Bosphorus数据库上进行不同算法局部遮挡下的人脸识别实验,Rank-1识别率达到了最高的92.0%;进一步在Bosphorus数据库上进行不同算法的时间复杂度对比实验,耗费时间最短,为8.17 s。该算法在准确率和耗时方面均取得了最好的效果。结论 基于径向线和局部特征的3维人脸方法能有效提取径向线周围的局部信息;局部特征的代价函数生成的相似向量有效减小了局部遮挡带来的影响。实验结果表明本文算法具有较高的精度和较短的耗时,同时对人脸的局部遮挡具有一定的鲁棒性。该算法适用于局部遮挡下的3维人脸识别,但是对于鼻尖部分被遮挡的人脸,无法进行识别。