基于局部二值模式和深度学习的人脸识别

针对人脸识别中深度学习直接提取人脸特征时忽略了其局部结构特征的问题,提出一种将分块局部二值模式(LBP)与深度学习相结合的人脸识别方法.首先,将人脸图像分块,利用均匀LBP算子分别提取图像各局部的LBP直方图特征,再按照顺序连接在一起形成整个人脸的LBP纹理特征; 其次,将得到的LBP特征作为深度信念网络(DBN)的输入,逐层训练网络,并在顶层形成分类面; 最后,用训练好的深度信念网络对人脸样本进行识别.在ORL、YALE和FERET人脸库上的实验结果表明,所提算法与采用支持向量机(SVM)的方法相比,在小样本的人脸识别中有很好的识别效果.     

基于深度学习的重叠人脸检测

人脸检测在人机界面、安全系统、人脸识别、基于内容的图像检索等不同应用中起着重要作用。随着计算机图像技术的发展,人脸检测的方法也越来越多。但是利用现有的人脸检测方法检测重叠人脸时,虽然能够检测出部分人脸,但是相比于单人脸的检测,算法的效率和准确性都有所欠缺。针对这个问题,提出了一种基于深度学习的重叠人脸检测方法。首先基于机器学习方法,构建出多个人脸特征分类器,然后再利用肤色检测的方法对分类器得到的候选人脸进行二次检测,最后利用提出的一种NMS算法对候选人脸进行进一步的处理,从而检测出精确的人脸。为了验证算法的高效性和准确性,进行了多个人脸检测算法的对比实验,结果表明,该算法在效率和准确性方面都有较大提高。     

Learning warps based similarity for pose-unconstrained face recognition

Face recognition techniques are widely used in many applications, such as automatic detection of crime scenes from surveillance cameras for public safety. In these real cases, the pose and illumination variances between two matching faces have a big influence on the identification performance. Handling pose changes is an especially challenging task. In this paper, we propose the learning warps based similarity method to deal with face recognition across the pose problem. Warps are learned between two patches from probe faces and gallery faces using the Lucas-Kanade algorithm. Based on these warps, a frontal face registered in the gallery is transformed into a series of non-frontal viewpoints, which enables non-frontal probe face matching with the frontal gallery face. Scale-invariant feature transform (SIFT) keypoints (interest points) are detected from the generated viewpoints and matched with the probe faces. Moreover, based on the learned warps, the probability likelihood is used to calculate the probability of two faces being the same subject. Finally, a hybrid similarity combining the number of matching keypoints and the probability likelihood is proposed to describe the similarity between a gallery face and a probe face. Experimental results show that our proposed method achieves better recognition accuracy than other algorithms it was compared to, especially when the pose difference is within 40 degrees.

     

Improving mixture of experts for view-independent face recognition using teacher-directed learning

In this paper we develop a new learning method, called teacher-directed learning (TDL), for mixture of experts (ME) to perform view-independent face recognition. In the basic form of ME the problem space is automatically divided into several subspaces for the experts, and the outputs of experts are combined by a gating network. In our proposed method, the ME is directed to adapt to a particular partitioning corresponding to predetermined views. To do this, we apply a new learning method to ME, called TDL, in a way that according to the pose of the input training sample, only the weights of the corresponding experts are updated. We apply TDL to MEs, composed of MLP experts and a radial basis function gating network, with different representation schemes: global, single-view and overlapping eigenspace. We test them with previously intermediate unseen views of faces. The experimental results support our claim that directing the experts to a predetermined partitioning of the face space improves the performance of the conventional ME for view-independent face recognition. Comparison with some of the most related methods indicates that the proposed model yields excellent recognition rate in view-independent face recognition.     

Recognition in the near infrared spectrum for face,gender and facial expressions

Visible face recognition systems are subjected to failure when recognizing the faces in unconstrained scenarios. So, recognizing faces under variable and low illumination conditions are more important since most of the security breaches happen during night time. Near Infrared (NIR) spectrum enables to acquire high quality images, even without any external source of light and hence it is a good method for solving the problem of illumination. Further, the soft biometric trait, gender classification and non verbal communication, facial expression recognition has also been addressed in the NIR spectrum. In this paper, a method has been proposed to recognize the face along with gender classification and facial expression recognition in NIR spectrum. The proposed method is based on transfer learning and it consists of three core components, i) training with small scale NIR images ii) matching NIR-NIR images (homogeneous) and iii) classification. Training on NIR images produce features using transfer learning which has been pre-trained on large scale VIS face images. Next, matching is performed between NIR-NIR spectrum of both training and testing faces. Then it is classified using three, separate SVM classifiers, one for face recognition, the second one for gender classification and the third one for facial expression recognition. It has been observed that the method gives state-of-the-art accuracy on the publicly available, challenging, benchmark datasets CASIA NIR-VIS 2.0, Oulu-CASIA NIR-VIS, PolyU, CBSR, IIT Kh and HITSZ for face recognition. Further, for gender classification the Oulu-CASIA NIR-VIS, PolyU,and IIT Kh has been analyzed and for facial expression the Oulu-CASIA NIR-VIS dataset has been analyzed.

     

一种高性能的人脸识别方法

在基于人脸图像的身份认证系统中,最关键的技术就是如何提取人脸图像的高质量特征以及如何进行分类识别,该文就提出了一种快速、准确的人脸图像识别方法。该方法利用基于核函数的学习算法,进行人脸图像的特征提取和分类。首先,该方法分别利用核主分量分析以及核Fisher算法提取人脸图像的特征,然后对这些特征进行合理的组合以构成组合特征向量,再利用支持向量机进行识别。实验结果显示,所提出的高性能人脸识别方法的识别率高,即使对于轻度光照不均匀的人脸图像、人脸姿势的有限变化图像,也能获得较高的识别率;同时,该方法的训练速度和识别速度也非常快,完全满足人脸识别系统实时性要求。     

一种基于LBP特征与流形知识的人脸识别方法

人脸识别是计算机视觉领域的研究热点,应用背景广泛。近年来,流形被认为是视觉感知的基础,流形学习算法被用来发现图像的内在特征。如何利用流形学习后的低维内蕴变量成为相关研究的核心问题。但是利用传统的流形学习算法降维得到的人脸低维特征在可分性上存在一定的不足。此外,流形学习算法对光照和姿态变化敏感。针对这两个问题,提出了一种基于局部二值模式（LBP）和流形知识的人脸识别方法。该方法首先利用LBP算子对人脸图像进行局部特征描述,然后使用流形学习算法获得高维特征数据的低维内蕴变量,并用泰勒展开式近似该流形,获取流形知识,最后利用流形知识估计流形距离来实现人脸识别。实验证明,该方法增强了人脸识别对光照变化的鲁棒性,从而提高了识别性能。     

FaceSeg: Automatic Face Segmentation for Real-Time Video

Segmenting human faces automatically is very important for face recognition and verification, security system, and computer vision. In this paper, we present an accurate segmentation system for cutting human faces out from video sequences in real-time. First, a learning based face detector is developed to rapidly find human faces. To speed up the detection process, a face rejection cascade is constructed to remove most of negative samples while retaining all the face samples. Then, we develop a coarse-to-fine segmentation approach to extract the faces based on a min-cut optimization. Finally, a new matting algorithm is proposed to estimate the alpha-matte based on an adaptive trimap generation method. Experimental results demonstrate the effectiveness and robustness of our proposed method that can compete with the well-known interactive methods in real-time.     

基于混合核函数支持向量机和遗传算法的人脸识别

提出了一种基于混合核函数支持向量机和遗传算法的识别方法,用于人脸识别。该方法结合了支持向量机的学习性能和遗传算法的寻优性能,与传统的方法相比,具有速度快、误差少、效率高的特点,在实验中能够较精确地对人脸进行识别。     

基于多姿态特征融合生成对抗网络的人脸校正方法

针对人脸校正中单幅图像难以解决大姿态侧脸的问题,提出一种基于多姿态特征融合生成对抗网络（MFFGAN）的人脸校正方法,利用多幅不同姿态侧脸之间的相关信息来进行人脸校正,并采用对抗机制对网络参数进行调整。该方法设计了一种新的网络,包括由多姿态特征提取、多姿态特征融合、正脸合成三个模块组成的生成器,以及用于对抗训练的判别器。多姿态特征提取模块利用多个卷积层提取侧脸图像的多姿态特征;多姿态特征融合模块将多姿态特征融合成包含多姿态侧脸信息的融合特征;而正脸合成模块在进行姿态校正的过程中加入融合特征,通过探索多姿态侧脸图像之间的特征依赖关系来获取相关信息与全局结构,可以有效提高校正结果。实验结果表明,与现有基于深度学习的人脸校正方法相比,所提方法恢复出的正脸图像不仅轮廓清晰,而且从两幅侧脸中恢复出的正脸图像的识别率平均提高了1.9个百分点,并且输入侧脸图像越多,恢复出的正脸图像的识别率越高,表明所提方法可以有效融合多姿态特征来恢复出轮廓清晰的正脸图像。     

基于多姿态特征融合生成对抗网络的人脸校正方法

针对人脸校正中单幅图像难以解决大姿态侧脸的问题，提出一种基于多姿态特征融合生成对抗网络（MFFGAN）的人脸校正方法，利用多幅不同姿态侧脸之间的相关信息来进行人脸校正，并采用对抗机制对网络参数进行调整。该方法设计了一种新的网络，包括由多姿态特征提取、多姿态特征融合、正脸合成三个模块组成的生成器，以及用于对抗训练的判别器。多姿态特征提取模块利用多个卷积层提取侧脸图像的多姿态特征；多姿态特征融合模块将多姿态特征融合成包含多姿态侧脸信息的融合特征；而正脸合成模块在进行姿态校正的过程中加入融合特征，通过探索多姿态侧脸图像之间的特征依赖关系来获取相关信息与全局结构，可以有效提高校正结果。实验结果表明，与现有基于深度学习的人脸校正方法相比，所提方法恢复出的正脸图像不仅轮廓清晰，而且从两幅侧脸中恢复出的正脸图像的识别率平均提高了1.9个百分点，并且输入侧脸图像越多，恢复出的正脸图像的识别率越高，表明所提方法可以有效融合多姿态特征来恢复出轮廓清晰的正脸图像。     

3D reconstruction and face recognition using kernel-based ICA and neural networks

Kernel-based nonlinear characteristic extraction and classification algorithms are popular new research directions in machine learning. In this paper, we propose an improved photometric stereo scheme based on improved kernel-independent component analysis method to reconstruct 3D human faces. Next, we fetch the information of 3D faces for facial face recognition. For reconstruction, we obtain the correct normal vector’s sequence to form the surface, and use a method for enforcing integrability to reconstruct 3D objects. We test our algorithm on a number of real images captured from the Yale Face Database B, and use three kinds of methods to fetch characteristic values. Those methods are called contour-based, circle-based, and feature-based methods. Then, a three-layer, feed-forward neural network trained by a back-propagation algorithm is used to realize a classifier. All the experimental results were compared to those of the existing human face reconstruction and recognition approaches tested on the same images. The experimental results demonstrate that the proposed improved kernel independent component analysis (IKICA) method is efficient in reconstruction and face recognition applications.     

Audio-visual human recognition using semi-supervised spectral learning and hidden Markov models

This paper presents a multimodal system for reliable human identity recognition under variant conditions. Our system fuses the recognition of face and speech with a general probabilistic framework. For face recognition, we propose a new spectral learning algorithm, which considers not only the discriminative relations among the training data but also the generative models for each class. Due to the tedious cost of face labeling in practice, our spectral face learning utilizes a semi-supervised strategy. That is, only a small number of labeled faces are used in our training step, and the labels are optimally propagated to other unlabeled training faces. Besides requiring much less labeled data, our algorithm also enables a natural way to explicitly train an outlier model that approximately represents unauthorized faces. To boost the robustness of our system for human recognition under various environments, our face recognition is further complemented by a speaker identification agent. Specifically, this agent models the statistical variations of fixed-phrase speech using speaker-dependent word hidden Markov models. Experiments on benchmark databases validate the effectiveness of our face recognition and speaker identification agents, and demonstrate that the recognition accuracy can be apparently improved by integrating these two independent biometric sources together.     

基于视频监控的人脸识别方法

目前人脸识别方法主要针对静态图像进行识别,而在监控视频中,不同视频帧人脸具有相关性且只有部分人脸能够有效反映人脸信息。根据监控视频中人脸图像变化特性,提出了一种基于监控视频的人脸识别方法。首先通过结合人脸检测与跟踪技术获得视频人脸序列,然后以视频人脸序列中部分人脸图像识别结果为导向选取全部人脸序列图像中的代表人脸图像进行识别,最后根据选取的全部人脸图像识别结果综合反映人脸信息。实验结果表明,该方法能够在确保识别率和误识率的前提下有效提升监控视频中人脸识别的实时性。     

基于老化模型的跨年龄人脸识别研究进展

跨年龄人脸识别是目前人脸识别中的一大难点问题,人脸特征会随着年龄的增长发生变化,导致识别准确率降低,利用老化模型生成老化图像后进行人脸识别为该问题提供了一种解决方案。随着计算机技术和深度学习的广泛应用,人脸老化的真实性、老化效果、算法效率都得到了明显的提升,系统综述了基于老化模型的跨年龄人脸识别的研究现状,对人脸老化方法进行了详细地梳理,系统介绍了老化模型的方法演变和各类方法的优缺点,并对现有的模型评价方法进行了总结归纳。对现有的可用于跨年龄人脸识别的数据集进行了详细介绍,从数据量、年龄跨度、年龄准确性、数据集使用情况等方面进行了对比分析。结合实际应用对基于老化模型的跨年龄人脸识别中待解决的问题进行了分析和讨论,并对未来研究方向做出预测和展望。     

基于肤色模型和FloatBoost的多姿态人脸检测

多姿态人脸检测是人脸识别系统必须解决的关键问题之一。利用光照鲁棒的肤色模型来搜索待检图像的可能人脸区域并进行肤色分割,结合分割区域的几何信息确定最终的候选人脸区域,然后对人脸的关键特征进行定位,按规则计算重要特征块的中心,将这些中心点确定的符合条件的候选区域利用FloatBoost进行分类,最终实现了快速准确的多姿态人脸检测。     

结合形变模型与图像修复的人脸姿态矫正

目的 人脸姿态偏转是影响人脸识别准确率的一个重要因素,本文利用3维人脸重建中常用的3维形变模型以及深度卷积神经网络,提出一种用于多姿态人脸识别的人脸姿态矫正算法,在一定程度上提高了大姿态下人脸识别的准确率。方法 对传统的3维形变模型拟合方法进行改进,利用人脸形状参数和表情参数对3维形变模型进行建模,针对面部不同区域的关键点赋予不同的权值,加权拟合3维形变模型,使得具有不同姿态和面部表情的人脸图像拟合效果更好。然后,对3维人脸模型进行姿态矫正并利用深度学习对人脸图像进行修复,修复不规则的人脸空洞区域,并使用最新的局部卷积技术同时在新的数据集上重新训练卷积神经网络,使得网络参数达到最优。结果 在LFW（labeled faces in the wild）人脸数据库和StirlingESRC（Economic Social Research Council）3维人脸数据库上,将本文算法与其他方法进行比较,实验结果表明,本文算法的人脸识别精度有一定程度的提高。在LFW数据库上,通过对具有任意姿态的人脸图像进行姿态矫正和修复后,本文方法达到了96.57%的人脸识别精确度。在StirlingESRC数据库上,本文方法在人脸姿态为±22°的情况下,人脸识别准确率分别提高5.195%和2.265%;在人脸姿态为±45°情况下,人脸识别准确率分别提高5.875%和11.095%;平均人脸识别率分别提高5.53%和7.13%。对比实验结果表明,本文提出的人脸姿态矫正算法有效提高了人脸识别的准确率。结论 本文提出的人脸姿态矫正算法,综合了3维形变模型和深度学习模型的优点,在各个人脸姿态角度下,均能使人脸识别准确率在一定程度上有所提高。     

基于判别低秩矩阵恢复和协同表示的遮挡人脸识别

针对训练样本和测试样本均受到严重的噪声污染的人脸识别问题,传统的子空间学习方法和经典的基于稀疏表示的分类(SRC)方法的识别性能都将急剧下降。另外,基于稀疏表示的方法也存在算法复杂度较高的问题。为了在一定程度上缓解上述问题,提出一种基于判别低秩矩阵恢复和协同表示的遮挡人脸识别方法。首先,低秩矩阵恢复可以有效地从被污损的训练样本中恢复出干净的、具备低秩结构的训练样本,而结构非相关性约束的引入可以有效提高恢复数据的鉴别能力。然后,通过学习原始污损数据与恢复出的低秩数据之间的低秩投影矩阵,将受污损的测试样本投影到相应的低维子空间,以修正污损测试样本。最后,利用协同表示的分类方法(CRC)对修正后的测试样本进行分类,获取最终的识别结果。在Extended Yale B和AR数据库上的实验结果表明,本文方法对遮挡人脸识别具有更好的识别性能。     

Face Photo-Sketch Synthesis and Recognition

In this paper, we propose a novel face photo-sketch synthesis and recognition method using a multiscale Markov Random Fields (MRF) model. Our system has three components: 1) given a face photo, synthesizing a sketch drawing; 2) given a face sketch drawing, synthesizing a photo; and 3) searching for face photos in the database based on a query sketch drawn by an artist. It has useful applications for both digital entertainment and law enforcement. We assume that faces to be studied are in a frontal pose, with normal lighting and neutral expression, and have no occlusions. To synthesize sketch/photo images, the face region is divided into overlapping patches for learning. The size of the patches decides the scale of local face structures to be learned. From a training set which contains photo-sketch pairs, the joint photo-sketch model is learned at multiple scales using a multiscale MRF model. By transforming a face photo to a sketch (or transforming a sketch to a photo), the difference between photos and sketches is significantly reduced, thus allowing effective matching between the two in face sketch recognition. After the photo-sketch transformation, in principle, most of the proposed face photo recognition approaches can be applied to face sketch recognition in a straightforward way. Extensive experiments are conducted on a face sketch database including 606 faces, which can be downloaded from our Web site (http://mmlab.ie.cuhk.edu.hk/facesketch.html).