Cross-view information interaction and feedback network for face hallucination

Hallucinating a photo-realistic frontal face image from a low-resolution (LR) non-frontal face image is beneficial for a series of face-related applications. However, previous efforts either focus on super-resolving high-resolution (HR) face images from nearly frontal LR counterparts or frontalizing non-frontal HR faces. It is necessary to address all these challenges jointly for real-world face images in unconstrained environment. In this paper, we develop a novel Cross-view Information Interaction and Feedback Network (CVIFNet), which simultaneously handles the non-frontal LR face image super-resolution (SR) and frontalization in a unified framework and interacts them with each other to further improve their performance. Specifically, the CVIFNet is composed of two feedback sub-networks for frontal and profile face images. Considering the reliable correspondence between frontal and non-frontal face images can be crucial and contribute to face hallucination in a different manner, we design a cross-view information interaction module (CVIM) to aggregate HR representations of different views produced by the SR and frontalization processes to generate finer face hallucination results. Besides, since 3D rendered facial priors contain rich hierarchical features, such as low-level (e.g., sharp edge and illumination) and perception level (e.g., identity) information, we design an identity-preserving consistency loss based on 3D rendered facial priors, which can ensure that the high-frequency details of frontal face hallucination result are consistent with the profile. Extensive experiments demonstrate the effectiveness and advancement of CVIFNet.     

面向人脸表情识别的迁移卷积神经网络研究

人脸表情识别是模式识别研究的一个重要领域,现实环境中人脸表情识别容易受到光照、姿态、个体表情差异等因素的影响,识别效果仍有待提高。为了取得更好的人脸表情识别效果,本文提出一种基于迁移卷积神经网络的人脸表情识别方法,本文在训练得到人脸识别网络模型的基础上,采用迁移学习方法将所得人脸识别模型迁移到人脸表情识别任务上,并提出Softmax-MSE损失函数和双激活层(Double Activate Layer, DAL)结构,以提高模型的识别能力。在FER2013数据库和SFEW2.0数据库上的实验表明,本文所提方法分别取得了61.59%和47.23%的主流识别效果。      

一种用于细粒度人脸识别的眼镜去除方法

为解决眼镜遮挡会降低人脸识别性能的难点,借鉴深度卷积神经网络在超分辨率方面的成功应用,该文提出一种用于细粒度人脸识别的眼镜自动去除方法ERCNN.用卷积层、池化层、MFM特征选取模块和反卷积层设计ERCNN网络模型,自动学习戴眼镜和未戴眼镜人脸图像对之间的映射关系,实现端到端的眼镜去除.然后,收集大量监控场景下的人脸图像,以及互联网上公开的人脸图像作为训练集;同时构建SLLFW数据集,作为眼镜去除和人脸识别的测试集.最后,通过与传统的眼镜去除方法进行对比试验,该文算法的各项评价指标优于传统方法,能有效的去除真实人脸图像中眼镜;同时在SLLFW人脸数据集上形成的全框眼镜、半框眼镜和无框眼镜人脸数据集上对多种人脸识别算法进行对比试验.试验表明,在FAR为1%的情况下,利用该文方法对F-SLLFW, H-SLLFW和R-SLLFW数据集的人脸图像进行眼镜去除后,SphereFace算法的TAR分别达到90.05%, 91.14%和92.33%,比未去除眼镜的识别率分别提高了3.92%, 3.08%和1.26%;同样,在FAR为0.1%的情况下,比SphereFace算法的TAR分别提高了10.06%, 4.29%和2.13%,说明该文方法有助于提升细粒度人脸识别的识别精度.     

High-quality face image generation based on generative adversarial networks

Conventional face image generation using generative adversarial networks (GAN) is limited by the quality of generated images since generator and discriminator use the same backpropagation network. In this paper, we discuss algorithms that can improve the quality of generated images, that is, high-quality face image generation. In order to achieve stability of network, we replace MLP with convolutional neural network (CNN) and remove pooling layers. We conduct comprehensive experiments on LFW, CelebA datasets and experimental results show the effectiveness of our proposed method.     

结合局部二值模式和卷积神经网络的人脸美丽预测

卷积神经网络(Convolution Neural Network,CNN)用于人脸美丽预测,能学习到深层次的特征表达,但提取的是全局特征,忽略了人脸的局部信息,因此,泛化能力不强。为此,本文提出一种结合局部二值模式(Local binary pattern , LBP)和卷积神经网络的人脸美丽预测算法。首先,利用数据增强技术扩大数据库规模;其次,将LBP纹理图像和原始灰度图像进行通道融合;再采用1×1卷积操作进行通道特征图的线性组合,从而实现网络跨通道的信息整合,提升人脸美丽预测精度。基于大规模亚洲女性人脸美丽数据库(Large Scale Asian Female Beauty Database, LSAFBD)的实验结果表明,该算法在分类和回归预测中均取得了较好效果,优于其他模型的人脸美丽预测算法;表明在卷积神经网络中加入纹理图像能有效提升人脸美丽预测精度。      

Automatic facial expression recognition: feature extraction and selection

In this paper, we investigate feature extraction and feature selection methods as well as classification methods for automatic facial expression recognition (FER) system. The FER system is fully automatic and consists of the following modules: face detection, facial detection, feature extraction, selection of optimal features, and classification. Face detection is based on AdaBoost algorithm and is followed by the extraction of frame with the maximum intensity of emotion using the inter-frame mutual information criterion. The selected frames are then processed to generate characteristic features using different methods including: Gabor filters, log Gabor filter, local binary pattern (LBP) operator, higher-order local autocorrelation (HLAC) and a recent proposed method called HLAC-like features (HLACLF). The most informative features are selected based on both wrapper and filter feature selection methods. Experiments on several facial expression databases show comparisons of different methods.     

词汇树融合SIFT的热成像和视觉图像人脸识别

针对热成像和视觉图像人脸识别问题,提出了一种基于词汇树融合尺度不变特征变换方法。首先,对视觉和热成像图像分别单独进行提取,利用Viola-Jones层叠检测器从自然图像中检测出人脸;然后,利用SIFT描述符从尺度空间提取稳定特征;最后,使用词汇树进行分类,利用评分融合和决策融合算法提高系统的精确性和安全性。在拍摄的41个人的脸部图像上的实验表明了该方法的有效性,识别率可接近100%,相比其他几种较为新颖的人脸识别方法,该方法取得了更高的识别精度,并且在一定程度上降低了计算耗时。     

Face recognition system based on CNN and LBP features for classifier optimization and fusion

Face recognition has been a hot-topic in the field of pattern recognition where feature extraction and classification play an important role. However, convolutional neural network (CNN) and local binary pattern (LBP) can only extract single features of facial images, and fail to select the optimal classifier. To deal with the problem of classifier parameter optimization, two structures based on the support vector machine (SVM) optimized by artificial bee colony (ABC) algorithm are proposed to classify CNN and LBP features separately. In order to solve the single feature problem, a fusion system based on CNN and LBP features is proposed. The facial features can be better represented by extracting and fusing the global and local information of face images. We achieve the goal by fusing the outputs of feature classifiers. Explicit experimental results on Olivetti Research Laboratory (ORL) and face recognition technology (FERET) databases show the superiority of proposed approaches.     

基于MTCNN和Facenet的人脸识别

人脸识别技术在智能城市建设中广泛应用,传统人脸识别算法依赖人工设定的特征,通常会带来不可期望的人为因素和误差。随着计算机算力的提升,基于神经网络的人脸识别方法由于其准确高效深受工业界偏爱。提出了基于多任务卷积神经网络(MTCNN——Multi-task Cascaded Convolutional Networks)和Facenet的人脸识别方法,并实现了从图像处理到识别结果输出的整个人脸识别系统。     

Robust face anti-spoofing with depth information

With the prevalence of face authentication applications, the prevention of malicious attack from fake faces such as photos or videos, i.e., face anti-spoofing, has attracted much attention recently. However, while an increasing number of works on the face anti-spoofing have been reported based on 2D RGB cameras, most of them cannot handle various attacking methods. In this paper we propose a robust representation jointly modeling 2D textual information and depth information for face anti-spoofing. The textual feature is learned from 2D facial image regions using a convolutional neural network (CNN), and the depth representation is extracted from images captured by a Kinect. A face in front of the camera is classified as live if it is categorized as live using both cues. We collected a face anti-spoofing experimental dataset with depth information, and reported extensive experimental results to validate the robustness of the proposed method.     

基于双激活层深度卷积特征的人脸美丽预测研究

目前,人脸美丽预测存在数据规模小、分类难度大、深度特征研究不足等问题.为此,本文提出基于双激活层深度卷积特征的人脸美丽预测研究的解决方案.首先,采用数据增强和人脸对齐方法来增加训练集的样本数量和提高数据库的数据质量.其次,提出一种双激活层改进CNN模型,使其更适合人脸美丽预测应用.实验结果表明,本文所提方法在分类和回归预测方面均大幅度优于传统人脸美丽预测方法;同时,在主流的CNN模型中取得了较好的实时性和准确性,基于2000测试集的分类准确率达到61.1%,回归相关度达到0.8546.因此,双激活层在深层人脸美丽特征学习中发挥了重要作用,可广泛应用于人脸图像识别与处理.     

Multi-pose 3D face recognition based on 2D sparse representation

The increasing availability of 3D facial data offers the potential to overcome the difficulties inherent with 2D face recognition, including the sensitivity to illumination conditions and head pose variations. In spite of their rapid development, many 3D face recognition algorithms in the literature still suffer from the intrinsic complexity in representing and processing 3D facial data. In this paper, we propose the intrinsic 3D facial sparse representation (I3DFSR) algorithm for multi-pose 3D face recognition. In this algorithm, each 3D facial surface is first mapped homeomorphically onto a 2D lattice, where the value at each site is the depth of the corresponding vertex on the 3D surface. Each 2D lattice is then interpolated and converted into a 2D facial attribute image. Next, the sparse representation is applied to those attribute images. Finally, the identity of each query face can be obtained by using the corresponding sparse coefficients. The innovation of our approach lies in the strategy of converting irregular 3D facial surfaces into regular 2D attribute images such that 3D face recognition problem can be solved by using the sparse representation of those attribute images. We compare the proposed algorithm to three widely used 3D face recognition algorithms in the GavabDB database, to six state-of-the-art algorithms in the FRGC2.0 database, and to three baseline algorithms in the NPU3D database. Our results show that the proposed I3DFSR algorithm can substantially improve the accuracy and efficiency of multi-pose 3D face recognition.     

FaceHunter: A multi-task convolutional neural network based face detector

In this paper, we propose a new multi-task Convolutional Neural Network (CNN) based face detector, which is named FaceHunter for simplicity. The main idea is to make the face detector achieve a high detection accuracy and obtain much reliable face boxes. Reliable face boxes output will be much helpful for further face image analysis. To reach this goal, we design a deep CNN network with a multi-task loss, i.e., one is for discriminating face and non-face, and another is for face box regression. An adaptive pooling layer is added before full connection to make the network adaptive to variable candidate proposals, and the truncated SVD is applied to compress the parameters of the fully connected layers. To further speed up the detector, the convolutional feature map is directly used to generate the candidate proposals by using Region Proposal Network (RPN). The proposed FaceHunter is evaluated on the AFW dataset, FDDB dataset and Pascal Faces respectively, and extensive experiments demonstrate its powerful performance against several state-of-the-art detectors.     

Micro-expression recognition from local facial regions

MiE is a facial involuntary reaction that reflects the real emotion and thoughts of a human being. It is very difficult for a normal human to detect a Micro-Expression (MiE), since it is a very fast and local face reaction with low intensity. As a consequence, it is a challenging task for researchers to build an automatic system for MiE recognition. Previous works for MiE recognition have attempted to use the whole face, yet a facial MiE appears in a small region of the face, which makes the extraction of relevant features a hard task. In this paper, we propose a novel deep learning approach that leverages the locality aspect of MiEs by learning spatio-temporal features from local facial regions using a composite architecture of Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM). The proposed solution succeeds to extract relevant local features for MiEs recognition. Experimental results on benchmark datasets demonstrate the highest recognition accuracy of our solution with respect to state-of-the-art methods.     

一种基于相关性投影的人脸性别分类算法

针对于人脸图像检测的有效利用性,为了提高其检测的性能,提出一种新的基于 监督学习的优化相关性投影(ORP)人脸性别分类算法,并将其应用到基 于Eigenface算法与Fisherface算法的人脸识别中,以及应 用WPCA到基于PGA的性别分类中。本文算法首先基于带权主成分分析(WPCA)算法来降低脸部 维度,将脸部特征提取出;然后,对其进行优化,同时 计算ORP的误差函数;最后,最小化脸部ORP误差函数,计算特征向量的 欧式距离,进行人脸性别分类。将提出方法与 传统方法进行对比,在FERET数据库上进行了实验,证明了本文方法的有效性,获得了优 于传统方法的识别率。     

低分辨率KINECT传感器采集三维点云的人脸识别

针对传统的三维人脸识别算法成本较高且不能很好地处理带有光照、表情等变化人脸识别的问题,设计了低分辨率Kinect传感器采集三维点云的鲁棒人脸识别系统。首先,通过鼻尖检测、人脸剪裁、姿势校正、对称填充及平滑采样得到规范的纹理图像;然后,在纹理图像上运用判别色彩空间变换,从而最大化类与类之间的分离性;最后,利用多模态稀疏编码有效地重建误差以得到查询图像与训练集之间的相似度,并利用Z-得分技术完成最终的人脸识别。在通用人脸数据库CurtinFaces、PIE及AR上的识别率可高达96.7%,实验结果表明,相比其它几种人脸识别算法,本文算法取得了更好的识别效果。     

基于统计学习优化SIFT的面部遮挡人脸识别

针对传统的人脸识别算法受面部遮挡的影响导致很难兼顾鲁棒性和保持原始图像核心信息的问题,本文提出了一种基于统计学习优化尺度不变特征变换的面部遮挡人脸识别算法。首先,利用SIFT将所有给定训练图像用一组局部特征描述符表示出来;然后,通过执行统计学习获得正常脸部图像SIFT特征的概率分布函数,利用获得的概率分布函数在新观察到的测试图像中检测异常SIFT特征;最后,计算测试图像与训练图像之间的相似度,并利用K近邻分类器完成人脸识别。在AR人脸数据库上的实验验证了本文算法的有效性及可靠性,实验结果表明,相比其它几种较为先进的人脸识别算法,本文算法取得了更强的识别鲁棒性。     

Structural similarity classifier for facial expression recognition

A novel Gabor filter structural similarity algorithm (GFSSIM) is proposed for facial expression recognition (FER) on noisy images. Low-resolution facial images with low SNRs are specifically dealt with FER system. The features are extracted using 40 Gabor filters, and a feature subset is selected for classification. The test image is classified based on proposed GFSSIM algorithm. The experimental results show that the recognition rate for heavily deteriorated images outperforms the conventional classifier method. In addition, the proposed method is more efficient from the computational complexity point of view.